mirror of
https://git.gnunet.org/libmicrohttpd.git
synced 2026-09-25 04:09:31 +03:00
279 lines
14 KiB
Plaintext
279 lines
14 KiB
Plaintext
Seed corpus for the src/fuzz harnesses
|
|
======================================
|
|
|
|
`<harness>-NN.bin` is the built-in seed corpus of that harness, dumped
|
|
to disk. Regenerate at any time with
|
|
|
|
./fuzz_<harness> --write-corpus=<this directory>
|
|
|
|
or, from the build tree,
|
|
|
|
make -C src/fuzz refresh-corpus
|
|
|
|
Replay everything (this is what a CI regression run should do):
|
|
|
|
make -C src/fuzz check-corpus
|
|
# or, per harness:
|
|
./fuzz_request --corpus-dir=src/fuzz/corpus
|
|
|
|
Files belonging to another harness are simply uninteresting inputs for
|
|
the harness that reads them, so pointing every harness at the whole
|
|
directory is fine and gives some extra cross-pollination.
|
|
|
|
The set is kept minimal: every seed here adds coverage that no other
|
|
seed provides. After changing a seed table, re-check that with
|
|
|
|
<target> -merge=1 <empty dir> <this directory>
|
|
|
|
from an OSS-Fuzz style build (contrib/oss-fuzz/build.sh); a seed the
|
|
merge does not copy across is redundant and should be deleted from the
|
|
table rather than left in the corpus. Note that fuzz_request is
|
|
mildly non-deterministic -- MHD timestamps its nonces and its
|
|
connection timeouts -- so a single merge decision worth ~0.1% of the
|
|
regions is noise; only drop a seed that several independent merges
|
|
agree on.
|
|
|
|
|
|
fuzz_request seeds
|
|
------------------
|
|
|
|
00 content-length-body body oracle, Content-Length
|
|
01 chunked-with-extensions body oracle, chunk extensions
|
|
-> chunk-extension CRLF bug
|
|
02 chunked-split body oracle, chunk boundary split
|
|
across two send() calls
|
|
03 small-pool-trailing-query-arg 128..512 byte connection pool, no
|
|
header lines, "?novalue"
|
|
-> read-buffer shift-back bug
|
|
04 small-pool-trailing-query-arg-2 same with "?a=1&b"
|
|
05 digest-unknown-algorithm algorithm=BOGUS
|
|
-> MHD_DIGEST_AUTH_ALGO3_INVALID
|
|
MHD_PANIC()
|
|
06 digest-overlong-response 401 challenge, then a replay on a
|
|
fresh connection with the
|
|
harvested nonce and a 128 hex
|
|
digit response=
|
|
-> stack buffer overflow
|
|
07 digest-userhash userhash=true with a 128 char
|
|
username
|
|
08 basic-auth Authorization: Basic
|
|
09 multipart-post chunked multipart/form-data
|
|
10 urlencoded-post application/x-www-form-urlencoded
|
|
11 folded-header obs-fold continuation line
|
|
12 pipelined two requests in one segment
|
|
|
|
Seeds 00-12 leave the API-selection block (bytes 4-9, see section 2.2.1
|
|
of ../README) zero and only drive the request parser. Seeds 13-39 each
|
|
switch on one area of it; without them those entry points are reachable
|
|
only by guessing six configuration bytes.
|
|
|
|
13 ext-callback-chunked MHD_SIZE_UNKNOWN content reader,
|
|
response headers and a trailer
|
|
14 ext-callback-error known-length reader that fails
|
|
part way through
|
|
15 ext-fd-response MHD_create_response_from_fd()
|
|
16 ext-fd-at-offset-response ..._from_fd_at_offset()
|
|
17 ext-pipe-response ..._from_pipe()
|
|
18 ext-iovec-response ..._from_iovec()
|
|
19 ext-empty-response-hdrapi ..._empty() plus get/del header
|
|
and MHD_set_response_options()
|
|
20 ext-digest-check-digest3 pre-computed userdigest path
|
|
21 ext-digest-v1-wrappers MHD_digest_auth_check()
|
|
22 ext-digest-request-info MHD_digest_auth_get_request_info3()
|
|
23 ext-external-event-loop MHD_get_fdset2()/run_from_select2()
|
|
plus the timeout accessors
|
|
24 ext-external-event-loop-v1 the v1 fdset/run_from_select pair
|
|
25 ext-connection-api every introspection accessor
|
|
26 ext-suspend-resume suspend, resume from the pump loop
|
|
27 ext-suspend-two-connections two connections parked at once
|
|
28 ext-upgrade 101 + MHD_upgrade_action()
|
|
29 ext-buffer-persistent buffer/MHD_RESPMEM_PERSISTENT
|
|
30 ext-buffer-free-callback ..._with_free_callback()
|
|
31 ext-from-data MHD_create_response_from_data()
|
|
32 ext-fd64-response ..._from_fd64()
|
|
33 ext-basic-auth-v1 MHD_basic_auth_get_username_password()
|
|
34 ext-basic-auth-challenge MHD_queue_basic_auth_fail_response()
|
|
35 ext-basic-auth-challenge-utf8 ..._response3() with UTF-8 charset
|
|
36 ext-digest-check-digest-v1 MHD_digest_auth_check_digest()
|
|
37 ext-digest-check-digest2 ..._digest2() (never with
|
|
MHD_DIGEST_ALG_AUTO, see K8)
|
|
38 ext-digest-get-username-v1 MHD_digest_auth_get_username()
|
|
39 ext-digest-get-username3 ..._username3()
|
|
|
|
fuzz_str seeds
|
|
--------------
|
|
|
|
Cover MHD_hex_to_bin (including the 128 character input that
|
|
digestauth.c used to allow into a 32 byte buffer), MHD_bin_to_hex[_z],
|
|
the percent-decoders (strict, lenient, in place), MHD_str_unquote,
|
|
MHD_str_quote, MHD_base64_to_bin_n, MHD_str[x]_to_uint64_n_ and the
|
|
token helpers.
|
|
|
|
fuzz_auth_header seeds
|
|
----------------------
|
|
|
|
Well-formed and broken Digest parameter lists (unknown algorithm,
|
|
extended `username*` notation, unterminated quoted strings, empty
|
|
parameter list, over-long values) plus Basic `token68` variants, and
|
|
-- with bit 0x02 of byte 0 -- the connection-less digest helpers at
|
|
several algorithms and several deliberately undersized output
|
|
buffers.
|
|
|
|
fuzz_postprocessor seeds
|
|
------------------------
|
|
|
|
urlencoded bodies with broken percent escapes, multipart bodies with
|
|
ordinary, degenerate ("-") and quote-containing boundaries, a
|
|
multipart Content-Type without a boundary, and a non-POST
|
|
Content-Type.
|
|
|
|
|
|
distilled/
|
|
----------
|
|
|
|
The edge-minimal residue of a fuzzing campaign: 2467 inputs, 222 KB,
|
|
named `<harness>-dNNN.bin` and routed by that prefix exactly like the
|
|
seeds above.
|
|
|
|
Unlike `<harness>-NN.bin` these are *not* generated from a built-in
|
|
table, so `refresh-corpus` neither writes nor clobbers them; and unlike
|
|
`known-findings/` they are not hand-edited and carry no individual
|
|
meaning. Do not document them one by one -- the set is only ever
|
|
regenerated wholesale.
|
|
|
|
Provenance: an 8 hour, 12 core, 1.62 billion execution campaign
|
|
(ASan+UBSan, libFuzzer, hourly restarts) starting from the seed corpus
|
|
above. The campaign corpus was 18 226 inputs / 12 MB, which is too much
|
|
to carry in a source tree, so it was reduced by greedy set cover over
|
|
*edges* rather than by `-merge=1`, which minimises for features and
|
|
keeps roughly seven times as many files:
|
|
|
|
<target> -merge=1 -merge_control_file=mcf.txt <empty dir> <corpus>
|
|
|
|
writes a `COV` line per input, and the cover is computed from those
|
|
offline. Because these harnesses are mildly non-deterministic, the
|
|
cover is built only from edges that two independent merges agree an
|
|
input reproduces -- the same "several independent merges" rule stated at
|
|
the top of this file.
|
|
|
|
**An engine-produced corpus is not pristine, and this directory has to
|
|
be filtered before it can be committed.** `fuzz_request` inputs carry a
|
|
ground-truth body declaration in their `op == 1` segment, and the body
|
|
oracle is armed only for a `fuzz_pristine` input precisely because
|
|
"random mutations invalidate such declarations" (`fuzz_common.h`).
|
|
libFuzzer mutates that segment independently of the wire segments, so a
|
|
campaign corpus is full of inputs whose declaration no longer describes
|
|
their own request -- 383 of 6884 here, 5.6%. Under libFuzzer that is
|
|
harmless: `-DFUZZ_NO_MAIN` compiles out every assignment to
|
|
`fuzz_pristine`, so the oracle never arms and the campaign never
|
|
notices. But `--corpus-dir` sets `fuzz_pristine = 1` for *every* file it
|
|
reads, so replaying such an input reports a "request body desync"
|
|
against a declaration that was never true -- a false positive, and one
|
|
that looks exactly like a request-smuggling finding.
|
|
|
|
Committing the raw cover therefore broke `make check-corpus` with 42
|
|
such reports. The fix is to sweep every candidate with
|
|
`--file=` first and exclude the ones that fire *before* computing the
|
|
cover, which is what produced the set in this directory; all 2467 files
|
|
replay clean through all eight harnesses. Anyone replaying a downloaded
|
|
ClusterFuzz corpus locally will hit the same thing, and it is not a bug
|
|
in MHD.
|
|
|
|
The set reaches ~99% of the campaign's edges, not 100%: 3763/3798 for
|
|
fuzz_request, and exactly 100% for the four direct-API harnesses, which
|
|
are deterministic. Two thirds of that shortfall is the exclusion above
|
|
-- 64 edges were reached only by inputs with a stale declaration -- and
|
|
the rest is edges no single input reproduces reliably. Forcing the
|
|
latter in by adding every input that covers a rare edge was tried and
|
|
moved the number by one, so that part is inherent to minimising rather
|
|
than a fixable omission. Trading 1% of edges for 86% fewer files is the
|
|
intended bargain. Do not chase it.
|
|
|
|
To regenerate after a long campaign: keep the campaign corpus, sweep it
|
|
for oracle-firing inputs, run two independent merges over what is left,
|
|
recompute the cover, and replace this directory wholesale. Check the
|
|
result the same way it was checked here -- replay both the campaign
|
|
corpus and the distilled set with `-runs=0` three times each and compare
|
|
the `INITED cov:` figures, because a single measurement of either is
|
|
worth a few edges of noise -- and finish with `make check-corpus`.
|
|
|
|
|
|
known-findings/
|
|
---------------
|
|
|
|
Reproducers for the issues that this suite found in MHD itself; see
|
|
section 6 of ../README, which records the status of each. All of them
|
|
are fixed on master, so all of them replay clean, and `check-corpus`
|
|
replays this directory too.
|
|
|
|
A file is named `K<n>-<harness>-<what>.bin` for the harness that found
|
|
it; the ones without a harness in the name predate that convention and
|
|
are all fuzz_request inputs. `contrib/oss-fuzz/make_seed_corpus.sh`
|
|
routes each into the seed corpus of its own harness on that basis.
|
|
|
|
They are kept as a separate, explicitly named set rather than being
|
|
folded into the main corpus because the files there are *generated*: a
|
|
hand-written input would be clobbered or renumbered by the next
|
|
`refresh-corpus`. Replay one with
|
|
|
|
./fuzz_request --file=src/fuzz/corpus/known-findings/K1-digest-empty-realm.bin
|
|
|
|
K1-digest-empty-realm.bin digestauth.c is_param_equal()
|
|
mhd_assert (0 != param->value.len)
|
|
K2-chunkext-no-space.bin connection.c handle_recv_no_space()
|
|
K3-chunkext-stop-with-error.bin connection.c
|
|
transmit_error_response_len()
|
|
K4a-wsp-first-header.bin connection.c get_req_header(),
|
|
CLIENT_DISCIPLINE_LVL <= -1
|
|
K4b-empty-header-name.bin connection.c get_req_header(),
|
|
CLIENT_DISCIPLINE_LVL <= -2
|
|
K5-bare-cr-keep.bin connection.c get_req_header(),
|
|
CLIENT_DISCIPLINE_LVL == -3
|
|
K6-nonce-length-collision.bin digestauth.c check_nonce_nc()
|
|
mhd_assert (0 == nn->nonce[noncelen])
|
|
K7-upgrade-after-must-close.bin connection.c build_header_response()
|
|
mhd_assert (upgrade -> MUST_UPGRADE);
|
|
a single request carrying both
|
|
Content-Length and chunked
|
|
Transfer-Encoding, then an upgrade
|
|
K9-fuzz_postprocessor- postprocessor.c post_process_multipart()
|
|
nested-boundary-leak.bin leaked the previous nested boundary
|
|
on every extra nested
|
|
multipart/mixed part
|
|
K14-fuzz_eventloop- daemon.c call_handlers() asserted a
|
|
force-close-not-closed.bin post-condition that
|
|
MHD_connection_handle_read() does
|
|
not guarantee when the read buffer
|
|
is full
|
|
*** still open, see ../../../patches ***
|
|
Only fires on an --enable-asserts
|
|
*ASan* build: the redzones change
|
|
the pool arithmetic that reaches it.
|
|
K15-fuzz_eventloop- daemon.c close_all_connections()
|
|
stop-with-queued-connection.bin asserted that only a daemon with an
|
|
internal polling thread can have
|
|
connections queued by
|
|
MHD_add_connection(), while the code
|
|
that fills that list queues for any
|
|
thread-safe daemon. An external
|
|
event loop that adds a connection and
|
|
stops before the next MHD_run()
|
|
aborts on an --enable-asserts build.
|
|
|
|
These files are *not* regenerated by `--write-corpus`; they are edited by
|
|
hand. When the input format changes they have to be migrated, and the
|
|
migration has to be checked -- replay each one with `--verbose` before
|
|
and after and confirm the daemon, handler, body and challenge counts are
|
|
unchanged. The last such change made the API-selection block
|
|
unconditional, which is six zero bytes inserted at offset 4 for K1-K6.
|
|
|
|
K8 has no reproducer here: its trigger is an argument the application
|
|
chooses, not network input, so the harness never performs it. See
|
|
section 6 of ../README for how to reach it by hand.
|
|
|
|
A reproducer added while its finding is still open will make
|
|
`make check-corpus` fail, which is intended -- that is what a regression
|
|
test for an unfixed bug does. `contrib/oss-fuzz/make_seed_corpus.sh`
|
|
skips such a file so that ClusterFuzz does not spend every run
|
|
rediscovering it; the rule it uses is whether `patches/$ID.diff` exists.
|