It turns out this client is violating the HTTP spec by setting the "Transfer-Encoding: Chunked"
as well as "Content-Length: 0"
Here are the client headers:
POST /ee4/live.isml/Streams(Encoder1) HTTP/1.1
Transfer-Encoding: Chunked
User-Agent: ExpressionEncoder
Host: 10.11.1.29
Content-Length: 0
Connection: Keep-Alive
Cache-Control: no-cache
This is what HTTP 1.1 spec says (in section 4.4):
Messages MUST NOT include both a Content-Length header field and a non-identity transfer-coding. If the message does include
a non- identity transfer-coding, the Content-Length MUST be ignored.
libmicrohttpd does the opposite of what the 4.4 section says if both headers are present. It only uses the content-length and ignores the
chunked encoding.
I patched libmicrohttpd with the attached patch that does the opposite. It ignores the content-length if chunked encoding is also specified.
And with that patch libmicrohttpd can be a publishing point for MS Expression Encoder.
What is your take on this?
-eivind
=> answer: follow the spec ;-)
Hi
We've been having some mysterious parameter loss of POST parameters in
OpenVAS's GSA. This only happens with IE8 and Chrome. We saw this with
libmicrohttpd 0.9.19 and 0.9.20.
The cause looks to be an error in libmicrohttpd. Patch to 0.9.20 to
resolve below.
In post_process_multipart in postprocessor.c the PP_Init state calls
find_boundary to find the first boundary. If there is junk before the
first boundary it just reads over the junk. However, it is also reading
over the actual boundary when there was too little data to determine
whether the next character is the start of the boundary.
In the error case Chrome seems to sends the POST request in multiple
writes. The first chunk includes a single "-" from the first boundary at
end of the headers. Thus libmicrohttpd has a partial boundary to deal
with.
I guess Chrome intends to send just the headers but gets the count wrong
due to sending the initial P of the POST on its own (all the browsers do
that for some reason). Firefox on the other hand sends the headers and the
body in a single write, so it always works.
Thanks, and thanks for libmicrohttpd!
Matt
On GNU based systems the tree related functions (tsearch, tfind, tdestroy)
are provided by the libc (with the interface specification in search.h). On
non-GNU systems this functionality may or may not be available. That's the
case for Android which ships its own, simplified, version of libc called
Bionic. Bionic does not contains neither search.h nor an implementation of
the above mentioned functions.
This patch adds detection for the presence of search.h and if the header file
is not found, it uses an internal version of search.h and functions tsearch,
tfind, and tdestroy. The internal version is based on the source code from
FreeBSD and is compiled if and only if the configure script did not find the
search.h header file.
-- Jan Janak
>>
I face an issue while handling big https posts with libmicrohttpd 0.9.22.
I'm using an "external" select based on MHD_get_fdset and MHD_run.
Let's assume 16K of encrypted data arrive on the socket.
When the data arrives, my select correctly returns and MHD_run is called.
Do_read will first call gnu_tls to decrypt the data.
Apparently gnu_tls reads the full 16K so starting here, from the point of
view of socket/select, there is no more data to read.
Do_read however presents a buffer of only 9K to gnu_tls to retrieve the data
(9K as the result of a try_grow_read_buffer call).
So gnu_tls returns 9K of clear data and keeps the rest 7K in its own buffer
I think.
Do_read handles the 9K and then MHD_run returns;
Since from the socket point of view no data is pending read, select will
then wait until it finally timeout.
Only after this timeout, MHD_run is run again and the remaining 7K are
correctly processed.
I found two dirty workarounds for this issue :
1/ increase the MDH_BUF_INC_SIZE to an arbitrarily large value so it will
always be greater than gnu_tls own buffers
2/ run multiple MHD_run after each select (let's say 10 times)
What do you think of this issue ?
What could be a cleaner way of handling this ?
<<
Approach: detect that we *might* be in this situation because TLS fills
the entire buffer we give to gnuTLS. If so, set timeout to 0 and read
again.
Christian Grothoff