The MHD goes in a busy loop when it is configured with MHD_USE_POLL_INTERNALLY and a connection times out.
When the connection times out, the connection is closed at connection.c:2646, which sets connection->event_loop_info = MHD_EVENT_LOOP_INFO_CLEANUP.
When the loop info is set to MHD_EVENT_LOOP_INFO_CLEANUP, the main function of the thread loop, MHD_poll_all, never calls back the connection idle callback, which would have cleaned the connection and exit the loop.
I resolved the issue in my development code by adding pos->idle_handler (pos) at daemon.c:2477 in MHD_poll_all (SVN 35533).
The busy loop could be tested using a small enough connection timeout and netcat:
nc -v -w 100 <IP ADDRESS> <IP PORT>
CPU usage will reach 100% in one of the CPU and will remain at that level even when the netcat has closed its end of the connection.
Thanks,
Louis Benoit
I was trying to use MHD_suspend_connection() and MHD_resume_connection() to implement long polling on a request. I am using options MHD_USE_SELECT_INTERNALLY and MHD_USE_POLL options. I noticed the server thread went to 100% CPU after the first MHD_resume_connection() call. Looking at the code in MHD_poll_all() I can see that the file descriptor from daemon->wpipe[0] gets inserted into the list of file descriptors to poll but this file descriptor is never read so every time this function is called it returns immediately. The code works fine if I drop to using just MHD_USE_SELECT_INTERNALLY.
The patch below fixes the problem.
Regards,
Denis
I am using libmicrohttpd in a multithreaded application in the
thread-per-connection mode. In order to maintain statistics and such on
the various clients, the application should be notified about
connections being started and stopped. Something like the
MHD_OPTION_NOTIFY_COMPLETED for requests, but then for connections. As
far as I could find libmicrohttpd does not (yet) provide that functionality.
Attached is a patch that does add this functionality by registering a
connection notification callback function: MHD_OPTION_NOTIFY_CONNECTION
(inspired by the MHD_OPTION_NOTIFY_COMPLETED option).
Could this be included in libmicrohttpd?
=>
The patch was only working for multithreaded apps, so I adjusted it
to cover other threading models, fixed a merge issue and added the
ability to associate a void* with the callbacks (and obtain it via
MHD connection info). And I updated the manual & ChangeLog. Now
I think it can be included ;-).
Date: Wed, 29 Oct 2014 09:59:09 +0100
Subject: [PATCH 2/2] Add MHD_DAEMON_INFO_CURRENT_CONNECTIONS to
MHD_DaemonInfoType.
The MHD_DAEMON_INFO_CURRENT_CONNECTIONS returns number of current
connections handled by the daemon.
Useful after MHD_quiesce_daemon to find out whether all connections
have been served.
MHD_USE_DUAL_STACK in libmicrohttpd currently just *inhibits setting* the
IPV6_V6ONLY socket option, but per Microsoft's documentation
http://msdn.microsoft.com/en-us/library/windows/desktop/bb513665(v=vs.85).aspx
the default on Windows is that this is enabled, thus MHD_USE_DUAL_STACK will
not work (since it leaves the default). libmicrohttpd should probably just
unconditionally set IPV6_V6ONLY to the desired value when the option is
available.
diff --git a/src/microhttpd/daemon.c b/src/microhttpd/daemon.c
index 0a33b77..3cbf28e 100644
--- a/src/microhttpd/daemon.c
+++ b/src/microhttpd/daemon.c
@@ -3493,8 +3493,7 @@ MHD_start_daemon_va (unsigned int flags,
}
daemon->socket_fd = socket_fd;
- if ( (0 != (flags & MHD_USE_IPv6)) &&
- (MHD_USE_DUAL_STACK != (flags & MHD_USE_DUAL_STACK)) )
+ if (0 != (flags & MHD_USE_IPv6))
{
#ifdef IPPROTO_IPV6
#ifdef IPV6_V6ONLY
@@ -3503,10 +3502,11 @@ MHD_start_daemon_va (unsigned int flags,
and may also be missing on older POSIX systems; good luck if you have
any of those,
your IPv6 socket may then also bind against IPv4 anyway... */
#ifndef WINDOWS
- const int on = 1;
+ const int
#else
- const char on = 1;
+ const char
#endif
+ on = (MHD_USE_DUAL_STACK != (flags & MHD_USE_DUAL_STACK));
if ( (0 > SETSOCKOPT (socket_fd,
IPPROTO_IPV6, IPV6_V6ONLY,
&on, sizeof (on))) &&
Hi
We've been having some mysterious parameter loss of POST parameters in
OpenVAS's GSA. This only happens with IE8 and Chrome. We saw this with
libmicrohttpd 0.9.19 and 0.9.20.
The cause looks to be an error in libmicrohttpd. Patch to 0.9.20 to
resolve below.
In post_process_multipart in postprocessor.c the PP_Init state calls
find_boundary to find the first boundary. If there is junk before the
first boundary it just reads over the junk. However, it is also reading
over the actual boundary when there was too little data to determine
whether the next character is the start of the boundary.
In the error case Chrome seems to sends the POST request in multiple
writes. The first chunk includes a single "-" from the first boundary at
end of the headers. Thus libmicrohttpd has a partial boundary to deal
with.
I guess Chrome intends to send just the headers but gets the count wrong
due to sending the initial P of the POST on its own (all the browsers do
that for some reason). Firefox on the other hand sends the headers and the
body in a single write, so it always works.
Thanks, and thanks for libmicrohttpd!
Matt
On GNU based systems the tree related functions (tsearch, tfind, tdestroy)
are provided by the libc (with the interface specification in search.h). On
non-GNU systems this functionality may or may not be available. That's the
case for Android which ships its own, simplified, version of libc called
Bionic. Bionic does not contains neither search.h nor an implementation of
the above mentioned functions.
This patch adds detection for the presence of search.h and if the header file
is not found, it uses an internal version of search.h and functions tsearch,
tfind, and tdestroy. The internal version is based on the source code from
FreeBSD and is compiled if and only if the configure script did not find the
search.h header file.
-- Jan Janak
am I right in the assumption, that the documentation at
http://www.gnu.org/s/libmicrohttpd/tutorial.html#Supporting-basic-authentication
is newer than authorization_example.c in the src/examples/ subdirectory?
If so here is a patch to make authorization_example.c use the functions from
the tutorial instead of raw base64 strings.
Regards
Sven
-
Will Bryant <will.bryant@gmail.com>
To:
libmicrohttpd development and user mailinglist <libmicrohttpd@gnu.org>
Date:
Yesterday 03:59:23 PM
Attachments:
0001-use-a-pipe-to-signal-shutdown-to-select-and-poll.patch
I attach my current work. I have attempted to preserve and restore as much of the 'old' pipe shutdown code that was present in earlier versions of the library, but have removed the sections that aren't relevant now that there is a socket-by-socket client shutdown procedure.
On OS X, all the 'regular' tests pass. The two perf test programs both fail on a random subset of the tests with "Failed to bind to port 1081: Address already in use" (or the other ports that the perf tests use).
I initially thought that this was a problem with this patch, but after I couldn't find any problem with it I tried backporting the perf test programs to the older 0.9.7 version of libmicrohttpd, which is before the listening socket shutdown code went in, and it turned out that the older versions failed in the same way.
On OpenIndiana (oi_151a - but should behave the same as other SunOS/Solaris/OpenSolaris/Illumos kernels), the same problem occurs. (I haven't tried the backport there too.)
Linux is known to have slightly different behavior for the SOL_REUSEADDR option which might explain this, but it's quite possible that this is just a timing issue with the cleanup of the client threads in these perf test programs (if you wait for 1s between the tests then the problem goes away).
On OS X, using both debugging printfs and shelling out to lsof -itcp, I have confirmed that the listening socket has been closed successfully before the error occurs. So I am starting to suspect that it is not possible to bind to an address that was used as a server socket until all the client sockets have also been cleaned up - but that's just a theory at this point.
Anyway, as far as this patch goes, OS X looks a lot healthier - as healthy as it was in 0.9.7 (0.9.8 and 0.9.9 both failed to build entirely).
There is one further test failure on openindiana, which I believe is not related to this patch - all the same 'regular' tests pass except daemontest_get_response_cleanup, which dies due to an unhandled SIGPIPE when trying to write to the socket after killing the curl process:
Program received signal SIGPIPE, Broken pipe.
[Switching to Thread 2 (LWP 2)]
0xfed634f7 in __so_send () from /lib/libc.so.1
(gdb) bt
#0 0xfed634f7 in __so_send () from /lib/libc.so.1
#1 0xfed511d7 in _so_send () from /lib/libc.so.1
#2 0xfe7dcb26 in send () from /lib/libsocket.so.1
#3 0xfef56d29 in send_param_adapter (connection=0x20, other=0xfe99080f, i=0) at daemon.c:764
#4 0xfef54a0b in do_write (connection=0x8073ad0) at connection.c:1495
#5 0xfef54f6f in MHD_connection_handle_write (connection=0x8073ad0) at connection.c:1909
#6 0xfef57b3c in MHD_select (daemon=0x80737a8, may_block=0) at daemon.c:1367
#7 0xfef57ff4 in MHD_select_thread (cls=0x80737a8) at daemon.c:1615
#8 0xfed5f0f3 in _thrp_setup () from /lib/libc.so.1
#9 0xfed5f3a0 in ?? () from /lib/libc.so.1
#10 0xfe9a0240 in ?? ()
#11 0x00000000 in ?? ()
(gdb) info threads
* 4 Thread 2 (LWP 2) 0xfed634f7 in __so_send () from /lib/libc.so.1
3 LWP 2 0xfed634f7 in __so_send () from /lib/libc.so.1
2 Thread 1 (LWP 1) 0xfed63e87 in __waitid () from /lib/libc.so.1
1 LWP 1 0xfed63e87 in __waitid () from /lib/libc.so.1
(gdb) thread 2
[Switching to thread 2 (Thread 1 (LWP 1))]#0 0xfed63e87 in __waitid () from /lib/libc.so.1
(gdb) bt
#0 0xfed63e87 in __waitid () from /lib/libc.so.1
#1 0xfed515bd in waitid () from /lib/libc.so.1
#2 0xfed00035 in waitpid () from /lib/libc.so.1
#3 0x08051423 in kill_curl (pid=4479) at daemontest_get_response_cleanup.c:70
#4 0x0805157d in main (argc=1, argv=0x6b) at daemontest_get_response_cleanup.c:141
0001-use-a-pipe-to-signal-shutdown-to-select-and-poll.patch
From:
Eivind Sarto <ivan@espial.com>
To:
"libmicrohttpd@gnu.org" <libmicrohttpd@gnu.org>
Date:
Today 09:32:21 pm
Spam Status: Spamassassin 0% probability of being spam.
Full report:
Probability=No, score=-2.6 required=7.0 tests=BAYES_00 autolearn=ham version=3.2.5-tuminfo_1
There appears to be a bug in MHD_create_response_from_fd_at_offset().
Calling this function with anything other than a zero offset will cause wrong data
or no data (sendfile fails if length < 0).
If you use this call with any application that uses ranges, this bug will trigger.
In src/daemon/daemon.c: send_param_adapter()
.....
/* can use sendfile */
offset = (off_t) connection->response_write_position + connection->response->fd_off;
#ifdef BUGFIX
/* correct */
left = connection->response->total_size - connection->response_write_position;
#else
left = connection->response->total_size - offset;
#endif
if (left > SSIZE_MAX)
left = SSIZE_MAX; /* cap at return value limit */
ret = sendfile (connection->socket_fd,
fd,
&offset,
left);
-eivind
From:
Andreas Wehrmann <a.wehrmann@centersystems.com>
To:
libmicrohttpd@gnu.org
Date:
Today 08:58:43 am
Spam Status: Spamassassin 0% probability of being spam.
Full report:
Probability=No, score=-3.2 required=7.0 tests=AWL,BAYES_00 autolearn=ham version=3.2.5-tuminfo_1
Hello!
I wrote a little testpage that I deliver using libmicrohttpd using
digest authentication.
The testpage consists of four files (framed page + image file).
When I initially connected to the webserver via the browser it correctly
challenged me
for my credentials. However, after entering the username and password
the index file
got loaded but it happened that the browser then challenged me again for
each
additional file to be loaded.
Since this is very annoying I tried increasing the nonce table size to 3000
(was default) but it was no good.
I then dug a little deeper and found out, that the hash algorithm to
determine the index
for a given nonce always returned zero thus overwriting other nonces.
The offending line is at check_nonce_nc() in digestauth.c:313:
off = (off << 8) | (*np & (off >> 24));
whereas is should be:
off = (off << 8) | (*np ^ (off >> 24));
Since "off" is initialized with zero and an unsigned integer
a logical AND returns zero which is not right obviously.
After this fix, the server challenged me only once and I got "random"
indices.
I found the problem in libmicrohttpd 0.9.5.
Best regards,
Andreas Wehrmann
--
Dipl.-Ing. (FH) Andreas Wehrmann
Software Development
--------------------------------------------------------------
Center Communication Systems GmbH
A-1210 Wien, Ignaz-Köck-Straße 19
Sitz in Wien
FN 796 88p, Firmenbuchgericht Wien
www.centersystems.com
Tel.: +43 (0) 190 199 - 3616
Mobile: +43 (0) 664 884 75916
Fax: +43 (0) 190 199 - 2110
E-Mail: a.wehrmann@centersystems.com
From:
Erik Slagter <erik@slagter.name>
To:
Christian Grothoff <grothoff@net.in.tum.de>
CC:
libmicrohttpd@gnu.org
Date:
Yesterday 17:31:46
Spam Status: Spamassassin 0% probability of being spam.
Full report:
Probability=No, score=-2.6 required=7.0 tests=AWL,BAYES_00,DKIM_SIGNED, DKIM_VERIFIED autolearn=ham version=3.2.5-tuminfo_1
> >> I think I found & fixed the issue in SVN 12778. Please try SVN HEAD and
> > > report if it fixes the problem.
> > Okay, will do so. But first I'll be on vacation for two weeks.
> Sure.
The issue indeed has been solved. It's quite a pity the distribution's
packages lag behind by ages, so again I'll have to make my own packages
now.
I've run into another problem and also I think I have found the cause.
When a connection is closed by the peer, my application tends to
segfault.
GDB says:
Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7ffff6a6c710 (LWP 27005)]
0x00007ffff77a4db0 in pthread_mutex_lock () from /lib64/libpthread.so.0
Missing separate debuginfos, use: debuginfo-install glibc-2.11.2-1.x86_64 libgcc-4.4.4-10.fc12.x86_64 libid3tag-0.15.1b-9.fc12.x86_64 libstdc++-4.4.4-10.fc12.x86_64 zlib-1.2.3-23.fc12.x86_64
(gdb) where
#0 0x00007ffff77a4db0 in pthread_mutex_lock () from /lib64/libpthread.so.0
#1 0x00007ffff7bd97ce in MHD_destroy_response (response=0x7ffff7ed5010) at response.c:341
#2 0x00007ffff7bd760f in MHD_cleanup_connections (daemon=0x62eb60) at daemon.c:964
#3 0x00007ffff7bd8fbc in MHD_select_thread (cls=0x62eb60) at daemon.c:1205
#4 0x00007ffff77a2a3a in start_thread () from /lib64/libpthread.so.0
#5 0x00007ffff6d6077d in clone () from /lib64/libc.so.6
#6 0x0000000000000000 in ?? ()
It looks like pos->response is being "destroyed" twice. If I remove one of the instances, the segfault is gone.
Index: src/daemon/daemon.c
===================================================================
--- src/daemon/daemon.c (revision 12985)
+++ src/daemon/daemon.c (working copy)
@@ -952,7 +952,7 @@
abort();
}
}
- MHD_destroy_response (pos->response);
+ // MHD_destroy_response (pos->response);
MHD_pool_destroy (pos->pool);
#if HTTPS_SUPPORT
if (pos->tls_session != NULL)
From:
Gerrit Telkamp <g.telkamp@domologic.de>
To:
libmicrohttpd@gnu.org
Date:
Today 11:42:30
Attachments:
libmicrohttpd-windows.patch
It was a bit tricky to build libmicrohttpd (SVN trunk) for Windows.
I'm using the newest MSYS/MinGW environment (GCC 4.5.1).
I've optimized the autoconf feature, so this should be easier in future.
Please find enclosed my patches.
In detail:
1- libmicrohttpd needs the "PlibC" library under Windows (libplibc.a).
Methods of PlibC are referenced by src/daemon/daemon.c.
autoconf assumes that PlibC is installed on the Build environment and
does not check this.
If PlibC is not installed, the pthreads check (see "acinclude.m4") will
fail and stops with the wrong error message "Your system is not
supporting pthreads". This is very confusing.
The patch includes an additional check for PlibC, before pthreads is
used. The check is only done under cygwin and mingw environments.
Furthermore, an older version of plibc.h was in the include directory.
This gives some problems if you are using a newer version of the PlibC
library. So I propose to exclude src/include/plibc.h from the SVN
Files affected:
acinclude.m4 (modified)
configure.ac (modified)
src/include/plibc.h (deleted)
2- setsockopt() uses a different parameter type under Windows ("optval"
is const char). This caused an error when src/daemon/daemon.c was compiled.
The patch enclosed includes a different call of the setsockopt() on
Windows machines.
Important: IPV6_V6ONLY is not available on Windows machines < Windows 7!
So this parameter is "0". But there is a TODO comment, maybe someone has
an idea how to solve this on Windows XP machines.
Files affected:
src/daemon/daemon.c (modified)
I have tested the patch on a MinGW / MSYS environment only.
Best regards
Gerrit.
libmicrohttpd-windows.patch
Evgeny Grin (Karlson2k)
Christian Grothoff
silvioprog
Sree Harsha Totakura