I think this is going too technical for the -users list, so I set Folowup-To clamav-devel...
There _are_ current issues with thread-based clamd (and thread-based software in general, as everything (memory, fd's etc)is shared between threads:
No wonder there are issues with threading in clamd. When I pointed the author at "Programming with Posix Threads" he responded that he had no time to read books... So I guess the problem could also be a bit related to not fully understanding the topic ;-)
Basically you need to keep track of a seperate stack containing thread cleanup routines so you will actually loose performance doing that (for each malloc/free, open/close etc).
I already fixed a number of memleaks and possible segfaults on the 0.60 CVS tree, but there were some issues left (see below)...
I just can't comment that, as I don't use sendmail nor milter, and even hadn't a look at the milter code.
- some users (especially milter users) report clamd "freezing", effectively stopping mail queueing
This situation can be detected and dealt wit if the communication between the milter and clamd uses a timeout for reads and writes (setsockopt()). The clamav-milter, however, does not set timeouts on the sockets and relies on the libmilter timeouts. In the case of failure only sendmails default behaviour is available, which normally means that mail queuing is stopped.
There is third-party software which can (optionally) deliver mail even if clamd is defunct. If the timeout expires due clamd "gone mad" it marks messages as unchecked and delivers them the usual way. As a side effect, the situation is logged and can thus be detected.
As mentioned above, cleaning up resources is definitely harder in a threaded environment, where in a process-based world, you can just exit(). Most of the memleaks I left off were one-timers (e.g. configuration data), or stuff related to thread cleanup (as I'm normally not into thread programming). After patching, there were no more memleaks in "normal" operation, which meant recursively scanning all my harddisks.- resource cleanup in clamd is not optimal (especially on internal failures, where fd's are not closed and some memory is not free()'d)
This is not related to using threads or not. It's just bad style and needs correction anyway IMHO.
I haven't looked at the code, but I believe new processes are spawned just like new threads before (on startup, timeout or fatal error condition). But I'll look for that.
I think it's a great leap toward real production quality for clamd. A
For sure. It is always good to have the choice. I quess that once the threading problems are solved, the threaded solution will be much more performant. On a busy server it will be problematic to spawn a process for every mail message. That will lead to other ressource starvation problems.
I use it in production environments at a number of clients too. One of it being an (albeit small) ISP. But since I use it with amavisd-new, I can only comment on the clamav core, not the mbox or archive code. Following the posts on the ML, that code is much much better in 0.65, but I doubt it's completely bug free.
IMHO ClamAV already has production quality. At least we use it on some production servers and since the release of 0.65 there was not a single problem with the software. The team really made a big, big effort between 0.60 and 0.65.
- mb
As I have a bit spare time again, I'll continue my work on clamav this weekend. Perhaps I'll now also give clamav-milter a try ;-)
Thomas
------------------------------------------------------- This SF.net email is sponsored by: SF.net Giveback Program. Does SourceForge.net help you be more productive? Does it help you create better code? SHARE THE LOVE, and help us help YOU! Click Here: http://sourceforge.net/donate/ _______________________________________________ Clamav-users mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/clamav-users
