[Pymilter] Forking pymilter?

[Dwayne Litzenberger]

On October 24, 2007 12:48:13 pm Stuart D. Gathman wrote:
> I found it easy enough to wrap non-thread-safe APIs with mutexes from
> the thread module.  I needed to do that with libdspam, for example.
> If it comes up a lot, an @synchronized decorator would
> be easy enough to do, if it isn't in 2.5 already (I'm still on 2.4).

python-ldap already wraps its API in a mutex.  That's useless, though, because 
when you make a call to, say, ldap.search_s ("_s" stands for synchronous), 
python-ldap acquires a library-wide lock.  Any other thread that tries to 
access the ldap library in the meantime (including to access other LDAP 
servers) blocks until the previous call completes.  At that point, we've done 
away with concurrency, which defeats the purpose of using threads in the 
first place.

I ended up spending a lot of time writing code to pass messages between the 
threads created by libmilter and a thread I created specifically to access 
python-ldap.  If I had been able to fork a server for each connection, I 
could have simply used ldap.search_s and been done with it.

libmilter demands a particular threading model that happens to be incompatible 
with OpenLDAP.  What's not clear to me is why the Sendmail people thought 
this was necessary.  IMHO, it would have been better to publish a document 
specifying how to communicate over the milter socket, and to provide 
automatic thread creation as an optional library feature.

> Keeping all your data in the connection object makes the actual milter
> code easy to keep threadsafe.

Which is fine, unless you want to reuse code that already exists for other 
purposes and therefore doesn't do that.

In any case, at least I can deal with this (albeit, unnecessary) complexity in 
Python rather than in C.  Thanks for that.

-- 
Dwayne Litzenberger, B.A.Sc.
Information Technology Analyst

Open Systems Canada Limited
1627 Broad Street
Regina, SK S4P1X3
Office: 306.359.6725
http://www.oscl.ca/