[Pymilter] really long processing

[Stuart D. Gathman]

On Tue, 11 May 2004, Eric S. Johansson wrote:

> Stuart D. Gathman wrote:
> 
> > pymilter is multi-threaded.  There can be thousands of mail connections
> > in progress - depending on memory and sendmail configuration.  I
> > routinely see hundreds.  By using the "stackless" python VM (which
> > uses linked lists for stack frames instead of allocating a fixed stack
> > area per thread), you can support hundreds of thousands of simultaneous
> > connections.
> 
> interesting.  Which version of Python is stackless?  I'm using 2.3 now 
> on one machine and 2.2.1 on another.  I've been heavily encouraged by 
> some of pythonista to just stick with 2.3.

Stackless is another implementation derived from the CPython reference
implementation: http://www.stackless.com/

> which raises the question of how do you detect if multithreaded is active?

libmilter requires threading.  If you are coding a milter, then
threading is active.

> also, if I am calling an external program, which I do in both filter and 
> Stamper, will that kill threading (using popen and family)?

No. Unless the standard library is broken - which is wasn't the last
I checked.

> another question is if I am busy processing something externally, can 
> milter throw up a "stop" flag and ask the mail server to queue the 
> message and try again later (i.e. in 60 seconds) rather than doing the 
> queuing in threads?

If you compile sendmail and pymilter with _FFR_SMFI_PROGRESS, then
you can call the progress() method to tell sendmail to reset its
timeout and keep waiting.  This still ties up a thread (not a 
problem with stackless - and usually not a problem in general since
sendmail currently runs a separate sendmail process per milter thread).

Without SMFI_PROGRESS, you must configure the milter timeouts for the
worst case.

If you return TEMPFAIL, then the sender will retry later. 

If you compile sendmail and pymilter with _FFR_QUARANTINE, then
you can use the quarantine(reason) method to tell sendmail to put a message on
hold.  I have not played with QUARANTINE, and do not know how such
a message gets put back into play.

-- 
	      Stuart D. Gathman <stuart at bmsi.com>
    Business Management Systems Inc.  Phone: 703 591-0911 Fax: 703 591-6154
"Confutatis maledictis, flamis acribus addictis" - background song for
a Microsoft sponsored "Where do you want to go from here?" commercial.