[Pymilter] really long processing

[Eric S. Johansson]

Stuart D. Gathman wrote:

>>which raises the question of how do you detect if multithreaded is active?
> 
> 
> libmilter requires threading.  If you are coding a milter, then
> threading is active.

I apologize for being totally blond here but I do need to understand.

Then what I take from the past few messages is that if I'm writing a 
pymilter, it will use threads automatically.  I don't need to do 
anything.  It's also implied that if I want to control them, I should 
use the python threads module.  There are certain advantages to using a 
stackless environment with regards to threads.  Unfortunately, I am 
pretty much constrained to counting on ordinary cpython.

the reason I was asking about testing for threading was so that I would 
no whether or not use process resource locks within my code in 
conjunction with the file based locks.  It's probably harmless if I just 
add the resource locks.


>>also, if I am calling an external program, which I do in both filter and 
>>Stamper, will that kill threading (using popen and family)?
> 
> 
> No. Unless the standard library is broken - which is wasn't the last
> I checked.

fair enough.  I will need to verify this however.  I've been burned by 
too many assumptions of my own and others.


>>another question is if I am busy processing something externally, can 
>>milter throw up a "stop" flag and ask the mail server to queue the 
>>message and try again later (i.e. in 60 seconds) rather than doing the 
>>queuing in threads?
> 
> 
> If you compile sendmail and pymilter with _FFR_SMFI_PROGRESS, then
> you can call the progress() method to tell sendmail to reset its
> timeout and keep waiting.  This still ties up a thread (not a 
> problem with stackless - and usually not a problem in general since
> sendmail currently runs a separate sendmail process per milter thread).
> 
> Without SMFI_PROGRESS, you must configure the milter timeouts for the
> worst case.

since I can't control what users have for sendmail compiles (especially 
with emerges and rpms), I will need to just use the worst-case time 
figure which is Infinity.  I assume there is some way to not timeout.

> If you return TEMPFAIL, then the sender will retry later. 

that's possibly another option.  If I'm busy generating stamps and I 
need to generate another, just force a retry.  The retry what happen in 
conjunction with an ordinary queue scan?  is there a way to trigger a 
retry when done?  Is it possible to put messages on a separate queue for 
the milter?

obviously, I need to do more reading.  I wish they would come out with a 
new bat book documenting the stuff.

> If you compile sendmail and pymilter with _FFR_QUARANTINE, then
> you can use the quarantine(reason) method to tell sendmail to put a message on
> hold.  I have not played with QUARANTINE, and do not know how such
> a message gets put back into play.

that's probably tied in with the (relatively) new multiple queue 
structures.  if I figure out anything, I will let you know.

thanks again for answering all these questions

---eric


<!DSPAM:FC75CEFCDB309112500894692>