From stuart at bmsi.com Fri Jan 11 15:06:41 2013 From: stuart at bmsi.com (Stuart D Gathman) Date: Fri, 11 Jan 2013 15:06:41 -0500 Subject: [Pymilter] custom reply text after quarantine? In-Reply-To: <50D74966.9000708@whyscream.net> References: <50D74966.9000708@whyscream.net> Message-ID: <50F070D1.2070702@bmsi.com> On 12/23/2012 01:11 PM, Tom Hendrikx expounded in part: > based on this [1] message, I wanted to try reporting something like this > too: > > self.quarantine('Message is {} according to milter'.format(result)) > self.setreply('250', '5.7.1', 'Message is quarantined') > return Milter.ACCEPT > > This triggers a traceback ending in: > File "/usr/lib/python2.7/dist-packages/Milter/__init__.py", line 374, > in setreply > return self._ctx.setreply(rcode,xcode,msg,*ml) > milter.error: cannot set reply > > Is there any way to accomplished the above? You cannot set a 5.7.1 extended status with a 250 status. If you still have problems, what callback are you doing the above in? From tom at whyscream.net Sat Jan 12 16:38:19 2013 From: tom at whyscream.net (Tom Hendrikx) Date: Sat, 12 Jan 2013 22:38:19 +0100 Subject: [Pymilter] custom reply text after quarantine? In-Reply-To: <50F070D1.2070702@bmsi.com> References: <50D74966.9000708@whyscream.net> <50F070D1.2070702@bmsi.com> Message-ID: <50F1D7CB.3060505@whyscream.net> On 11-01-13 21:06, Stuart D Gathman wrote: > On 12/23/2012 01:11 PM, Tom Hendrikx expounded in part: >> based on this [1] message, I wanted to try reporting something like this >> too: >> >> self.quarantine('Message is {} according to milter'.format(result)) >> self.setreply('250', '5.7.1', 'Message is quarantined') >> return Milter.ACCEPT >> >> This triggers a traceback ending in: >> File "/usr/lib/python2.7/dist-packages/Milter/__init__.py", line 374, >> in setreply >> return self._ctx.setreply(rcode,xcode,msg,*ml) >> milter.error: cannot set reply >> >> Is there any way to accomplished the above? > You cannot set a 5.7.1 extended status with a 250 status. > If you still have problems, what callback are you doing the above in? I want to do this during eom() Changing the esmtp code from 5.7.1. to 2.5.0 gives me the same error. -- Tom From stuart at bmsi.com Sat Jan 12 19:01:59 2013 From: stuart at bmsi.com (Stuart D Gathman) Date: Sat, 12 Jan 2013 19:01:59 -0500 (EST) Subject: [Pymilter] custom reply text after quarantine? In-Reply-To: <50F1D7CB.3060505@whyscream.net> References: <50D74966.9000708@whyscream.net> <50F070D1.2070702@bmsi.com> <50F1D7CB.3060505@whyscream.net> Message-ID: On Jan 12, Tom Hendrikx transmitted in part: >>> Is there any way to accomplished the above? >> You cannot set a 5.7.1 extended status with a 250 status. >> If you still have problems, what callback are you doing the above in? > > I want to do this during eom() > Changing the esmtp code from 5.7.1. to 2.5.0 gives me the same error. It is the 250 that is the problem. Read the C API page: https://www.milter.org/developers/api/smfi_setreply rcode The three-digit (RFC 821/2821) SMTP reply code, as a null-terminated string. rcode cannot be NULL, and must be a valid 4XX or 5XX reply code. Here is an example of setreply in a production milter: def eom(self): if self.is_bounce and len(self.recipients) > 1: self.log("REJECT: DSN to multiple recipients") self.setreply('550','5.7.1', 'DSN to multiple recipients') return Milter.REJECT ... From stuart at bmsi.com Sat Jan 12 23:37:16 2013 From: stuart at bmsi.com (Stuart D Gathman) Date: Sat, 12 Jan 2013 23:37:16 -0500 Subject: [Pymilter] python3 support? In-Reply-To: <50A95D7E.2030100@whyscream.net> References: <509D9130.2080000@whyscream.net> <50A95D7E.2030100@whyscream.net> Message-ID: <50F239FC.1030701@bmsi.com> On 11/18/2012 05:13 PM, Tom Hendrikx wrote: > Nice! I checked it out today, but I see you only got to creating the > branch, not adding/changing any code in it. I'll check back once in a > while. If there's anything I can help with, please let me know. PS > Sorry for the late reply, I was quite busy this week. I updated miltermodule.c with my understanding of the C API changes, and it compiles. Now for the python code! From tom at whyscream.net Sun Jan 13 06:48:03 2013 From: tom at whyscream.net (Tom Hendrikx) Date: Sun, 13 Jan 2013 12:48:03 +0100 Subject: [Pymilter] custom reply text after quarantine? In-Reply-To: References: <50D74966.9000708@whyscream.net> <50F070D1.2070702@bmsi.com> <50F1D7CB.3060505@whyscream.net> Message-ID: <50F29EF3.7040905@whyscream.net> On 13-01-13 01:01, Stuart D Gathman wrote: > On Jan 12, Tom Hendrikx transmitted in part: > >>>> Is there any way to accomplished the above? >>> You cannot set a 5.7.1 extended status with a 250 status. >>> If you still have problems, what callback are you doing the above in? >> >> I want to do this during eom() >> Changing the esmtp code from 5.7.1. to 2.5.0 gives me the same error. > > It is the 250 that is the problem. Read the C API page: > > https://www.milter.org/developers/api/smfi_setreply > > rcode The three-digit (RFC 821/2821) SMTP reply code, as a > null-terminated string. rcode cannot be NULL, and must be a valid 4XX or > 5XX reply code. > > Here is an example of setreply in a production milter: > > def eom(self): > if self.is_bounce and len(self.recipients) > 1: > self.log("REJECT: DSN to multiple recipients") > self.setreply('550','5.7.1', 'DSN to multiple recipients') > return Milter.REJECT > ... I already use the 550 variant with succes, but I also wanted a custom reply text for accepted messages, but I guess that's not possible through the milter api. Thanks for the research :) -- Tom From tom at whyscream.net Sun Jan 13 07:09:17 2013 From: tom at whyscream.net (Tom Hendrikx) Date: Sun, 13 Jan 2013 13:09:17 +0100 Subject: [Pymilter] poll() returned exception for socket, abort Message-ID: <50F2A3ED.6050005@whyscream.net> Hi, I'm currently adding daemon support to my milter, using a simplified version of the routine as documented in PEP3143 and implemented in http://pypi.python.org/pypi/python-daemon/ For some reason, the milter aborts after processing the single connection. The first connection is handled properly, and a few seconds after the disconnect is completed, syslog reports: "python: MyMilter: poll() returned exception for socket, abort", and the milter process is gone. I narrowed it down to closing all file descriptors. When using this piece of code: # Close file descriptors save_fds = [sys.stderr.fileno()] for fd in reversed(range(2048)): if fd not in save_fds: try: os.close(fd) print('Closed fd {}'.format(fd)) except OSError, err: if err.errno == errno.EBADF: # File descriptor was not open pass This works, as long as there is at least one fd listed in save_fds. It doesn't matter whether it's stdout, stderr or stdin: anyone will do. As soon as I set 'safe_fds = [], and all fds are closed, the above error turns up and the milter exits after the first connection. Any ideas? I can't even find where this error message is generated... -- Tom From tom at whyscream.net Sun Jan 13 07:23:05 2013 From: tom at whyscream.net (Tom Hendrikx) Date: Sun, 13 Jan 2013 13:23:05 +0100 Subject: [Pymilter] poll() returned exception for socket, abort In-Reply-To: <50F2A3ED.6050005@whyscream.net> References: <50F2A3ED.6050005@whyscream.net> Message-ID: <50F2A729.4040800@whyscream.net> On 13-01-13 13:09, Tom Hendrikx wrote: > Hi, > > I'm currently adding daemon support to my milter, using a simplified > version of the routine as documented in PEP3143 and implemented in > http://pypi.python.org/pypi/python-daemon/ > > For some reason, the milter aborts after processing the single > connection. The first connection is handled properly, and a few seconds > after the disconnect is completed, syslog reports: "python: MyMilter: > poll() returned exception for socket, abort", and the milter process is > gone. > > I narrowed it down to closing all file descriptors. When using this > piece of code: > > # Close file descriptors > save_fds = [sys.stderr.fileno()] > for fd in reversed(range(2048)): > if fd not in save_fds: > try: > os.close(fd) > print('Closed fd {}'.format(fd)) > except OSError, err: > if err.errno == errno.EBADF: > # File descriptor was not open > pass > > This works, as long as there is at least one fd listed in save_fds. It > doesn't matter whether it's stdout, stderr or stdin: anyone will do. As > soon as I set 'safe_fds = [], and all fds are closed, the above error > turns up and the milter exits after the first connection. > > Any ideas? I can't even find where this error message is generated... > Err, I left the stderr fd open, and also disabled redirecting sys.stderr to os.devnull, and now I got a traceback in terminal stating "milter.error: out of thread resources". That gives room for more research, so continuing the search... From tom at whyscream.net Sun Jan 13 08:12:46 2013 From: tom at whyscream.net (Tom Hendrikx) Date: Sun, 13 Jan 2013 14:12:46 +0100 Subject: [Pymilter] python3 support? In-Reply-To: <50F239FC.1030701@bmsi.com> References: <509D9130.2080000@whyscream.net> <50A95D7E.2030100@whyscream.net> <50F239FC.1030701@bmsi.com> Message-ID: <50F2B2CE.8030706@whyscream.net> On 13-01-13 05:37, Stuart D Gathman wrote: > On 11/18/2012 05:13 PM, Tom Hendrikx wrote: >> Nice! I checked it out today, but I see you only got to creating the >> branch, not adding/changing any code in it. I'll check back once in a >> while. If there's anything I can help with, please let me know. PS >> Sorry for the late reply, I was quite busy this week. > > I updated miltermodule.c with my understanding of the C API changes, and > it compiles. Now for the python code! Great! Just did a cvs up, compiles without issues here. -- Tom From stuart at bmsi.com Sun Jan 13 15:10:54 2013 From: stuart at bmsi.com (Stuart D Gathman) Date: Sun, 13 Jan 2013 15:10:54 -0500 (EST) Subject: [Pymilter] poll() returned exception for socket, abort In-Reply-To: <50F2A729.4040800@whyscream.net> References: <50F2A3ED.6050005@whyscream.net> <50F2A729.4040800@whyscream.net> Message-ID: On Jan 13, Tom Hendrikx transmitted in part: >> I'm currently adding daemon support to my milter, using a simplified >> version of the routine as documented in PEP3143 and implemented in >> http://pypi.python.org/pypi/python-daemon/ >> >> For some reason, the milter aborts after processing the single >> connection. The first connection is handled properly, and a few seconds Can you post your main code? Things I can think of: o libmilter insists on handling signals, you probably don't want daemon to do so. Making sure you daemonize *before* making any milter calls (like register, main) would probably work. I would disable all signal stuff in daemon. o The milter setup (register, main) would again need to be after daemon. Closing the socket to sendmail that libmilter opened would be bad. o I can't think how you would manage it, other than call daemon from a callback, but it should not be called for every connection. o Maybe daemon is changing a signal that libmilter depends on? From tom at whyscream.net Sun Jan 13 16:48:39 2013 From: tom at whyscream.net (Tom Hendrikx) Date: Sun, 13 Jan 2013 22:48:39 +0100 Subject: [Pymilter] poll() returned exception for socket, abort In-Reply-To: References: <50F2A3ED.6050005@whyscream.net> <50F2A729.4040800@whyscream.net> Message-ID: <50F32BB7.7030704@whyscream.net> On 13-01-13 21:10, Stuart D Gathman wrote: > On Jan 13, Tom Hendrikx transmitted in part: > >>> I'm currently adding daemon support to my milter, using a simplified >>> version of the routine as documented in PEP3143 and implemented in >>> http://pypi.python.org/pypi/python-daemon/ >>> >>> For some reason, the milter aborts after processing the single >>> connection. The first connection is handled properly, and a few seconds > > Can you post your main code? Code is at github. Main startup routine: https://github.com/whyscream/pydspam/blob/master/dspam/milter.py#L258 And the daemon routine: https://github.com/whyscream/pydspam/blob/master/dspam/utils.py#L17 > Things I can think of: > > o libmilter insists on handling signals, you probably don't want > daemon to do so. Making sure you daemonize *before* making any > milter calls (like register, main) would probably work. I would > disable all signal stuff in daemon. > > o The milter setup (register, main) would again need to be after > daemon. Closing the socket to sendmail that libmilter opened would > be bad. I had a signal handler enabled for SIGTERM, removing it now (although I don't think that was the culprit). All daemon related stuff is done before calling Milter.runmilter() > > o I can't think how you would manage it, other than call daemon from a > callback, but it should not be called for every connection. > > o Maybe daemon is changing a signal that libmilter depends on? The code at github seems to work fine now, after I found out that the process keeps running when I only close stdin/stdout/stderr, and leave other fds alone. That's not what I wanted to do, but it seems to help somehow. -- Tom