From joao at 7lan.net Tue Oct 4 16:35:17 2016 From: joao at 7lan.net (joao at 7lan.net) Date: Tue, 04 Oct 2016 20:35:17 +0000 Subject: [Pymilter] milter daemonize Message-ID: Hi list, I want to daemonize my pymilter based milter, but I 'm getting some issues using the "daemon" package. Do you guy have any suggestion to daemonize my code? Thanks. -------------- next part -------------- An HTML attachment was scrubbed... URL: From stuart at gathman.org Wed Oct 5 14:33:04 2016 From: stuart at gathman.org (Stuart Gathman) Date: Wed, 5 Oct 2016 14:33:04 -0400 Subject: [Pymilter] milter daemonize In-Reply-To: References: Message-ID: On 10/04/2016 04:35 PM, joao at 7lan.net wrote: > Hi list, > > I want to daemonize my pymilter based milter, but I 'm getting some > issues using the "daemon" package. Do you guy have any suggestion to > daemonize my code? Here is the sysvinit script used for EL6: https://github.com/sdgathman/milter/blob/master/milter.rc Lines 51-53 are the actual call to daemonize. What specific issues are you having? I am now working on a package for EL7 (to test python3). -------------- next part -------------- An HTML attachment was scrubbed... URL: From tom at whyscream.net Sun Oct 16 16:22:07 2016 From: tom at whyscream.net (Tom Hendrikx) Date: Sun, 16 Oct 2016 22:22:07 +0200 Subject: [Pymilter] Python3: how to proceed Message-ID: <4a94ff84-301f-89e0-ef9c-73d92ef69dc8@whyscream.net> Hi, I wanted to invest some time in getting the python3 code working. This is what I tried so far: - created a virtualenv with python3.5 - cloned the pymilter master branch - patched extension using 'patch miltermodule.c milter.patch' - ran 'pip install -e .' All of this was succesful Then I created a minimal milter runner that runs the builtin logging-only milter: ---------------- import os import Milter def main(): os.umask(000) Milter.factory = Milter.Milter print("Python3 test milter starting") Milter.runmilter( "python3-test-milter", "/tmp/python3-test-milter.sock", 600) print("Python3 test milter halting") if __name__ == "__main__": main() ---------------- I run the milter, hook it up into postfix and open a telnet connection to the MTA. This immediately results a traceback: $ python run.py Python3 test milter starting Traceback (most recent call last): File "/opt/pymilter-py3/Milter/__init__.py", line 659, in negotiate_callback m._setctx(ctx) File "/opt/pymilter-py3/Milter/__init__.py", line 260, in _setctx ctx.setpriv(self) AttributeError: 'milterContext' object has no attribute 'setpriv' Traceback (most recent call last): File "/opt/pymilter-py3/Milter/__init__.py", line 676, in close_callback m = ctx.getpriv() AttributeError: 'milterContext' object has no attribute 'getpriv' This looks as if the extension code (which defines the milterContext object is not ready yet. Am I missing something? I'm a python developer and want to help getting this to work, but I never really learned C, or wrote any python extension code. KInd regards, Tom -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: OpenPGP digital signature URL: From stuart at gathman.org Sun Oct 16 21:46:54 2016 From: stuart at gathman.org (Stuart D. Gathman) Date: Sun, 16 Oct 2016 21:46:54 -0400 (EDT) Subject: [Pymilter] Python3: how to proceed In-Reply-To: <4a94ff84-301f-89e0-ef9c-73d92ef69dc8@whyscream.net> References: <4a94ff84-301f-89e0-ef9c-73d92ef69dc8@whyscream.net> Message-ID: On Sun, 16 Oct 2016, Tom Hendrikx wrote: > - created a virtualenv with python3.5 > - cloned the pymilter master branch > - patched extension using 'patch miltermodule.c milter.patch' > - ran 'pip install -e .' > > All of this was succesful > > Then I created a minimal milter runner that runs the builtin > logging-only milter: That should work. But I'm just getting started testing the py3 conversion. I'm testing with python3.4 in CentOS-7 and sendmail. I'm getting a systemd service going and get an error in xxfi_negotiate, gathering details now. If it can't find getpriv, then obviously I didn't follow the insns fully for the py3 context method table... -- Stuart D. Gathman "Confutatis maledictis, flamis acribus addictis" - background song for a Microsoft sponsored "Where do you want to go from here?" commercial. From tom at whyscream.net Mon Oct 17 03:30:55 2016 From: tom at whyscream.net (Tom Hendrikx) Date: Mon, 17 Oct 2016 09:30:55 +0200 Subject: [Pymilter] Python3: how to proceed In-Reply-To: References: <4a94ff84-301f-89e0-ef9c-73d92ef69dc8@whyscream.net> Message-ID: <68f23245-27c4-2093-8706-393f30dbf4fc@whyscream.net> On 17-10-16 03:46, Stuart D. Gathman wrote: > On Sun, 16 Oct 2016, Tom Hendrikx wrote: > >> - created a virtualenv with python3.5 >> - cloned the pymilter master branch >> - patched extension using 'patch miltermodule.c milter.patch' >> - ran 'pip install -e .' >> >> All of this was succesful >> >> Then I created a minimal milter runner that runs the builtin >> logging-only milter: > > That should work. But I'm just getting started testing > the py3 conversion. I'm testing with python3.4 in CentOS-7 and > sendmail. I'm getting a systemd service going and get an > error in xxfi_negotiate, gathering details now. Postfix is reporting also something with negotiate I think: Oct 16 21:56:30 alison postfix-in/smtpd[9674]: warning: milter unix:/tmp/python3-test-milter.sock: can't read SMFIC_OPTNEG reply packet header: Success Oct 16 21:56:30 alison postfix-in/smtpd[9674]: warning: milter unix:/tmp/python3-test-milter.sock: read error in initial handshake Oct 16 21:56:30 alison postfix-in/smtpd[9674]: NOQUEUE: milter-reject: CONNECT from unknown[2001:980:cfb1:1:c450:9e2b:a0b0:7196]: 451 4.7.1 Service unavailable - try again later; proto=SMTP It seems we're stuck on the same error ;/ > > If it can't find getpriv, then obviously I didn't follow the insns fully > for the py3 context method table... > From stuart at gathman.org Mon Oct 17 20:03:41 2016 From: stuart at gathman.org (Stuart D. Gathman) Date: Mon, 17 Oct 2016 20:03:41 -0400 (EDT) Subject: [Pymilter] Python3: how to proceed In-Reply-To: <68f23245-27c4-2093-8706-393f30dbf4fc@whyscream.net> References: <4a94ff84-301f-89e0-ef9c-73d92ef69dc8@whyscream.net> <68f23245-27c4-2093-8706-393f30dbf4fc@whyscream.net> Message-ID: I pushed changes to milter.patch to master that get me a lot farther. Apparently, you do need to call PyType_Ready() in python3. Log from sample.py (which comes with pymilter): 2016Oct17 16:37:51 [1] connect from [192.168.11.42] at ('192.168.11.42', 40500) 2016Oct17 16:39:03 [1] hello from joy.gathman.org 2016Oct17 16:39:22 [1] mail from () 2016Oct17 16:39:54 [1] rcpt to () 2016Oct17 16:40:19 [1] Subject: test 1 At which point: Oct 17 16:40:19 c7.gathman.org sendmail[9700]: u9HKbpcL009700: from=, size=22, class=0, nrcpts=1, msgid=<201610172039.u9HKbpcL009700 at c7.gathman.org>, proto=SMTP, daemon=MTA, relay=[192.168.11.42] Oct 17 16:40:19 c7.gathman.org sendmail[9700]: u9HKbpcL009700: Milter: data, reject=451 4.3.0 pymilter: untrapped exception in pythonfilter Oct 17 16:40:19 c7.gathman.org sendmail[9700]: u9HKbpcL009700: to=, delay=00:00:25, pri=30022, stat=pymilter: untrapped exception in pythonfilter -- Stuart D. Gathman "Confutatis maledictis, flamis acribus addictis" - background song for a Microsoft sponsored "Where do you want to go from here?" commercial. From stuart at gathman.org Mon Oct 17 20:58:44 2016 From: stuart at gathman.org (Stuart D. Gathman) Date: Mon, 17 Oct 2016 20:58:44 -0400 (EDT) Subject: [Pymilter] Python3: how to proceed In-Reply-To: <68f23245-27c4-2093-8706-393f30dbf4fc@whyscream.net> References: <4a94ff84-301f-89e0-ef9c-73d92ef69dc8@whyscream.net> <68f23245-27c4-2093-8706-393f30dbf4fc@whyscream.net> Message-ID: The sample.py milter is working in python3 now. Here are some bytes/str decisions that have to be made moving to python3. The connect callback gets a hostname - this is still a str. What do you think? Make that bytes also? The helo callback gets whatever the SMTP client sends for hostname. Should probably be bytes. The header callback is getting str. I'm thinking that header should also get bytes, although that will mean adding some code changes for existing milters (e.g. 'foo' -> b'foo'). The header name and value passed to header callback are not decoded - although milters are always free to depunycode for instance. The body callback gets passed bytes now. The unknown callback (commands MTA doesn't recognize) is still a str. Change to bytes? milter.Context.getsymval currently returns str. The MTA is actually passing null terminated bytes. Change that? -- Stuart D. Gathman "Confutatis maledictis, flamis acribus addictis" - background song for a Microsoft sponsored "Where do you want to go from here?" commercial. From tom at whyscream.net Wed Oct 19 16:37:22 2016 From: tom at whyscream.net (Tom Hendrikx) Date: Wed, 19 Oct 2016 22:37:22 +0200 Subject: [Pymilter] Python3: how to proceed In-Reply-To: References: <4a94ff84-301f-89e0-ef9c-73d92ef69dc8@whyscream.net> <68f23245-27c4-2093-8706-393f30dbf4fc@whyscream.net> Message-ID: On 18-10-16 02:58, Stuart D. Gathman wrote: > The sample.py milter is working in python3 now. Here are > some bytes/str decisions that have to be made moving to python3. > > The connect callback gets a hostname - this is still a str. What do you > think? Make that bytes also? > > The helo callback gets whatever the SMTP client sends for hostname. > Should probably be bytes. > > The header callback is getting str. I'm thinking that header should > also get bytes, although that will mean adding some code changes for > existing milters (e.g. 'foo' -> b'foo'). The header name and value > passed to header callback are not decoded - although milters are always > free to depunycode for instance. > > The body callback gets passed bytes now. > > The unknown callback (commands MTA doesn't recognize) is still a str. > Change to bytes? > > milter.Context.getsymval currently returns str. The MTA is actually > passing null terminated bytes. Change that? > I think most raw data interfaces (sockets etc) in Python3 expose bytes to the programmer, so giving bytes back everywhere seems logical. This will cause some work for milter maintainers, but migrating code to python3 was never 'free'. And being consistent in all data returned by pymilter methods makes things easier, for both new implementations and migrating existing ones. Kind regards, Tom -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: OpenPGP digital signature URL: