From daniele at grinta.net  Thu Nov  3 13:33:19 2011
From: daniele at grinta.net (Daniele Nicolodi)
Date: Thu, 03 Nov 2011 18:33:19 +0100
Subject: [Pymilter] Python spamd client and spamc-milter
Message-ID: <4EB2D05F.1090206@grinta.net>

Hello,

I was looking for a simple way of integrating Spamassassin with Postfix,
I had a look at spamass-milter and honestly I didn't like it much. The
functionality it implements is simple but the implementation is quite
convoluted in my opinion and involves fiddling with mail headers (to
reconstruct the Received header, without much gained value IMHO) and
forking a new spamc process for each incoming message.

Therefore, I wrote a python spamd client (to avoid having to fork twice
on each request) and python implementation of a spamassassin milter
using the pymilter library. I'm not an expert on the topic, but the
code is very simple and does what I would expect in my testing.

The code is available in very preliminary form at:

http://dev.grinta.net/hg/pyspamc/

Is anyone interested in it? Should polish and publish it properly, or
the project has no value? Thank you for your comments.

Cheers,
-- 
Daniele


From stuart at bmsi.com  Thu Nov  3 14:03:25 2011
From: stuart at bmsi.com (Stuart D. Gathman)
Date: Thu, 3 Nov 2011 14:03:25 -0400 (EDT)
Subject: [Pymilter] Python spamd client and spamc-milter
In-Reply-To: <4EB2D05F.1090206@grinta.net>
References: <4EB2D05F.1090206@grinta.net>
Message-ID: <alpine.LRH.2.00.1111031354560.28477@bmsred.bmsi.com>

On Thu, 3 Nov 2011, Daniele Nicolodi wrote:

> http://dev.grinta.net/hg/pyspamc/
>
> Is anyone interested in it? Should polish and publish it properly, or
> the project has no value? Thank you for your comments.

The general public is much more likely to use a milter (or anything else)
that does one thing and does it well.  Almost no one (except me) uses my
production milter (that does DKIM, GOSSiP, SPF, DSPAM, defang, archive,
wiretap, ...).  However, quite a few people use the much simpler spf-milter,
which only does SPF.

So I think your milter that does spamd, and only that, and does it well
would have a following.  As it is python, it is easy to add feature
bloat if they need it.

One of these days I'll get around to designing a plugin framework 
where a single python milter process can combine a number of 
do-one-thing modules.  Sendmail will, of course, compose multiple
milters, but there is limited communication between them (via email
headers only without heroic measures).

--
 	      Stuart D. Gathman <stuart at bmsi.com>
     Business Management Systems Inc.  Phone: 703 591-0911 Fax: 703 591-6154
"Confutatis maledictis, flammis acribus addictis" - background song for
a Microsoft sponsored "Where do you want to go from here?" commercial.


From jcea at jcea.es  Mon Nov  7 06:58:52 2011
From: jcea at jcea.es (Jesus Cea)
Date: Mon, 07 Nov 2011 12:58:52 +0100
Subject: [Pymilter] Python spamd client and spamc-milter
In-Reply-To: <4EB2D05F.1090206@grinta.net>
References: <4EB2D05F.1090206@grinta.net>
Message-ID: <4EB7C7FC.7080900@jcea.es>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 03/11/11 18:33, Daniele Nicolodi wrote:
> Hello,
> 
> I was looking for a simple way of integrating Spamassassin with
> Postfix, I had a look at spamass-milter and honestly I didn't like
> it much. The functionality it implements is simple but the
> implementation is quite convoluted in my opinion and involves
> fiddling with mail headers (to reconstruct the Received header,
> without much gained value IMHO) and forking a new spamc process for
> each incoming message.
> 
> Therefore, I wrote a python spamd client (to avoid having to fork
> twice on each request) and python implementation of a spamassassin
> milter using the pymilter library. I'm not an expert on the topic,
> but the code is very simple and does what I would expect in my
> testing.
> 
> The code is available in very preliminary form at:
> 
> http://dev.grinta.net/hg/pyspamc/
> 
> Is anyone interested in it? Should polish and publish it properly,
> or the project has no value? Thank you for your comments.

I was going thru the same problem and the same solution (a python
wrapper). Thanks for your code. I hope you keep it updated, properly
released, etc.

- -- 
Jesus Cea Avion                         _/_/      _/_/_/        _/_/_/
jcea at jcea.es - http://www.jcea.es/     _/_/    _/_/  _/_/    _/_/  _/_/
jabber / xmpp:jcea at jabber.org         _/_/    _/_/          _/_/_/_/_/
.                              _/_/  _/_/    _/_/          _/_/  _/_/
"Things are not so easy"      _/_/  _/_/    _/_/  _/_/    _/_/  _/_/
"My name is Dump, Core Dump"   _/_/_/        _/_/_/      _/_/  _/_/
"El amor es poner tu felicidad en la felicidad de otro" - Leibniz
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQCVAwUBTrfH/Jlgi5GaxT1NAQIcogP8DiABpjTkLVmzFLSBqWrtGrm+qm+gZ7p2
duYiyamR7HkzxdYDDlEcNWTpHCK5s8YlyNIo0aseYxh107mqfdUpXEg3JUpMXjyd
emFJyG2OPeuJfV55/qavtBO0rnFw5UGfJnlvYL5H83SdMMK0wRA4g8H1DicRXakl
GMBN3seORDU=
=4uBb
-----END PGP SIGNATURE-----


From daniele at grinta.net  Mon Nov  7 08:45:48 2011
From: daniele at grinta.net (Daniele Nicolodi)
Date: Mon, 07 Nov 2011 14:45:48 +0100
Subject: [Pymilter] Python spamd client and spamc-milter
In-Reply-To: <4EB7C7FC.7080900@jcea.es>
References: <4EB2D05F.1090206@grinta.net> <4EB7C7FC.7080900@jcea.es>
Message-ID: <4EB7E10C.3040301@grinta.net>

On 07/11/11 12:58, Jesus Cea wrote:
>> Therefore, I wrote a python spamd client (to avoid having to fork
>> twice on each request) and python implementation of a spamassassin
>> milter using the pymilter library. I'm not an expert on the topic,
>> but the code is very simple and does what I would expect in my
>> testing.
> 
>> The code is available in very preliminary form at:
> 
>> http://dev.grinta.net/hg/pyspamc/
> 
>> Is anyone interested in it? Should polish and publish it properly,
>> or the project has no value? Thank you for your comments.
> 
> I was going thru the same problem and the same solution (a python
> wrapper). Thanks for your code. I hope you keep it updated, properly
> released, etc.

Hello Jesus,

you are the second person expressing interest in this work. Therefore
I'll try to polish it and package it properly. Expect a release and a
Debian package soon (well, as soon as I'll find some time to work on it)

I have found some shortcomings in the current version of the milter, for
example the reject message is not properly customized, and I would like
to add some logging.

Any contribution is definitely welcome.

PS: this is not a wrapper but an implementation of the spamd protocol in
python. I do not have idea on how well it performs.

Cheers,
-- 
Daniele


From LWapnitsky at ph.wrtdesign.com  Wed Nov 16 10:44:00 2011
From: LWapnitsky at ph.wrtdesign.com (Larry G. Wapnitsky)
Date: Wed, 16 Nov 2011 10:44:00 -0500
Subject: [Pymilter] milter.error: cannot set reply (redux)
Message-ID: <FAE5CA7BEE39AF438CE87BF8DBE1375106AC9819@ph-svr-exch1.wrtdesign.com>

Referencing this old post
(http://bmsi.com/pipermail/pymilter/2009-June/000200.html), I'm now
experiencing the same issue with Postfix 2.7.1 and the latest version of
pymilter.  I'm working from the template right now as a base to design
my own milter, but can't seem to get past this error without commenting
it out.

Has there been progress made on a resolve?

Thanks,
Larry




----wrtmail--%3423wrt%----


From stuart at bmsi.com  Wed Nov 16 23:09:05 2011
From: stuart at bmsi.com (Stuart D. Gathman)
Date: Wed, 16 Nov 2011 23:09:05 -0500 (EST)
Subject: [Pymilter] milter.error: cannot set reply (redux)
In-Reply-To: <FAE5CA7BEE39AF438CE87BF8DBE1375106AC9819@ph-svr-exch1.wrtdesign.com>
References: <FAE5CA7BEE39AF438CE87BF8DBE1375106AC9819@ph-svr-exch1.wrtdesign.com>
Message-ID: <alpine.LRH.2.00.1111162256200.22943@bmsred.bmsi.com>

On Wed, 16 Nov 2011, Larry G. Wapnitsky wrote:

> Referencing this old post
> (http://bmsi.com/pipermail/pymilter/2009-June/000200.html), I'm now
> experiencing the same issue with Postfix 2.7.1 and the latest version of
> pymilter.  I'm working from the template right now as a base to design
> my own milter, but can't seem to get past this error without commenting
> it out.
>
> Has there been progress made on a resolve?

Did FAQ #13 not help?

http://www.bmsi.com/python/faq.html

I don't know how to configure the timeouts for postfix.
You can also call the progress() method to let the MTA know you're
"still working on it".

Some MTAs do not accept '' for xcode (and sample.py should include a valid 
xcode).  Maybe setreply should check this and give a more specific diagnostic.

--
 	      Stuart D. Gathman <stuart at bmsi.com>
     Business Management Systems Inc.  Phone: 703 591-0911 Fax: 703 591-6154
"Confutatis maledictis, flammis acribus addictis" - background song for
a Microsoft sponsored "Where do you want to go from here?" commercial.


From LWapnitsky at ph.wrtdesign.com  Thu Nov 17 06:12:34 2011
From: LWapnitsky at ph.wrtdesign.com (Larry G. Wapnitsky)
Date: Thu, 17 Nov 2011 06:12:34 -0500
Subject: [Pymilter] milter.error: cannot set reply (redux)
References: <FAE5CA7BEE39AF438CE87BF8DBE1375106AC9819@ph-svr-exch1.wrtdesign.com>
	<alpine.LRH.2.00.1111162256200.22943@bmsred.bmsi.com>
Message-ID: <FAE5CA7BEE39AF438CE87BF8DBE1375105ECF98F@ph-svr-exch1.wrtdesign.com>

I figured it out.  The sample code I had was using 250 instead of 550.  Once I changed that (as Postfix can only accept 550), it all worked.

Thanks, Stuart.  Great work, still learning so I can roll my own milter.

Larry G. Wapnitsky, MCSE, MCP+I
IT Support & Administration Coordinator
WRT / Wallace Roberts & Todd, LLC
1700 Market Street, 28th Floor, Philadelphia, PA 19103
T (215) 430-5068                   F (215) 732-2551
C (215) 713-8635                   E LWapnitsky at ph.wrtdesign.com
www.wrtdesign.com




-----Original Message-----
From: Stuart D. Gathman [mailto:stuart at bmsi.com]
Sent: Wed 2011-11-16 23:09
To: Larry G. Wapnitsky
Cc: pymilter at bmsi.com
Subject: Re: [Pymilter] milter.error: cannot set reply (redux)
 
On Wed, 16 Nov 2011, Larry G. Wapnitsky wrote:

> Referencing this old post
> (http://bmsi.com/pipermail/pymilter/2009-June/000200.html), I'm now
> experiencing the same issue with Postfix 2.7.1 and the latest version of
> pymilter.  I'm working from the template right now as a base to design
> my own milter, but can't seem to get past this error without commenting
> it out.
>
> Has there been progress made on a resolve?

Did FAQ #13 not help?

http://www.bmsi.com/python/faq.html

I don't know how to configure the timeouts for postfix.
You can also call the progress() method to let the MTA know you're
"still working on it".

Some MTAs do not accept '' for xcode (and sample.py should include a valid 
xcode).  Maybe setreply should check this and give a more specific diagnostic.

--
 	      Stuart D. Gathman <stuart at bmsi.com>
     Business Management Systems Inc.  Phone: 703 591-0911 Fax: 703 591-6154
"Confutatis maledictis, flammis acribus addictis" - background song for
a Microsoft sponsored "Where do you want to go from here?" commercial.






----wrtmail--%3423wrt%----
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://gathman.org/pipermail/pymilter/attachments/20111117/c65113b3/attachment.html>

From durwin at mgtsciences.com  Fri Nov 18 16:09:35 2011
From: durwin at mgtsciences.com (durwin at mgtsciences.com)
Date: Fri, 18 Nov 2011 14:09:35 -0700
Subject: [Pymilter] Getting more header information
Message-ID: <OF79272EFF.10FA4897-ON8725794C.0073B2B5-8725794C.00743C37@mgtsciences.com>

I want to get the recipient address but can't seem to.  I have this in my 
sendmail.mc.

define(`confMILTER_MACROS_ENVRCPT',`r, v, Z, {i}, {mail_mailer}, 
{mail_addr}, {rcpt_mailer},  {rcpt_addr}, {auth_authen}, {auth_type}, 
{if_addr},  {_}')dnl

then using this to get the iformation.

self.getsymval('{auth_authen}')

but when I print the results it comes out 'None'.  I tried it from within 
envfrom.  What am I missing?


Thank you,

Durwin F. De La Rue



This email message and any attachments are for the sole use of the 
intended recipient(s) and may contain proprietary and/or confidential 
information which may be privileged or otherwise protected from 
disclosure. Any unauthorized review, use, disclosure or distribution is 
prohibited. If you are not the intended recipient(s), please contact the 
sender by reply email and destroy the original message and any copies of 
the message as well as any attachments to the original message.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://gathman.org/pipermail/pymilter/attachments/20111118/5fa197d4/attachment.html>

From stuart at bmsi.com  Fri Nov 18 16:19:10 2011
From: stuart at bmsi.com (Stuart D. Gathman)
Date: Fri, 18 Nov 2011 16:19:10 -0500 (EST)
Subject: [Pymilter] Getting more header information
In-Reply-To: <OF79272EFF.10FA4897-ON8725794C.0073B2B5-8725794C.00743C37@mgtsciences.com>
References: <OF79272EFF.10FA4897-ON8725794C.0073B2B5-8725794C.00743C37@mgtsciences.com>
Message-ID: <alpine.LRH.2.00.1111181616150.10717@bmsred.bmsi.com>

On Fri, 18 Nov 2011, durwin at mgtsciences.com wrote:

> I want to get the recipient address but can't seem to.  I have this in my
> sendmail.mc.
> 
> define(`confMILTER_MACROS_ENVRCPT',`r, v, Z, {i}, {mail_mailer},
> {mail_addr}, {rcpt_mailer},  {rcpt_addr}, {auth_authen}, {auth_type},
> {if_addr},  {_}')dnl

auth_authen is not available in envrcpt (the above is wrong).  It should be
available in envfrom by default.

> then using this to get the iformation.
> 
> self.getsymval('{auth_authen}')
> 
> but when I print the results it comes out 'None'.  I tried it from within
> envfrom.  What am I missing?

None is correct when no authentication has taken place.

--
 	      Stuart D. Gathman <stuart at bmsi.com>
     Business Management Systems Inc.  Phone: 703 591-0911 Fax: 703 591-6154
"Confutatis maledictis, flammis acribus addictis" - background song for
a Microsoft sponsored "Where do you want to go from here?" commercial.


From LWapnitsky at ph.wrtdesign.com  Mon Nov 21 12:50:39 2011
From: LWapnitsky at ph.wrtdesign.com (Larry G. Wapnitsky)
Date: Mon, 21 Nov 2011 12:50:39 -0500
Subject: [Pymilter] Emulating another MimeDefang function
Message-ID: <FAE5CA7BEE39AF438CE87BF8DBE1375106AC9834@ph-svr-exch1.wrtdesign.com>

I'm trying to add to the current features by creating a function similar
to MimeDefang whereby attachments can be saved to a folder and replaced
with warning text.  The stumbling block I'm currently hitting is getting
the names of each file attached to a message.  I've been working with
the code for a few days and seem to keep missing something.

 

A carrot for this horse would be very acceptable.

 

Thanks,

 

Larry 

 






----wrtmail--%3423wrt%----
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://gathman.org/pipermail/pymilter/attachments/20111121/3d2ec54d/attachment.html>

From stuart at bmsi.com  Mon Nov 21 13:19:14 2011
From: stuart at bmsi.com (Stuart D. Gathman)
Date: Mon, 21 Nov 2011 13:19:14 -0500 (EST)
Subject: [Pymilter] Emulating another MimeDefang function
In-Reply-To: <FAE5CA7BEE39AF438CE87BF8DBE1375106AC9834@ph-svr-exch1.wrtdesign.com>
References: <FAE5CA7BEE39AF438CE87BF8DBE1375106AC9834@ph-svr-exch1.wrtdesign.com>
Message-ID: <alpine.LRH.2.00.1111211255050.2441@bmsred.bmsi.com>

On Mon, 21 Nov 2011, Larry G. Wapnitsky wrote:

> I?m trying to add to the current features by creating a function similar to
> MimeDefang whereby attachments can be saved to a folder and replaced with
> warning text.  The stumbling block I?m currently hitting is getting the
> names of each file attached to a message.  I?ve been working with the code
> for a few days and seem to keep missing something.

Check out the mime.py module provided with pymilter.  Here is a simple
example of usage (extending milter-template).  For a more complex example
see bms.py in the milter package.  You can provide your own 
check_attachment method if desired.  The default removes attachments
with banned extensions, optionally including inside zip files.

...

   def eom(self):
     self.fp.seek(0)
     msg = email.message_from_file(self.fp)
     # pass header changes in top level message to sendmail
     msg.headerchange = self._headerChange

     # filter leaf attachments through _chk_attach
     assert not msg.ismodified()
     # FIXME: copy self.fp to a save directory and save filename in
     # self.savname
     rc = mime.defang(msg,self.savname)
     defanged = msg.ismodified()
     if defanged:
       # Body modified, copy modified message to a temp file
       out = tempfile.TemporaryFile()
       try:
 	msg.dump(out)
 	out.seek(0)
 	# Since we wrote headers with '\n' (no CR),
 	# the following header/body split should always work.
 	msg = out.read().split('\n\n',1)[-1]
 	self.replacebody(msg)     # feed modified message to sendmail

     self.setreply('250','2.5.1','Grokked by pymilter')
     return Milter.ACCEPT

   def _headerChange(self,msg,name,value):
     if value:   # add header
       self.addheader(name,value)
     else:       # delete all headers with name
       h = msg.getheaders(name)
       if h:
         for i in range(len(h),0,-1):
           self.chgheader(name,i-1,'')

--
 	      Stuart D. Gathman <stuart at bmsi.com>
     Business Management Systems Inc.  Phone: 703 591-0911 Fax: 703 591-6154
"Confutatis maledictis, flammis acribus addictis" - background song for
a Microsoft sponsored "Where do you want to go from here?" commercial.


From LWapnitsky at ph.wrtdesign.com  Mon Nov 21 13:22:41 2011
From: LWapnitsky at ph.wrtdesign.com (Larry G. Wapnitsky)
Date: Mon, 21 Nov 2011 13:22:41 -0500
Subject: [Pymilter] Emulating another MimeDefang function
Message-ID: <1e4e601cca87a$87ed7d1f$1e02660a@wrtdesign.com>

I'll check it out.  I don't seem to have the bms.py file in my download.




-----Original Message-----
From: Stuart D. Gathman [stuart at bmsi.com]
Received: Monday, 21 Nov 2011, 13:19
To: Larry G. Wapnitsky [LWapnitsky at ph.wrtdesign.com]
CC: pymilter at bmsi.com [pymilter at bmsi.com]
Subject: Re: [Pymilter] Emulating another MimeDefang function


On Mon, 21 Nov 2011, Larry G. Wapnitsky wrote:

> I?m trying to add to the current features by creating a function similar to
> MimeDefang whereby attachments can be saved to a folder and replaced with
> warning text.  The stumbling block I?m currently hitting is getting the
> names of each file attached to a message.  I?ve been working with the code
> for a few days and seem to keep missing something.

Check out the mime.py module provided with pymilter.  Here is a simple
example of usage (extending milter-template).  For a more complex example
see bms.py in the milter package.  You can provide your own 
check_attachment method if desired.  The default removes attachments
with banned extensions, optionally including inside zip files.

...

   def eom(self):
     self.fp.seek(0)
     msg = email.message_from_file(self.fp)
     # pass header changes in top level message to sendmail
     msg.headerchange = self._headerChange

     # filter leaf attachments through _chk_attach
     assert not msg.ismodified()
     # FIXME: copy self.fp to a save directory and save filename in
     # self.savname
     rc = mime.defang(msg,self.savname)
     defanged = msg.ismodified()
     if defanged:
       # Body modified, copy modified message to a temp file
       out = tempfile.TemporaryFile()
       try:
 	msg.dump(out)
 	out.seek(0)
 	# Since we wrote headers with '\n' (no CR),
 	# the following header/body split should always work.
 	msg = out.read().split('\n\n',1)[-1]
 	self.replacebody(msg)     # feed modified message to sendmail

     self.setreply('250','2.5.1','Grokked by pymilter')
     return Milter.ACCEPT

   def _headerChange(self,msg,name,value):
     if value:   # add header
       self.addheader(name,value)
     else:       # delete all headers with name
       h = msg.getheaders(name)
       if h:
         for i in range(len(h),0,-1):
           self.chgheader(name,i-1,'')

--
 	      Stuart D. Gathman <stuart at bmsi.com>
     Business Management Systems Inc.  Phone: 703 591-0911 Fax: 703 591-6154
"Confutatis maledictis, flammis acribus addictis" - background song for
a Microsoft sponsored "Where do you want to go from here?" commercial.





----wrtmail--%3423wrt%----

<!DSPAM:133C75D395E11560410343425>


From stuart at bmsi.com  Mon Nov 21 13:35:07 2011
From: stuart at bmsi.com (Stuart D. Gathman)
Date: Mon, 21 Nov 2011 13:35:07 -0500 (EST)
Subject: [Pymilter] Emulating another MimeDefang function
In-Reply-To: <1e4e601cca87a$87ed7d1f$1e02660a@wrtdesign.com>
References: <1e4e601cca87a$87ed7d1f$1e02660a@wrtdesign.com>
Message-ID: <alpine.LRH.2.00.1111211331150.2441@bmsred.bmsi.com>

On Mon, 21 Nov 2011, Larry G. Wapnitsky wrote:

> I'll check it out.  I don't seem to have the bms.py file in my download.

http://sourceforge.net/projects/pymilter/files/milter/

--
 	      Stuart D. Gathman <stuart at bmsi.com>
     Business Management Systems Inc.  Phone: 703 591-0911 Fax: 703 591-6154
"Confutatis maledictis, flammis acribus addictis" - background song for
a Microsoft sponsored "Where do you want to go from here?" commercial.


From stuart at bmsi.com  Mon Nov 21 15:03:38 2011
From: stuart at bmsi.com (Stuart D. Gathman)
Date: Mon, 21 Nov 2011 15:03:38 -0500 (EST)
Subject: [Pymilter] Emulating another MimeDefang function
In-Reply-To: <FAE5CA7BEE39AF438CE87BF8DBE1375106AC9837@ph-svr-exch1.wrtdesign.com>
References: <FAE5CA7BEE39AF438CE87BF8DBE1375106AC9834@ph-svr-exch1.wrtdesign.com>
	<alpine.LRH.2.00.1111211255050.2441@bmsred.bmsi.com>
	<FAE5CA7BEE39AF438CE87BF8DBE1375106AC9837@ph-svr-exch1.wrtdesign.com>
Message-ID: <alpine.LRH.2.00.1111211459001.4409@bmsred.bmsi.com>

On Mon, 21 Nov 2011, Larry G. Wapnitsky wrote:

> I still don't see how this is getting me the filenames I need.  Also,

The check= argument to defang is a function that is passed the filename.

> check_attachment seems to be recursive and references all the defang
> functions, hence increasing my confusion.

MIME is a recursive structure, hence any attachment and/or filename 
enumeration must be recursive.

> Additionally, when I send a multipart MIME message to my through my
> server, ismultipart() reports false, but output of the message parts
> says my message is mime multipart.

You will have to provide a specific example.

--
 	      Stuart D. Gathman <stuart at bmsi.com>
     Business Management Systems Inc.  Phone: 703 591-0911 Fax: 703 591-6154
"Confutatis maledictis, flammis acribus addictis" - background song for
a Microsoft sponsored "Where do you want to go from here?" commercial.



From LWapnitsky at ph.wrtdesign.com  Mon Nov 21 15:30:01 2011
From: LWapnitsky at ph.wrtdesign.com (Larry G. Wapnitsky)
Date: Mon, 21 Nov 2011 15:30:01 -0500
Subject: [Pymilter] Emulating another MimeDefang function
In-Reply-To: <FAE5CA7BEE39AF438CE87BF8DBE1375105ECF994@ph-svr-exch1.wrtdesign.com>
References: <FAE5CA7BEE39AF438CE87BF8DBE1375106AC9834@ph-svr-exch1.wrtdesign.com>
	<alpine.LRH.2.00.1111211255050.2441@bmsred.bmsi.com>
	<FAE5CA7BEE39AF438CE87BF8DBE1375106AC9837@ph-svr-exch1.wrtdesign.com>
	<alpine.LRH.2.00.1111211459001.4409@bmsred.bmsi.com>
	<FAE5CA7BEE39AF438CE87BF8DBE1375105ECF994@ph-svr-exch1.wrtdesign.com>
Message-ID: <FAE5CA7BEE39AF438CE87BF8DBE1375106AC9838@ph-svr-exch1.wrtdesign.com>

Same thing happened with a message from GMail

Larry G. Wapnitsky MBA, MCSE, MCP+I
IT SUPPORT & ADMINISTRATION COORDINATOR
WRT  |  Wallace Roberts & Todd, LLC
1700 Market Street, 28th Fl
Philadelphia, PA 19103
T  215.430.5068
C  215.713.8635
E  lwapnitsky at WRTdesign.com


-----Original Message-----
From: Larry G. Wapnitsky 
Sent: Monday, November 21, 2011 3:08 PM
To: Stuart D. Gathman
Subject: RE: [Pymilter] Emulating another MimeDefang function

OK...still puzzling through the defang stuff.  I'm trying to trace
through and still don't see what I'm looking for.

As for the part about misreporting the multiparts, I'm attaching a file
to a message in Thunderbird and sending it through my mail server.  I
have the output logged for diagnostics as follows:


2011Nov21 15:11:23 milter startup
2011Nov21 15:11:30 [1] connect from [10.102.50.66] at ('10.102.50.66',
51781)
2011Nov21 15:11:30 [1] False
2011Nov21 15:11:30 [1] This is a multi-part message in MIME format.
--------------090109040705000201000008
Content-Type: multipart/alternative;
 boundary="------------080609030809060906010707"


--------------080609030809060906010707
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

test message*2*

--------------080609030809060906010707
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> <html>
  <head>

    <meta http-equiv="content-type" content="text/html;
charset=ISO-8859-1">
  </head>
  <body bgcolor="#ffffff" text="#000000">
    test message<b>2</b><br>
  </body>
</html>

--------------080609030809060906010707--

--------------090109040705000201000008
Content-Type: text/x-perl;
 name="hashtest1.pl"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename="hashtest1.pl"

#/usr/bin/env perl
use Digest::SHA1;

$sha1 = Digest::SHA1->new;

open $fh, $ARGV[0] or die $!;
$sha1->addfile($fh);

$digest = $sha1->hexdigest;

print $digest, "    $ARGV[0]\n";

close FILE;

--------------090109040705000201000008--

2011Nov21 15:11:30 [1] text/plain
Removing /tmp/py_testmilter.sock
2011Nov21 15:11:35 bms milter shutdown


Below is my working code.  Please note, I've only been working in Python
for about 1 week:



#! /usr/bin/env python

import Milter
import StringIO
import email
import email.message
import mime
import rfc822
import sys
import tempfile
import time
import unittest

from email import Errors

## ==  IP Information
from socket import AF_INET, AF_INET6
## ==

## === Define multiprocessing == ##

if True:
    from multiprocessing import Process as Thread, Queue
else:
    from threading import Thread
    from Queue import Queue

logq = Queue(maxsize=4)


def background():
    while True:
        t = logq.get()
        if not t: break
        msg,id,ts = t
        print "%s [%d]" % (time.strftime('%Y%b%d
%H:%M:%S',time.localtime(ts)),id),
        # 2005Oct13 02:34:11 [1] msg1 msg2 msg3 ...
        for i in msg: print i,
        print

## === End Define Multiprocesing === ##

class mltr_SaveAttachments(Milter.Base):

    def __init__(self):
        self.id = Milter.uniqueID()

    def log(self,*msg):
        logq.put((msg,self.id,time.time()))
    
    def connect(self, IPname, family, hostaddr):
        self.IP = hostaddr[0]
        self.port = hostaddr[1]
        if family == AF_INET6:
            self.flow = hostaddr[2]
            self.scope = hostaddr[3]
        else:
            self.flow = None
            self.scope = None
        self.IPname = IPname # Name from a reverse IP lookup
        self.H = None
        self.fp = None
        self.receiver = self.getsymval('j')
        self.log("connect from %s at %s" % (IPname, hostaddr) )
        return Milter.CONTINUE

    

    def envfrom(self,mailfrom,*str):
        # self.F = mailfrom
        # self.R = []
        # self.fromparms = Milter.dictfromlist(str)
        # self.user = self.getsymval('{auth_authen}')
        # self.log("mail from:", mailfrom, *str)
        self.fp = StringIO.StringIO()
        # self.canon_from = '@'.join(parse_addr(mailfrom))
        # self.fp.write('From %s %s\n' % (self.canon_from,time,ctime()))
        return Milter.CONTINUE

    def body(self, chunk):
        self.fp.write(chunk)
        return Milter.CONTINUE
  
    def replacebody(self,chunk):
       if self._body:
            self._body.write(chunk)
            self.bodyreplaced = True
#            self.log("mail body chunk")
#            self.log(chunk)
            return Milter.CONTINUE
       else:
            raise IOError,"replacebody not called from eom()"
            return Milter.TEMPFAIL

    def attachment(self):
#       parts = self._msg.get_payload()
#       self.log(len(parts))
#       self.log(parts)
#       for p in parts:
#           self.log("part")
        self.log(self._msg.ismultipart())
        parts = self._msg.get_payload()
        self.log(parts)
        t = self._msg.get_content_type().lower()
        self.log(t)
        


    def eom(self):
        self.fp.seek(0)
        #msg = email.message_from_file(self.fp)
        msg = mime.message_from_file(self.fp)
        self._msg = msg;
        self.attachment()

        self._body = StringIO.StringIO()


        self.tempname = fname = tempfile.mktemp(".tmp")
#        self.log(self.tempname)
        
        out = tempfile.TemporaryFile()
        
        try:
            msg.dump(out)
            out.seek(0)
            msg = rfc822.Message(out)
        
            msg.rewindbody()
            while 1:
                buf = out.read(8192)
                if len(buf) == 0: break
                self.replacebody(buf)
#            return Milter.ACCEPT
            return Milter.TEMPFAIL
        finally:
            out.close()
        return Milter.TEMPFAIL
        

## ===

def main():
    bt = Thread(target=background)
    bt.start()
    socketname = "/tmp/py_testmilter.sock"
    timeout = 600
    Milter.factory = mltr_SaveAttachments
    flags = Milter.CHGBODY + Milter.CHGHDRS + Milter.ADDHDRS
    flags += Milter.ADDRCPT
    flags += Milter.DELRCPT
    Milter.set_flags(flags)     # tell Sendmail/Postfix which features
we use
    print "%s milter startup" % time.strftime('%Y%b%d %H:%M:%S')
    sys.stdout.flush()
    Milter.runmilter("py_testmilter",socketname,timeout)
    logq.put(None)
    bt.join()
    print "%s bms milter shutdown" % time.strftime('%Y%b%d %H:%M:%S')

if __name__ == "__main__":
    main()



-----Original Message-----
From: Stuart D. Gathman [mailto:stuart at bmsi.com]
Sent: Mon 2011-11-21 15:03
To: Larry G. Wapnitsky
Subject: RE: [Pymilter] Emulating another MimeDefang function
 
On Mon, 21 Nov 2011, Larry G. Wapnitsky wrote:

> I still don't see how this is getting me the filenames I need.  Also,

The check= argument to defang is a function that is passed the filename.

> check_attachment seems to be recursive and references all the defang 
> functions, hence increasing my confusion.

MIME is a recursive structure, hence any attachment and/or filename
enumeration must be recursive.

> Additionally, when I send a multipart MIME message to my through my 
> server, ismultipart() reports false, but output of the message parts 
> says my message is mime multipart.

You will have to provide a specific example.

--
 	      Stuart D. Gathman <stuart at bmsi.com>
     Business Management Systems Inc.  Phone: 703 591-0911 Fax: 703
591-6154 "Confutatis maledictis, flammis acribus addictis" - background
song for a Microsoft sponsored "Where do you want to go from here?"
commercial.





----wrtmail--%3423wrt%----


From stuart at bmsi.com  Mon Nov 21 15:31:37 2011
From: stuart at bmsi.com (Stuart D. Gathman)
Date: Mon, 21 Nov 2011 15:31:37 -0500 (EST)
Subject: [Pymilter] Emulating another MimeDefang function
In-Reply-To: <FAE5CA7BEE39AF438CE87BF8DBE1375105ECF994@ph-svr-exch1.wrtdesign.com>
References: <FAE5CA7BEE39AF438CE87BF8DBE1375106AC9834@ph-svr-exch1.wrtdesign.com>
	<alpine.LRH.2.00.1111211255050.2441@bmsred.bmsi.com>
	<FAE5CA7BEE39AF438CE87BF8DBE1375106AC9837@ph-svr-exch1.wrtdesign.com>
	<alpine.LRH.2.00.1111211459001.4409@bmsred.bmsi.com>
	<FAE5CA7BEE39AF438CE87BF8DBE1375105ECF994@ph-svr-exch1.wrtdesign.com>
Message-ID: <alpine.LRH.2.00.1111211517250.5335@bmsred.bmsi.com>

On Mon, 21 Nov 2011, Larry G. Wapnitsky wrote:

> OK...still puzzling through the defang stuff.  I'm trying to trace through
> and still don't see what I'm looking for.

The getnames() method extracts the attachment names zipnames() extracts
zip filename.  For defang purposes, there is no standard for extracting names.
It all depends on bugs in LookOut.  Lookout thinks *any* Content-Type
attribute value that looks like a filename might be a file name, and
attempts to execute the attachment it it has an executable extension.

> As for the part about misreporting the multiparts, I'm attaching a file to a
> message in Thunderbird and sending it through my mail server.  I have the
> output logged for diagnostics as follows:

You do not collect the email headers via the header callback.

Here is a very simple one from milter-template.py:

   @Milter.noreply
   def header(self, name, hval):
     self.fp.write("%s: %s\n" % (name,hval))     # add header to buffer
     return Milter.CONTINUE

   @Milter.noreply
   def eoh(self):
     self.fp.write("\n")                         # terminate headers
     return Milter.CONTINUE

--
 	      Stuart D. Gathman <stuart at bmsi.com>
     Business Management Systems Inc.  Phone: 703 591-0911 Fax: 703 591-6154
"Confutatis maledictis, flammis acribus addictis" - background song for
a Microsoft sponsored "Where do you want to go from here?" commercial.



From stuart at bmsi.com  Mon Nov 21 15:42:35 2011
From: stuart at bmsi.com (Stuart D. Gathman)
Date: Mon, 21 Nov 2011 15:42:35 -0500 (EST)
Subject: [Pymilter] Emulating another MimeDefang function
In-Reply-To: <FAE5CA7BEE39AF438CE87BF8DBE1375106AC9839@ph-svr-exch1.wrtdesign.com>
References: <FAE5CA7BEE39AF438CE87BF8DBE1375106AC9834@ph-svr-exch1.wrtdesign.com>
	<alpine.LRH.2.00.1111211255050.2441@bmsred.bmsi.com>
	<FAE5CA7BEE39AF438CE87BF8DBE1375106AC9837@ph-svr-exch1.wrtdesign.com>
	<alpine.LRH.2.00.1111211459001.4409@bmsred.bmsi.com>
	<FAE5CA7BEE39AF438CE87BF8DBE1375105ECF994@ph-svr-exch1.wrtdesign.com>
	<alpine.LRH.2.00.1111211517250.5335@bmsred.bmsi.com>
	<FAE5CA7BEE39AF438CE87BF8DBE1375106AC9839@ph-svr-exch1.wrtdesign.com>
Message-ID: <alpine.LRH.2.00.1111211538270.5725@bmsred.bmsi.com>

On Mon, 21 Nov 2011, Larry G. Wapnitsky wrote:

> Will try this.

If you were curious about the standard place for filenames (as opposed
the many places LookOut checks), look at the get_filename() method in
email.Message (python comes with source).  It looks first in the
Content-Disposition header field for the filename attribute (the "correct"
place), and secondly in the name attribute of Content-Type.

--
 	      Stuart D. Gathman <stuart at bmsi.com>
     Business Management Systems Inc.  Phone: 703 591-0911 Fax: 703 591-6154
"Confutatis maledictis, flammis acribus addictis" - background song for
a Microsoft sponsored "Where do you want to go from here?" commercial.


From stuart at bmsi.com  Mon Nov 21 13:54:15 2011
From: stuart at bmsi.com (Stuart D. Gathman)
Date: Mon, 21 Nov 2011 13:54:15 -0500 (EST)
Subject: [Pymilter] Getting more header information
In-Reply-To: <OF097014A9.F60DB002-ON8725794F.00667133-8725794F.00669A00@mgtsciences.com>
References: <OF79272EFF.10FA4897-ON8725794C.0073B2B5-8725794C.00743C37@mgtsciences.com>
	<alpine.LRH.2.00.1111181616150.10717@bmsred.bmsi.com>
	<OF097014A9.F60DB002-ON8725794F.00667133-8725794F.00669A00@mgtsciences.com>
Message-ID: <alpine.LRH.2.00.1111211346260.3486@bmsred.bmsi.com>

On Mon, 21 Nov 2011, durwin at mgtsciences.com wrote:

> That was just there from some experimentation.  What I was looking for is
> the recipient address, the TO.  Is that not rcpt_addr?
> IF so, I a not getting it.

The envelope recipient is passed to envrcpt.  The contents of the To: header
field is not available at SMTP envelope time, but you can collect it
in the header callback.  Macros are MTA dependent, but on sendmail the
rcpt_addr macro is the recipient address after virtualuser table aliasing.

--
 	      Stuart D. Gathman <stuart at bmsi.com>
     Business Management Systems Inc.  Phone: 703 591-0911 Fax: 703 591-6154
"Confutatis maledictis, flammis acribus addictis" - background song for
a Microsoft sponsored "Where do you want to go from here?" commercial.



From LWapnitsky at ph.wrtdesign.com  Wed Nov 23 10:38:22 2011
From: LWapnitsky at ph.wrtdesign.com (Larry G. Wapnitsky)
Date: Wed, 23 Nov 2011 10:38:22 -0500
Subject: [Pymilter] Emulating another MimeDefang function
In-Reply-To: <alpine.LRH.2.00.1111211538270.5725@bmsred.bmsi.com>
References: <FAE5CA7BEE39AF438CE87BF8DBE1375106AC9834@ph-svr-exch1.wrtdesign.com>
	<alpine.LRH.2.00.1111211255050.2441@bmsred.bmsi.com>
	<FAE5CA7BEE39AF438CE87BF8DBE1375106AC9837@ph-svr-exch1.wrtdesign.com>
	<alpine.LRH.2.00.1111211459001.4409@bmsred.bmsi.com>
	<FAE5CA7BEE39AF438CE87BF8DBE1375105ECF994@ph-svr-exch1.wrtdesign.com>
	<alpine.LRH.2.00.1111211517250.5335@bmsred.bmsi.com>
	<FAE5CA7BEE39AF438CE87BF8DBE1375106AC9839@ph-svr-exch1.wrtdesign.com>
	<alpine.LRH.2.00.1111211538270.5725@bmsred.bmsi.com>
Message-ID: <FAE5CA7BEE39AF438CE87BF8DBE1375106AC9843@ph-svr-exch1.wrtdesign.com>

So, I was able to remove the attachments in a function I've written, but
when the message gets sent, the attachments are still there rather than
my modified message.  Not sure what I'm missing.

Code below:


#! /usr/bin/env python

import Milter
import StringIO
import email
import email.Message
import hashlib
import mime
import os
import sys
import tempfile
import time
import rfc822


from email import Errors
from email.Message import Message

## ==  IP Information
from socket import AF_INET, AF_INET6
## ==

## === Define multiprocessing == ##

if True:
    from multiprocessing import Process as Thread, Queue
else:
    from threading import Thread
    from Queue import Queue

logq = Queue(maxsize=4)


def background():
    while True:
        t = logq.get()
        if not t: break
        msg,id,ts = t
        print "%s [%d]" % (time.strftime('%Y%b%d
%H:%M:%S',time.localtime(ts)),id),
        # 2005Oct13 02:34:11 [1] msg1 msg2 msg3 ...
        for i in msg: print i,
        print

## === End Define Multiprocesing === ##

class mltr_SaveAttachments(Milter.Base):

    def __init__(self):
        self.id = Milter.uniqueID()

    def log(self,*msg):
        logq.put((msg,self.id,time.time()))

    def connect(self, IPname, family, hostaddr):
        self.IP = hostaddr[0]
        self.port = hostaddr[1]
        if family == AF_INET6:
            self.flow = hostaddr[2]
            self.scope = hostaddr[3]
        else:
            self.flow = None
            self.scope = None
        self.IPname = IPname # Name from a reverse IP lookup
        self.H = None
        self.fp = None
        self.receiver = self.getsymval('j')
        self.log("connect from %s at %s" % (IPname, hostaddr) )
        return Milter.CONTINUE

    @Milter.noreply
    def header(self, name, hval):
        self.fp.write("%s: %s\n" % (name,hval)) # add header to buffer
        return Milter.CONTINUE

    @Milter.noreply
    def eoh(self):
        self.fp.write("\n")                             # terminate
headers
        return Milter.CONTINUE



    def envfrom(self,mailfrom,*str):
#        self.log("envfrom")
        # self.F = mailfrom
        # self.R = []
        # self.fromparms = Milter.dictfromlist(str)
        # self.user = self.getsymval('{auth_authen}')
        # self.log("mail from:", mailfrom, *str)
        self.fp = StringIO.StringIO()
        # self.canon_from = '@'.join(parse_addr(mailfrom))
        # self.fp.write('From %s %s\n' % (self.canon_from,time,ctime()))
        return Milter.CONTINUE


    def attachment(self):
        msg = self._msg
        attachDir = attach_dir(msg)
        removedParts = []

        for part in msg.walk():
            fname = ""

            if part.is_multipart():
                continue

            dtypes = part.get_params(None, 'Content-Disposition')

            if not dtypes:
                if part.get_content_type() == 'text/plain':
                    continue
                ctypes = part.getparams()
                if not ctypes:
                    continue
                for key,val in ctypes:
                    if key.lower() == 'name':
                        fname = val
            else:
                for key,val in dtypes:
                    if key.lower() == 'filename':
                        fname = val

            if fname:
                removedParts.append(fname)
                data = part.get_payload(decode=1)
                extract_attachment(data, attachDir, fname)
                part = delete_attachments(part)

        self.log(removedParts)
#        return Milter.CONTINUE
        return msg


    def eom(self):
        self.fp.seek(0)
        self._msg = mime.message_from_file(self.fp)
        self._msg = self.attachment()

        self.log(self._msg)
        return Milter.ACCEPT
#        return Milter.TEMPFAIL
## ===


def delete_attachments(part):
    for key,value in part.get_params():
        part.del_param(key)

    part.set_payload('[DELETED]\n')
    part.set_type('text/plain')
    del part['Content-Disposition']

    return part



def attach_dir(msg):
    tempname = fname = tempfile.mktemp(".tmp")
    out = tempfile.TemporaryFile()
    msg.dump(out)
    out.seek(0)
    buf = out.read()
    hashDir = hashit(buf)
    attachDir = dropDir + hashDir

    if not os.path.isdir(hashDir):
        os.mkdir(attachDir)

    return attachDir


def extract_attachment(data, attachDir, fname):
    exdir_file = attachDir + "/" + fname
    extracted = open(exdir_file, "wb")
    extracted.write(data)
    extracted.close()



def hashit(data):
    sha1 = hashlib.sha1()
    # f = open(filepath, 'rb')
    # try:
    #     sha1.update(f.read())
    # finally:
    #     f.close()
    sha1.update(data)

    return sha1.hexdigest()

dropDir = "/dropdir/"

def main():
    bt = Thread(target=background)
    bt.start()
    socketname = "/tmp/py_testmilter.sock"
    timeout = 600
    Milter.factory = mltr_SaveAttachments
    flags = Milter.CHGBODY + Milter.CHGHDRS + Milter.ADDHDRS
    flags += Milter.ADDRCPT
    flags += Milter.DELRCPT
    Milter.set_flags(flags)     # tell Sendmail/Postfix which features
we use
    print "%s milter startup" % time.strftime('%Y%b%d %H:%M:%S')
    sys.stdout.flush()
    Milter.runmilter("py_testmilter",socketname,timeout)
    logq.put(None)
    bt.join()
    print "%s bms milter shutdown" % time.strftime('%Y%b%d %H:%M:%S')

if __name__ == "__main__":
    main()



-----Original Message-----
From: Stuart D. Gathman [mailto:stuart at bmsi.com] 
Sent: Monday, November 21, 2011 3:43 PM
To: Larry G. Wapnitsky
Cc: pymilter at bmsi.com
Subject: RE: [Pymilter] Emulating another MimeDefang function

On Mon, 21 Nov 2011, Larry G. Wapnitsky wrote:

> Will try this.

If you were curious about the standard place for filenames (as opposed
the many places LookOut checks), look at the get_filename() method in
email.Message (python comes with source).  It looks first in the
Content-Disposition header field for the filename attribute (the
"correct"
place), and secondly in the name attribute of Content-Type.

--
 	      Stuart D. Gathman <stuart at bmsi.com>
     Business Management Systems Inc.  Phone: 703 591-0911 Fax: 703
591-6154 "Confutatis maledictis, flammis acribus addictis" - background
song for a Microsoft sponsored "Where do you want to go from here?"
commercial.




----wrtmail--%3423wrt%----

<!DSPAM:133D113DA422445417198307>



From LWapnitsky at ph.wrtdesign.com  Wed Nov 23 11:37:48 2011
From: LWapnitsky at ph.wrtdesign.com (Larry G. Wapnitsky)
Date: Wed, 23 Nov 2011 11:37:48 -0500
Subject: [Pymilter] Emulating another MimeDefang function
In-Reply-To: <FAE5CA7BEE39AF438CE87BF8DBE1375106AC9843@ph-svr-exch1.wrtdesign.com>
References: <FAE5CA7BEE39AF438CE87BF8DBE1375106AC9834@ph-svr-exch1.wrtdesign.com>
	<alpine.LRH.2.00.1111211255050.2441@bmsred.bmsi.com>
	<FAE5CA7BEE39AF438CE87BF8DBE1375106AC9837@ph-svr-exch1.wrtdesign.com>
	<alpine.LRH.2.00.1111211459001.4409@bmsred.bmsi.com>
	<FAE5CA7BEE39AF438CE87BF8DBE1375105ECF994@ph-svr-exch1.wrtdesign.com>
	<alpine.LRH.2.00.1111211517250.5335@bmsred.bmsi.com>
	<FAE5CA7BEE39AF438CE87BF8DBE1375106AC9839@ph-svr-exch1.wrtdesign.com>
	<alpine.LRH.2.00.1111211538270.5725@bmsred.bmsi.com>
	<FAE5CA7BEE39AF438CE87BF8DBE1375106AC9843@ph-svr-exch1.wrtdesign.com>
Message-ID: <FAE5CA7BEE39AF438CE87BF8DBE1375106AC9844@ph-svr-exch1.wrtdesign.com>

Updated code, still not working properly:

#! /usr/bin/env python

import Milter
import StringIO
import email
import email.Message
import hashlib
import mime
import os
import sys
import tempfile
import time
import rfc822


from email import Errors
from email.Message import Message

## ==  IP Information
from socket import AF_INET, AF_INET6
## ==

## === Define multiprocessing == ##

if True:
    from multiprocessing import Process as Thread, Queue
else:
    from threading import Thread
    from Queue import Queue

logq = Queue(maxsize=4)


def background():
    while True:
        t = logq.get()
        if not t: break
        msg,id,ts = t
        print "%s [%d]" % (time.strftime('%Y%b%d
%H:%M:%S',time.localtime(ts)),id),
        # 2005Oct13 02:34:11 [1] msg1 msg2 msg3 ...
        for i in msg: print i,
        print

## === End Define Multiprocesing === ##

class mltr_SaveAttachments(Milter.Base):
    
    def __init__(self):
        self.id = Milter.uniqueID()

    def close(self):
        # always called, even when abort is called.  Clean up
        # any external resources here.
        return Milter.CONTINUE

    def abort(self):
        # client disconnected prematurely
        return Milter.CONTINUE

    def log(self,*msg):
        logq.put((msg,self.id,time.time()))
    
    @Milter.noreply
    def connect(self, IPname, family, hostaddr):
        self.IP = hostaddr[0]
        self.port = hostaddr[1]
        if family == AF_INET6:
            self.flow = hostaddr[2]
            self.scope = hostaddr[3]
        else:
            self.flow = None
            self.scope = None
        self.IPname = IPname # Name from a reverse IP lookup
        self.H = None
        self.fp = None
        self.receiver = self.getsymval('j')
        self.log("connect from %s at %s" % (IPname, hostaddr) )
        return Milter.CONTINUE

    @Milter.noreply
    def header(self, name, hval):
        self.fp.write("%s: %s\n" % (name,hval))	# add header to buffer
        return Milter.CONTINUE
    
    @Milter.noreply
    def eoh(self):
        self.fp.write("\n")				# terminate
headers
        return Milter.CONTINUE
    
    @Milter.noreply
    def body(self,chunk):
        self.fp.write(chunk)
        return Milter.CONTINUE
    

    def envfrom(self,mailfrom,*str):
#        self.log("envfrom")
        # self.F = mailfrom
        # self.R = []
        # self.fromparms = Milter.dictfromlist(str)
        # self.user = self.getsymval('{auth_authen}')
        # self.log("mail from:", mailfrom, *str)
        self.fp = StringIO.StringIO()
        # self.canon_from = '@'.join(parse_addr(mailfrom))
        # self.fp.write('From %s %s\n' % (self.canon_from,time,ctime()))
        return Milter.CONTINUE


    def attachment(self):
        msg = self._msg
        attachDir = attach_dir(msg)
        removedParts = []
        payload = []
        
        
        for part in msg.walk():
            fname = ""
            
            if part.is_multipart():
                continue
            
            dtypes = part.get_params(None, 'Content-Disposition')

            if not dtypes:
                if part.get_content_type() == 'text/plain':
                    payload.append(part)
                    continue
                ctypes = part.getparams()
                if not ctypes:
                    continue
                for key,val in ctypes:
                    if key.lower() == 'name':
                        fname = val
            else:
                for key,val in dtypes:
                    if key.lower() == 'filename':
                        fname = val
                        
            if fname:
                removedParts.append(fname)
                data = part.get_payload(decode=1)
                extract_attachment(data, attachDir, fname)
                part = self.delete_attachments(part, fname)
                payload.append(part)

#        self.log(payload)
        del msg["content-type"]
        del msg["content-disposition"]
        del msg["content-transfer-encoding"]

        msg.set_type('plain/txt')
        msg.set_payload(payload)

        self._msg = msg

        return Milter.CONTINUE
        
    def delete_attachments(self, part,fname):
        for key,value in part.get_params():
            part.del_param(key)
            
        part.set_payload('[DELETED]\n')
        del part["content-type"]
        del part["content-disposition"]
        del part["content-transfer-encoding"]
        part["Content-Type"] = "text/plain, name="+fname+".TXT"
        return part


    def eom(self):
        self.fp.seek(0)
        self._msg = mime.message_from_file(self.fp)
        self.attachment()
        
        self.log("### MESSAGE ###")
        self.log(self._msg)

        return Milter.ACCEPT
#        return Milter.TEMPFAIL
## ===





def attach_dir(msg):
    tempname = fname = tempfile.mktemp(".tmp")
    out = tempfile.TemporaryFile()
    msg.dump(out)
    out.seek(0)
    buf = out.read()
    hashDir = hashit(buf)
    attachDir = dropDir + hashDir
    
    if not os.path.isdir(hashDir):
        os.mkdir(attachDir)

    return attachDir


def extract_attachment(data, attachDir, fname):
    exdir_file = attachDir + "/" + fname
    extracted = open(exdir_file, "wb")
    extracted.write(data)
    extracted.close()
    


def hashit(data):
    sha1 = hashlib.sha1()
    # f = open(filepath, 'rb')
    # try:
    #     sha1.update(f.read())
    # finally:
    #     f.close()
    sha1.update(data)
       
    return sha1.hexdigest()

dropDir = "/dropdir/"

def main():
    bt = Thread(target=background)
    bt.start()
    socketname = "/tmp/py_testmilter.sock"
    timeout = 600
    Milter.factory = mltr_SaveAttachments
    flags = Milter.CHGBODY + Milter.CHGHDRS + Milter.ADDHDRS
    flags += Milter.ADDRCPT
    flags += Milter.DELRCPT
    Milter.set_flags(flags)     # tell Sendmail/Postfix which features
we use
    print "%s milter startup" % time.strftime('%Y%b%d %H:%M:%S')
    sys.stdout.flush()
    Milter.runmilter("py_testmilter",socketname,timeout)
    logq.put(None)
    bt.join()
    print "%s milter shutdown" % time.strftime('%Y%b%d %H:%M:%S')

if __name__ == "__main__":
    main()

Larry G. Wapnitsky MBA, MCSE, MCP+I
IT SUPPORT & ADMINISTRATION COORDINATOR
WRT  |  Wallace Roberts & Todd, LLC
1700 Market Street, 28th Fl
Philadelphia, PA 19103
T  215.430.5068
C  215.713.8635
E  lwapnitsky at WRTdesign.com


-----Original Message-----
From: Larry G. Wapnitsky 
Sent: Wednesday, November 23, 2011 10:38 AM
To: 'Stuart D. Gathman'
Cc: pymilter at bmsi.com
Subject: RE: [Pymilter] Emulating another MimeDefang function

So, I was able to remove the attachments in a function I've written, but
when the message gets sent, the attachments are still there rather than
my modified message.  Not sure what I'm missing.

Code below:


#! /usr/bin/env python

import Milter
import StringIO
import email
import email.Message
import hashlib
import mime
import os
import sys
import tempfile
import time
import rfc822


from email import Errors
from email.Message import Message

## ==  IP Information
from socket import AF_INET, AF_INET6
## ==

## === Define multiprocessing == ##

if True:
    from multiprocessing import Process as Thread, Queue
else:
    from threading import Thread
    from Queue import Queue

logq = Queue(maxsize=4)


def background():
    while True:
        t = logq.get()
        if not t: break
        msg,id,ts = t
        print "%s [%d]" % (time.strftime('%Y%b%d
%H:%M:%S',time.localtime(ts)),id),
        # 2005Oct13 02:34:11 [1] msg1 msg2 msg3 ...
        for i in msg: print i,
        print

## === End Define Multiprocesing === ##

class mltr_SaveAttachments(Milter.Base):

    def __init__(self):
        self.id = Milter.uniqueID()

    def log(self,*msg):
        logq.put((msg,self.id,time.time()))

    def connect(self, IPname, family, hostaddr):
        self.IP = hostaddr[0]
        self.port = hostaddr[1]
        if family == AF_INET6:
            self.flow = hostaddr[2]
            self.scope = hostaddr[3]
        else:
            self.flow = None
            self.scope = None
        self.IPname = IPname # Name from a reverse IP lookup
        self.H = None
        self.fp = None
        self.receiver = self.getsymval('j')
        self.log("connect from %s at %s" % (IPname, hostaddr) )
        return Milter.CONTINUE

    @Milter.noreply
    def header(self, name, hval):
        self.fp.write("%s: %s\n" % (name,hval)) # add header to buffer
        return Milter.CONTINUE

    @Milter.noreply
    def eoh(self):
        self.fp.write("\n")                             # terminate
headers
        return Milter.CONTINUE



    def envfrom(self,mailfrom,*str):
#        self.log("envfrom")
        # self.F = mailfrom
        # self.R = []
        # self.fromparms = Milter.dictfromlist(str)
        # self.user = self.getsymval('{auth_authen}')
        # self.log("mail from:", mailfrom, *str)
        self.fp = StringIO.StringIO()
        # self.canon_from = '@'.join(parse_addr(mailfrom))
        # self.fp.write('From %s %s\n' % (self.canon_from,time,ctime()))
        return Milter.CONTINUE


    def attachment(self):
        msg = self._msg
        attachDir = attach_dir(msg)
        removedParts = []

        for part in msg.walk():
            fname = ""

            if part.is_multipart():
                continue

            dtypes = part.get_params(None, 'Content-Disposition')

            if not dtypes:
                if part.get_content_type() == 'text/plain':
                    continue
                ctypes = part.getparams()
                if not ctypes:
                    continue
                for key,val in ctypes:
                    if key.lower() == 'name':
                        fname = val
            else:
                for key,val in dtypes:
                    if key.lower() == 'filename':
                        fname = val

            if fname:
                removedParts.append(fname)
                data = part.get_payload(decode=1)
                extract_attachment(data, attachDir, fname)
                part = delete_attachments(part)

        self.log(removedParts)
#        return Milter.CONTINUE
        return msg


    def eom(self):
        self.fp.seek(0)
        self._msg = mime.message_from_file(self.fp)
        self._msg = self.attachment()

        self.log(self._msg)
        return Milter.ACCEPT
#        return Milter.TEMPFAIL
## ===


def delete_attachments(part):
    for key,value in part.get_params():
        part.del_param(key)

    part.set_payload('[DELETED]\n')
    part.set_type('text/plain')
    del part['Content-Disposition']

    return part



def attach_dir(msg):
    tempname = fname = tempfile.mktemp(".tmp")
    out = tempfile.TemporaryFile()
    msg.dump(out)
    out.seek(0)
    buf = out.read()
    hashDir = hashit(buf)
    attachDir = dropDir + hashDir

    if not os.path.isdir(hashDir):
        os.mkdir(attachDir)

    return attachDir


def extract_attachment(data, attachDir, fname):
    exdir_file = attachDir + "/" + fname
    extracted = open(exdir_file, "wb")
    extracted.write(data)
    extracted.close()



def hashit(data):
    sha1 = hashlib.sha1()
    # f = open(filepath, 'rb')
    # try:
    #     sha1.update(f.read())
    # finally:
    #     f.close()
    sha1.update(data)

    return sha1.hexdigest()

dropDir = "/dropdir/"

def main():
    bt = Thread(target=background)
    bt.start()
    socketname = "/tmp/py_testmilter.sock"
    timeout = 600
    Milter.factory = mltr_SaveAttachments
    flags = Milter.CHGBODY + Milter.CHGHDRS + Milter.ADDHDRS
    flags += Milter.ADDRCPT
    flags += Milter.DELRCPT
    Milter.set_flags(flags)     # tell Sendmail/Postfix which features
we use
    print "%s milter startup" % time.strftime('%Y%b%d %H:%M:%S')
    sys.stdout.flush()
    Milter.runmilter("py_testmilter",socketname,timeout)
    logq.put(None)
    bt.join()
    print "%s bms milter shutdown" % time.strftime('%Y%b%d %H:%M:%S')

if __name__ == "__main__":
    main()



-----Original Message-----
From: Stuart D. Gathman [mailto:stuart at bmsi.com] 
Sent: Monday, November 21, 2011 3:43 PM
To: Larry G. Wapnitsky
Cc: pymilter at bmsi.com
Subject: RE: [Pymilter] Emulating another MimeDefang function

On Mon, 21 Nov 2011, Larry G. Wapnitsky wrote:

> Will try this.

If you were curious about the standard place for filenames (as opposed
the many places LookOut checks), look at the get_filename() method in
email.Message (python comes with source).  It looks first in the
Content-Disposition header field for the filename attribute (the
"correct"
place), and secondly in the name attribute of Content-Type.

--
 	      Stuart D. Gathman <stuart at bmsi.com>
     Business Management Systems Inc.  Phone: 703 591-0911 Fax: 703
591-6154 "Confutatis maledictis, flammis acribus addictis" - background
song for a Microsoft sponsored "Where do you want to go from here?"
commercial.




----wrtmail--%3423wrt%----

<!DSPAM:133D14A27F324454233173089>



From daniele at grinta.net  Wed Nov 23 11:59:29 2011
From: daniele at grinta.net (Daniele Nicolodi)
Date: Wed, 23 Nov 2011 17:59:29 +0100
Subject: [Pymilter] Emulating another MimeDefang function
In-Reply-To: <FAE5CA7BEE39AF438CE87BF8DBE1375106AC9844@ph-svr-exch1.wrtdesign.com>
References: <FAE5CA7BEE39AF438CE87BF8DBE1375106AC9834@ph-svr-exch1.wrtdesign.com>	<alpine.LRH.2.00.1111211255050.2441@bmsred.bmsi.com>	<FAE5CA7BEE39AF438CE87BF8DBE1375106AC9837@ph-svr-exch1.wrtdesign.com>	<alpine.LRH.2.00.1111211459001.4409@bmsred.bmsi.com>	<FAE5CA7BEE39AF438CE87BF8DBE1375105ECF994@ph-svr-exch1.wrtdesign.com>	<alpine.LRH.2.00.1111211517250.5335@bmsred.bmsi.com>	<FAE5CA7BEE39AF438CE87BF8DBE1375106AC9839@ph-svr-exch1.wrtdesign.com>	<alpine.LRH.2.00.1111211538270.5725@bmsred.bmsi.com>	<FAE5CA7BEE39AF438CE87BF8DBE1375106AC9843@ph-svr-exch1.wrtdesign.com>
	<FAE5CA7BEE39AF438CE87BF8DBE1375106AC9844@ph-svr-exch1.wrtdesign.com>
Message-ID: <4ECD2671.1060805@grinta.net>

On 23/11/11 17:37, Larry G. Wapnitsky wrote:
> Updated code, still not working properly:

Where are you telling your SMTP to replace the body of the message?

Cheers,
-- 
Daniele


From LWapnitsky at ph.wrtdesign.com  Wed Nov 23 12:03:46 2011
From: LWapnitsky at ph.wrtdesign.com (Larry G. Wapnitsky)
Date: Wed, 23 Nov 2011 12:03:46 -0500
Subject: [Pymilter] RE: Pymilter Digest, Vol 12, Issue 7
In-Reply-To: <201111231700.pANH06qa001178@mail.bmsi.com>
References: <201111231700.pANH06qa001178@mail.bmsi.com>
Message-ID: <FAE5CA7BEE39AF438CE87BF8DBE1375106AC9845@ph-svr-exch1.wrtdesign.com>

Daniele-

I think that's what I'm missing (btw - I've moved a little further since
our last conversation...thanks for the tips :))

Larry G. Wapnitsky MBA, MCSE, MCP+I
IT SUPPORT & ADMINISTRATION COORDINATOR
WRT  |  Wallace Roberts & Todd, LLC
1700 Market Street, 28th Fl
Philadelphia, PA 19103
T  215.430.5068
C  215.713.8635
E  lwapnitsky at WRTdesign.com


-----Original Message-----
From: pymilter-bounces at bmsi.com [mailto:pymilter-bounces at bmsi.com] On
Behalf Of pymilter-request at bmsi.com
Sent: Wednesday, November 23, 2011 12:00 PM
To: pymilter at bmsi.com
Subject: Pymilter Digest, Vol 12, Issue 7

Send Pymilter mailing list submissions to
	pymilter at bmsi.com

To subscribe or unsubscribe via the World Wide Web, visit
	http://www.bmsi.com/mailman/listinfo/pymilter
or, via email, send a message with subject or body 'help' to
	pymilter-request at bmsi.com

You can reach the person managing the list at
	pymilter-owner at bmsi.com

When replying, please edit your Subject line so it is more specific than
"Re: Contents of Pymilter digest..."


Today's Topics:

   1. RE: Emulating another MimeDefang function (Larry G. Wapnitsky)
   2. RE: Emulating another MimeDefang function (Larry G. Wapnitsky)
   3. Re: Emulating another MimeDefang function (Daniele Nicolodi)


----------------------------------------------------------------------

Message: 1
Date: Wed, 23 Nov 2011 10:38:22 -0500
From: "Larry G. Wapnitsky" <LWapnitsky at ph.wrtdesign.com>
Subject: RE: [Pymilter] Emulating another MimeDefang function
To: "Stuart D. Gathman" <stuart at bmsi.com>
Cc: pymilter at bmsi.com
Message-ID:
	
<FAE5CA7BEE39AF438CE87BF8DBE1375106AC9843 at ph-svr-exch1.wrtdesign.com>
Content-Type: text/plain;	charset="us-ascii"

So, I was able to remove the attachments in a function I've written, but
when the message gets sent, the attachments are still there rather than
my modified message.  Not sure what I'm missing.

Code below:


#! /usr/bin/env python

import Milter
import StringIO
import email
import email.Message
import hashlib
import mime
import os
import sys
import tempfile
import time
import rfc822


from email import Errors
from email.Message import Message

## ==  IP Information
from socket import AF_INET, AF_INET6
## ==

## === Define multiprocessing == ##

if True:
    from multiprocessing import Process as Thread, Queue
else:
    from threading import Thread
    from Queue import Queue

logq = Queue(maxsize=4)


def background():
    while True:
        t = logq.get()
        if not t: break
        msg,id,ts = t
        print "%s [%d]" % (time.strftime('%Y%b%d
%H:%M:%S',time.localtime(ts)),id),
        # 2005Oct13 02:34:11 [1] msg1 msg2 msg3 ...
        for i in msg: print i,
        print

## === End Define Multiprocesing === ##

class mltr_SaveAttachments(Milter.Base):

    def __init__(self):
        self.id = Milter.uniqueID()

    def log(self,*msg):
        logq.put((msg,self.id,time.time()))

    def connect(self, IPname, family, hostaddr):
        self.IP = hostaddr[0]
        self.port = hostaddr[1]
        if family == AF_INET6:
            self.flow = hostaddr[2]
            self.scope = hostaddr[3]
        else:
            self.flow = None
            self.scope = None
        self.IPname = IPname # Name from a reverse IP lookup
        self.H = None
        self.fp = None
        self.receiver = self.getsymval('j')
        self.log("connect from %s at %s" % (IPname, hostaddr) )
        return Milter.CONTINUE

    @Milter.noreply
    def header(self, name, hval):
        self.fp.write("%s: %s\n" % (name,hval)) # add header to buffer
        return Milter.CONTINUE

    @Milter.noreply
    def eoh(self):
        self.fp.write("\n")                             # terminate
headers
        return Milter.CONTINUE



    def envfrom(self,mailfrom,*str):
#        self.log("envfrom")
        # self.F = mailfrom
        # self.R = []
        # self.fromparms = Milter.dictfromlist(str)
        # self.user = self.getsymval('{auth_authen}')
        # self.log("mail from:", mailfrom, *str)
        self.fp = StringIO.StringIO()
        # self.canon_from = '@'.join(parse_addr(mailfrom))
        # self.fp.write('From %s %s\n' % (self.canon_from,time,ctime()))
        return Milter.CONTINUE


    def attachment(self):
        msg = self._msg
        attachDir = attach_dir(msg)
        removedParts = []

        for part in msg.walk():
            fname = ""

            if part.is_multipart():
                continue

            dtypes = part.get_params(None, 'Content-Disposition')

            if not dtypes:
                if part.get_content_type() == 'text/plain':
                    continue
                ctypes = part.getparams()
                if not ctypes:
                    continue
                for key,val in ctypes:
                    if key.lower() == 'name':
                        fname = val
            else:
                for key,val in dtypes:
                    if key.lower() == 'filename':
                        fname = val

            if fname:
                removedParts.append(fname)
                data = part.get_payload(decode=1)
                extract_attachment(data, attachDir, fname)
                part = delete_attachments(part)

        self.log(removedParts)
#        return Milter.CONTINUE
        return msg


    def eom(self):
        self.fp.seek(0)
        self._msg = mime.message_from_file(self.fp)
        self._msg = self.attachment()

        self.log(self._msg)
        return Milter.ACCEPT
#        return Milter.TEMPFAIL
## ===


def delete_attachments(part):
    for key,value in part.get_params():
        part.del_param(key)

    part.set_payload('[DELETED]\n')
    part.set_type('text/plain')
    del part['Content-Disposition']

    return part



def attach_dir(msg):
    tempname = fname = tempfile.mktemp(".tmp")
    out = tempfile.TemporaryFile()
    msg.dump(out)
    out.seek(0)
    buf = out.read()
    hashDir = hashit(buf)
    attachDir = dropDir + hashDir

    if not os.path.isdir(hashDir):
        os.mkdir(attachDir)

    return attachDir


def extract_attachment(data, attachDir, fname):
    exdir_file = attachDir + "/" + fname
    extracted = open(exdir_file, "wb")
    extracted.write(data)
    extracted.close()



def hashit(data):
    sha1 = hashlib.sha1()
    # f = open(filepath, 'rb')
    # try:
    #     sha1.update(f.read())
    # finally:
    #     f.close()
    sha1.update(data)

    return sha1.hexdigest()

dropDir = "/dropdir/"

def main():
    bt = Thread(target=background)
    bt.start()
    socketname = "/tmp/py_testmilter.sock"
    timeout = 600
    Milter.factory = mltr_SaveAttachments
    flags = Milter.CHGBODY + Milter.CHGHDRS + Milter.ADDHDRS
    flags += Milter.ADDRCPT
    flags += Milter.DELRCPT
    Milter.set_flags(flags)     # tell Sendmail/Postfix which features
we use
    print "%s milter startup" % time.strftime('%Y%b%d %H:%M:%S')
    sys.stdout.flush()
    Milter.runmilter("py_testmilter",socketname,timeout)
    logq.put(None)
    bt.join()
    print "%s bms milter shutdown" % time.strftime('%Y%b%d %H:%M:%S')

if __name__ == "__main__":
    main()



-----Original Message-----
From: Stuart D. Gathman [mailto:stuart at bmsi.com]
Sent: Monday, November 21, 2011 3:43 PM
To: Larry G. Wapnitsky
Cc: pymilter at bmsi.com
Subject: RE: [Pymilter] Emulating another MimeDefang function

On Mon, 21 Nov 2011, Larry G. Wapnitsky wrote:

> Will try this.

If you were curious about the standard place for filenames (as opposed
the many places LookOut checks), look at the get_filename() method in
email.Message (python comes with source).  It looks first in the
Content-Disposition header field for the filename attribute (the
"correct"
place), and secondly in the name attribute of Content-Type.

--
 	      Stuart D. Gathman <stuart at bmsi.com>
     Business Management Systems Inc.  Phone: 703 591-0911 Fax: 703
591-6154 "Confutatis maledictis, flammis acribus addictis" - background
song for a Microsoft sponsored "Where do you want to go from here?"
commercial.




----wrtmail--%3423wrt%----

<!DSPAM:133D113DA422445417198307>



------------------------------

Message: 2
Date: Wed, 23 Nov 2011 11:37:48 -0500
From: "Larry G. Wapnitsky" <LWapnitsky at ph.wrtdesign.com>
Subject: RE: [Pymilter] Emulating another MimeDefang function
To: "Stuart D. Gathman" <stuart at bmsi.com>
Cc: pymilter at bmsi.com
Message-ID:
	
<FAE5CA7BEE39AF438CE87BF8DBE1375106AC9844 at ph-svr-exch1.wrtdesign.com>
Content-Type: text/plain;	charset="us-ascii"

Updated code, still not working properly:

#! /usr/bin/env python

import Milter
import StringIO
import email
import email.Message
import hashlib
import mime
import os
import sys
import tempfile
import time
import rfc822


from email import Errors
from email.Message import Message

## ==  IP Information
from socket import AF_INET, AF_INET6
## ==

## === Define multiprocessing == ##

if True:
    from multiprocessing import Process as Thread, Queue
else:
    from threading import Thread
    from Queue import Queue

logq = Queue(maxsize=4)


def background():
    while True:
        t = logq.get()
        if not t: break
        msg,id,ts = t
        print "%s [%d]" % (time.strftime('%Y%b%d
%H:%M:%S',time.localtime(ts)),id),
        # 2005Oct13 02:34:11 [1] msg1 msg2 msg3 ...
        for i in msg: print i,
        print

## === End Define Multiprocesing === ##

class mltr_SaveAttachments(Milter.Base):
    
    def __init__(self):
        self.id = Milter.uniqueID()

    def close(self):
        # always called, even when abort is called.  Clean up
        # any external resources here.
        return Milter.CONTINUE

    def abort(self):
        # client disconnected prematurely
        return Milter.CONTINUE

    def log(self,*msg):
        logq.put((msg,self.id,time.time()))
    
    @Milter.noreply
    def connect(self, IPname, family, hostaddr):
        self.IP = hostaddr[0]
        self.port = hostaddr[1]
        if family == AF_INET6:
            self.flow = hostaddr[2]
            self.scope = hostaddr[3]
        else:
            self.flow = None
            self.scope = None
        self.IPname = IPname # Name from a reverse IP lookup
        self.H = None
        self.fp = None
        self.receiver = self.getsymval('j')
        self.log("connect from %s at %s" % (IPname, hostaddr) )
        return Milter.CONTINUE

    @Milter.noreply
    def header(self, name, hval):
        self.fp.write("%s: %s\n" % (name,hval))	# add header to buffer
        return Milter.CONTINUE
    
    @Milter.noreply
    def eoh(self):
        self.fp.write("\n")				# terminate
headers
        return Milter.CONTINUE
    
    @Milter.noreply
    def body(self,chunk):
        self.fp.write(chunk)
        return Milter.CONTINUE
    

    def envfrom(self,mailfrom,*str):
#        self.log("envfrom")
        # self.F = mailfrom
        # self.R = []
        # self.fromparms = Milter.dictfromlist(str)
        # self.user = self.getsymval('{auth_authen}')
        # self.log("mail from:", mailfrom, *str)
        self.fp = StringIO.StringIO()
        # self.canon_from = '@'.join(parse_addr(mailfrom))
        # self.fp.write('From %s %s\n' % (self.canon_from,time,ctime()))
        return Milter.CONTINUE


    def attachment(self):
        msg = self._msg
        attachDir = attach_dir(msg)
        removedParts = []
        payload = []
        
        
        for part in msg.walk():
            fname = ""
            
            if part.is_multipart():
                continue
            
            dtypes = part.get_params(None, 'Content-Disposition')

            if not dtypes:
                if part.get_content_type() == 'text/plain':
                    payload.append(part)
                    continue
                ctypes = part.getparams()
                if not ctypes:
                    continue
                for key,val in ctypes:
                    if key.lower() == 'name':
                        fname = val
            else:
                for key,val in dtypes:
                    if key.lower() == 'filename':
                        fname = val
                        
            if fname:
                removedParts.append(fname)
                data = part.get_payload(decode=1)
                extract_attachment(data, attachDir, fname)
                part = self.delete_attachments(part, fname)
                payload.append(part)

#        self.log(payload)
        del msg["content-type"]
        del msg["content-disposition"]
        del msg["content-transfer-encoding"]

        msg.set_type('plain/txt')
        msg.set_payload(payload)

        self._msg = msg

        return Milter.CONTINUE
        
    def delete_attachments(self, part,fname):
        for key,value in part.get_params():
            part.del_param(key)
            
        part.set_payload('[DELETED]\n')
        del part["content-type"]
        del part["content-disposition"]
        del part["content-transfer-encoding"]
        part["Content-Type"] = "text/plain, name="+fname+".TXT"
        return part


    def eom(self):
        self.fp.seek(0)
        self._msg = mime.message_from_file(self.fp)
        self.attachment()
        
        self.log("### MESSAGE ###")
        self.log(self._msg)

        return Milter.ACCEPT
#        return Milter.TEMPFAIL
## ===





def attach_dir(msg):
    tempname = fname = tempfile.mktemp(".tmp")
    out = tempfile.TemporaryFile()
    msg.dump(out)
    out.seek(0)
    buf = out.read()
    hashDir = hashit(buf)
    attachDir = dropDir + hashDir
    
    if not os.path.isdir(hashDir):
        os.mkdir(attachDir)

    return attachDir


def extract_attachment(data, attachDir, fname):
    exdir_file = attachDir + "/" + fname
    extracted = open(exdir_file, "wb")
    extracted.write(data)
    extracted.close()
    


def hashit(data):
    sha1 = hashlib.sha1()
    # f = open(filepath, 'rb')
    # try:
    #     sha1.update(f.read())
    # finally:
    #     f.close()
    sha1.update(data)
       
    return sha1.hexdigest()

dropDir = "/dropdir/"

def main():
    bt = Thread(target=background)
    bt.start()
    socketname = "/tmp/py_testmilter.sock"
    timeout = 600
    Milter.factory = mltr_SaveAttachments
    flags = Milter.CHGBODY + Milter.CHGHDRS + Milter.ADDHDRS
    flags += Milter.ADDRCPT
    flags += Milter.DELRCPT
    Milter.set_flags(flags)     # tell Sendmail/Postfix which features
we use
    print "%s milter startup" % time.strftime('%Y%b%d %H:%M:%S')
    sys.stdout.flush()
    Milter.runmilter("py_testmilter",socketname,timeout)
    logq.put(None)
    bt.join()
    print "%s milter shutdown" % time.strftime('%Y%b%d %H:%M:%S')

if __name__ == "__main__":
    main()

Larry G. Wapnitsky MBA, MCSE, MCP+I
IT SUPPORT & ADMINISTRATION COORDINATOR
WRT  |  Wallace Roberts & Todd, LLC
1700 Market Street, 28th Fl
Philadelphia, PA 19103
T  215.430.5068
C  215.713.8635
E  lwapnitsky at WRTdesign.com


-----Original Message-----
From: Larry G. Wapnitsky
Sent: Wednesday, November 23, 2011 10:38 AM
To: 'Stuart D. Gathman'
Cc: pymilter at bmsi.com
Subject: RE: [Pymilter] Emulating another MimeDefang function

So, I was able to remove the attachments in a function I've written, but
when the message gets sent, the attachments are still there rather than
my modified message.  Not sure what I'm missing.

Code below:


#! /usr/bin/env python

import Milter
import StringIO
import email
import email.Message
import hashlib
import mime
import os
import sys
import tempfile
import time
import rfc822


from email import Errors
from email.Message import Message

## ==  IP Information
from socket import AF_INET, AF_INET6
## ==

## === Define multiprocessing == ##

if True:
    from multiprocessing import Process as Thread, Queue
else:
    from threading import Thread
    from Queue import Queue

logq = Queue(maxsize=4)


def background():
    while True:
        t = logq.get()
        if not t: break
        msg,id,ts = t
        print "%s [%d]" % (time.strftime('%Y%b%d
%H:%M:%S',time.localtime(ts)),id),
        # 2005Oct13 02:34:11 [1] msg1 msg2 msg3 ...
        for i in msg: print i,
        print

## === End Define Multiprocesing === ##

class mltr_SaveAttachments(Milter.Base):

    def __init__(self):
        self.id = Milter.uniqueID()

    def log(self,*msg):
        logq.put((msg,self.id,time.time()))

    def connect(self, IPname, family, hostaddr):
        self.IP = hostaddr[0]
        self.port = hostaddr[1]
        if family == AF_INET6:
            self.flow = hostaddr[2]
            self.scope = hostaddr[3]
        else:
            self.flow = None
            self.scope = None
        self.IPname = IPname # Name from a reverse IP lookup
        self.H = None
        self.fp = None
        self.receiver = self.getsymval('j')
        self.log("connect from %s at %s" % (IPname, hostaddr) )
        return Milter.CONTINUE

    @Milter.noreply
    def header(self, name, hval):
        self.fp.write("%s: %s\n" % (name,hval)) # add header to buffer
        return Milter.CONTINUE

    @Milter.noreply
    def eoh(self):
        self.fp.write("\n")                             # terminate
headers
        return Milter.CONTINUE



    def envfrom(self,mailfrom,*str):
#        self.log("envfrom")
        # self.F = mailfrom
        # self.R = []
        # self.fromparms = Milter.dictfromlist(str)
        # self.user = self.getsymval('{auth_authen}')
        # self.log("mail from:", mailfrom, *str)
        self.fp = StringIO.StringIO()
        # self.canon_from = '@'.join(parse_addr(mailfrom))
        # self.fp.write('From %s %s\n' % (self.canon_from,time,ctime()))
        return Milter.CONTINUE


    def attachment(self):
        msg = self._msg
        attachDir = attach_dir(msg)
        removedParts = []

        for part in msg.walk():
            fname = ""

            if part.is_multipart():
                continue

            dtypes = part.get_params(None, 'Content-Disposition')

            if not dtypes:
                if part.get_content_type() == 'text/plain':
                    continue
                ctypes = part.getparams()
                if not ctypes:
                    continue
                for key,val in ctypes:
                    if key.lower() == 'name':
                        fname = val
            else:
                for key,val in dtypes:
                    if key.lower() == 'filename':
                        fname = val

            if fname:
                removedParts.append(fname)
                data = part.get_payload(decode=1)
                extract_attachment(data, attachDir, fname)
                part = delete_attachments(part)

        self.log(removedParts)
#        return Milter.CONTINUE
        return msg


    def eom(self):
        self.fp.seek(0)
        self._msg = mime.message_from_file(self.fp)
        self._msg = self.attachment()

        self.log(self._msg)
        return Milter.ACCEPT
#        return Milter.TEMPFAIL
## ===


def delete_attachments(part):
    for key,value in part.get_params():
        part.del_param(key)

    part.set_payload('[DELETED]\n')
    part.set_type('text/plain')
    del part['Content-Disposition']

    return part



def attach_dir(msg):
    tempname = fname = tempfile.mktemp(".tmp")
    out = tempfile.TemporaryFile()
    msg.dump(out)
    out.seek(0)
    buf = out.read()
    hashDir = hashit(buf)
    attachDir = dropDir + hashDir

    if not os.path.isdir(hashDir):
        os.mkdir(attachDir)

    return attachDir


def extract_attachment(data, attachDir, fname):
    exdir_file = attachDir + "/" + fname
    extracted = open(exdir_file, "wb")
    extracted.write(data)
    extracted.close()



def hashit(data):
    sha1 = hashlib.sha1()
    # f = open(filepath, 'rb')
    # try:
    #     sha1.update(f.read())
    # finally:
    #     f.close()
    sha1.update(data)

    return sha1.hexdigest()

dropDir = "/dropdir/"

def main():
    bt = Thread(target=background)
    bt.start()
    socketname = "/tmp/py_testmilter.sock"
    timeout = 600
    Milter.factory = mltr_SaveAttachments
    flags = Milter.CHGBODY + Milter.CHGHDRS + Milter.ADDHDRS
    flags += Milter.ADDRCPT
    flags += Milter.DELRCPT
    Milter.set_flags(flags)     # tell Sendmail/Postfix which features
we use
    print "%s milter startup" % time.strftime('%Y%b%d %H:%M:%S')
    sys.stdout.flush()
    Milter.runmilter("py_testmilter",socketname,timeout)
    logq.put(None)
    bt.join()
    print "%s bms milter shutdown" % time.strftime('%Y%b%d %H:%M:%S')

if __name__ == "__main__":
    main()



-----Original Message-----
From: Stuart D. Gathman [mailto:stuart at bmsi.com]
Sent: Monday, November 21, 2011 3:43 PM
To: Larry G. Wapnitsky
Cc: pymilter at bmsi.com
Subject: RE: [Pymilter] Emulating another MimeDefang function

On Mon, 21 Nov 2011, Larry G. Wapnitsky wrote:

> Will try this.

If you were curious about the standard place for filenames (as opposed
the many places LookOut checks), look at the get_filename() method in
email.Message (python comes with source).  It looks first in the
Content-Disposition header field for the filename attribute (the
"correct"
place), and secondly in the name attribute of Content-Type.

--
 	      Stuart D. Gathman <stuart at bmsi.com>
     Business Management Systems Inc.  Phone: 703 591-0911 Fax: 703
591-6154 "Confutatis maledictis, flammis acribus addictis" - background
song for a Microsoft sponsored "Where do you want to go from here?"
commercial.




----wrtmail--%3423wrt%----

<!DSPAM:133D14A27F324454233173089>



------------------------------

Message: 3
Date: Wed, 23 Nov 2011 17:59:29 +0100
From: Daniele Nicolodi <daniele at grinta.net>
Subject: Re: [Pymilter] Emulating another MimeDefang function
To: pymilter at bmsi.com
Message-ID: <4ECD2671.1060805 at grinta.net>
Content-Type: text/plain; charset=ISO-8859-1

On 23/11/11 17:37, Larry G. Wapnitsky wrote:
> Updated code, still not working properly:

Where are you telling your SMTP to replace the body of the message?

Cheers,
--
Daniele


------------------------------

_______________________________________________
Pymilter mailing list
Pymilter at bmsi.com
http://www.bmsi.com/mailman/listinfo/pymilter


End of Pymilter Digest, Vol 12, Issue 7
***************************************




----wrtmail--%3423wrt%----


From daniele at grinta.net  Wed Nov 23 12:15:32 2011
From: daniele at grinta.net (Daniele Nicolodi)
Date: Wed, 23 Nov 2011 18:15:32 +0100
Subject: [Pymilter] RE: Pymilter Digest, Vol 12, Issue 7
In-Reply-To: <FAE5CA7BEE39AF438CE87BF8DBE1375106AC9845@ph-svr-exch1.wrtdesign.com>
References: <201111231700.pANH06qa001178@mail.bmsi.com>
	<FAE5CA7BEE39AF438CE87BF8DBE1375106AC9845@ph-svr-exch1.wrtdesign.com>
Message-ID: <4ECD2A34.3080800@grinta.net>

On 23/11/11 18:03, Larry G. Wapnitsky wrote:
> Daniele-
> 
> I think that's what I'm missing (btw - I've moved a little further since
> our last conversation...thanks for the tips :))

What about reading the documentation then?

http://spidey2.bmsi.com/pymilter/classMilter_1_1Base.html#a436398d2ba1e122c2022b06654fcbb37

Cheers,
-- 
Daniele


From LWapnitsky at ph.wrtdesign.com  Fri Nov 25 08:12:19 2011
From: LWapnitsky at ph.wrtdesign.com (Larry G. Wapnitsky)
Date: Fri, 25 Nov 2011 08:12:19 -0500
Subject: [Pymilter] RE: PEmulating another MimeDefang function
Message-ID: <FAE5CA7BEE39AF438CE87BF8DBE1375106AC9848@ph-svr-exch1.wrtdesign.com>

Update - I've managed two separate things over the past few days, but
neither is accomplishing exactly what I need.

1)  I can get the body back along with the attachments, but no matter
what I define as the attachment name, Outlook (or any other apps) sees
them as ATT#####.txt

2)  I can't seem to keep the formatting of the original message.


Larry G. Wapnitsky MBA, MCSE, MCP+I
IT SUPPORT & ADMINISTRATION COORDINATOR
WRT  |  Wallace Roberts & Todd, LLC
1700 Market Street, 28th Fl
Philadelphia, PA 19103
T  215.430.5068
C  215.713.8635
E  lwapnitsky at WRTdesign.com


-----Original Message-----
From: Larry G. Wapnitsky 
Sent: Wednesday, November 23, 2011 12:04 PM
To: 'pymilter at bmsi.com'
Subject: RE: Pymilter Digest, Vol 12, Issue 7

Daniele-

I think that's what I'm missing (btw - I've moved a little further since
our last conversation...thanks for the tips :))

Larry G. Wapnitsky MBA, MCSE, MCP+I
IT SUPPORT & ADMINISTRATION COORDINATOR
WRT  |  Wallace Roberts & Todd, LLC
1700 Market Street, 28th Fl
Philadelphia, PA 19103
T  215.430.5068
C  215.713.8635
E  lwapnitsky at WRTdesign.com


-----Original Message-----
From: pymilter-bounces at bmsi.com [mailto:pymilter-bounces at bmsi.com] On
Behalf Of pymilter-request at bmsi.com
Sent: Wednesday, November 23, 2011 12:00 PM
To: pymilter at bmsi.com
Subject: Pymilter Digest, Vol 12, Issue 7

Send Pymilter mailing list submissions to
	pymilter at bmsi.com

To subscribe or unsubscribe via the World Wide Web, visit
	http://www.bmsi.com/mailman/listinfo/pymilter
or, via email, send a message with subject or body 'help' to
	pymilter-request at bmsi.com

You can reach the person managing the list at
	pymilter-owner at bmsi.com

When replying, please edit your Subject line so it is more specific than
"Re: Contents of Pymilter digest..."


Today's Topics:

   1. RE: Emulating another MimeDefang function (Larry G. Wapnitsky)
   2. RE: Emulating another MimeDefang function (Larry G. Wapnitsky)
   3. Re: Emulating another MimeDefang function (Daniele Nicolodi)


----------------------------------------------------------------------

Message: 1
Date: Wed, 23 Nov 2011 10:38:22 -0500
From: "Larry G. Wapnitsky" <LWapnitsky at ph.wrtdesign.com>
Subject: RE: [Pymilter] Emulating another MimeDefang function
To: "Stuart D. Gathman" <stuart at bmsi.com>
Cc: pymilter at bmsi.com
Message-ID:
	
<FAE5CA7BEE39AF438CE87BF8DBE1375106AC9843 at ph-svr-exch1.wrtdesign.com>
Content-Type: text/plain;	charset="us-ascii"

So, I was able to remove the attachments in a function I've written, but
when the message gets sent, the attachments are still there rather than
my modified message.  Not sure what I'm missing.

Code below:


#! /usr/bin/env python

import Milter
import StringIO
import email
import email.Message
import hashlib
import mime
import os
import sys
import tempfile
import time
import rfc822


from email import Errors
from email.Message import Message

## ==  IP Information
from socket import AF_INET, AF_INET6
## ==

## === Define multiprocessing == ##

if True:
    from multiprocessing import Process as Thread, Queue
else:
    from threading import Thread
    from Queue import Queue

logq = Queue(maxsize=4)


def background():
    while True:
        t = logq.get()
        if not t: break
        msg,id,ts = t
        print "%s [%d]" % (time.strftime('%Y%b%d
%H:%M:%S',time.localtime(ts)),id),
        # 2005Oct13 02:34:11 [1] msg1 msg2 msg3 ...
        for i in msg: print i,
        print

## === End Define Multiprocesing === ##

class mltr_SaveAttachments(Milter.Base):

    def __init__(self):
        self.id = Milter.uniqueID()

    def log(self,*msg):
        logq.put((msg,self.id,time.time()))

    def connect(self, IPname, family, hostaddr):
        self.IP = hostaddr[0]
        self.port = hostaddr[1]
        if family == AF_INET6:
            self.flow = hostaddr[2]
            self.scope = hostaddr[3]
        else:
            self.flow = None
            self.scope = None
        self.IPname = IPname # Name from a reverse IP lookup
        self.H = None
        self.fp = None
        self.receiver = self.getsymval('j')
        self.log("connect from %s at %s" % (IPname, hostaddr) )
        return Milter.CONTINUE

    @Milter.noreply
    def header(self, name, hval):
        self.fp.write("%s: %s\n" % (name,hval)) # add header to buffer
        return Milter.CONTINUE

    @Milter.noreply
    def eoh(self):
        self.fp.write("\n")                             # terminate
headers
        return Milter.CONTINUE



    def envfrom(self,mailfrom,*str):
#        self.log("envfrom")
        # self.F = mailfrom
        # self.R = []
        # self.fromparms = Milter.dictfromlist(str)
        # self.user = self.getsymval('{auth_authen}')
        # self.log("mail from:", mailfrom, *str)
        self.fp = StringIO.StringIO()
        # self.canon_from = '@'.join(parse_addr(mailfrom))
        # self.fp.write('From %s %s\n' % (self.canon_from,time,ctime()))
        return Milter.CONTINUE


    def attachment(self):
        msg = self._msg
        attachDir = attach_dir(msg)
        removedParts = []

        for part in msg.walk():
            fname = ""

            if part.is_multipart():
                continue

            dtypes = part.get_params(None, 'Content-Disposition')

            if not dtypes:
                if part.get_content_type() == 'text/plain':
                    continue
                ctypes = part.getparams()
                if not ctypes:
                    continue
                for key,val in ctypes:
                    if key.lower() == 'name':
                        fname = val
            else:
                for key,val in dtypes:
                    if key.lower() == 'filename':
                        fname = val

            if fname:
                removedParts.append(fname)
                data = part.get_payload(decode=1)
                extract_attachment(data, attachDir, fname)
                part = delete_attachments(part)

        self.log(removedParts)
#        return Milter.CONTINUE
        return msg


    def eom(self):
        self.fp.seek(0)
        self._msg = mime.message_from_file(self.fp)
        self._msg = self.attachment()

        self.log(self._msg)
        return Milter.ACCEPT
#        return Milter.TEMPFAIL
## ===


def delete_attachments(part):
    for key,value in part.get_params():
        part.del_param(key)

    part.set_payload('[DELETED]\n')
    part.set_type('text/plain')
    del part['Content-Disposition']

    return part



def attach_dir(msg):
    tempname = fname = tempfile.mktemp(".tmp")
    out = tempfile.TemporaryFile()
    msg.dump(out)
    out.seek(0)
    buf = out.read()
    hashDir = hashit(buf)
    attachDir = dropDir + hashDir

    if not os.path.isdir(hashDir):
        os.mkdir(attachDir)

    return attachDir


def extract_attachment(data, attachDir, fname):
    exdir_file = attachDir + "/" + fname
    extracted = open(exdir_file, "wb")
    extracted.write(data)
    extracted.close()



def hashit(data):
    sha1 = hashlib.sha1()
    # f = open(filepath, 'rb')
    # try:
    #     sha1.update(f.read())
    # finally:
    #     f.close()
    sha1.update(data)

    return sha1.hexdigest()

dropDir = "/dropdir/"

def main():
    bt = Thread(target=background)
    bt.start()
    socketname = "/tmp/py_testmilter.sock"
    timeout = 600
    Milter.factory = mltr_SaveAttachments
    flags = Milter.CHGBODY + Milter.CHGHDRS + Milter.ADDHDRS
    flags += Milter.ADDRCPT
    flags += Milter.DELRCPT
    Milter.set_flags(flags)     # tell Sendmail/Postfix which features
we use
    print "%s milter startup" % time.strftime('%Y%b%d %H:%M:%S')
    sys.stdout.flush()
    Milter.runmilter("py_testmilter",socketname,timeout)
    logq.put(None)
    bt.join()
    print "%s bms milter shutdown" % time.strftime('%Y%b%d %H:%M:%S')

if __name__ == "__main__":
    main()



-----Original Message-----
From: Stuart D. Gathman [mailto:stuart at bmsi.com]
Sent: Monday, November 21, 2011 3:43 PM
To: Larry G. Wapnitsky
Cc: pymilter at bmsi.com
Subject: RE: [Pymilter] Emulating another MimeDefang function

On Mon, 21 Nov 2011, Larry G. Wapnitsky wrote:

> Will try this.

If you were curious about the standard place for filenames (as opposed
the many places LookOut checks), look at the get_filename() method in
email.Message (python comes with source).  It looks first in the
Content-Disposition header field for the filename attribute (the
"correct"
place), and secondly in the name attribute of Content-Type.

--
 	      Stuart D. Gathman <stuart at bmsi.com>
     Business Management Systems Inc.  Phone: 703 591-0911 Fax: 703
591-6154 "Confutatis maledictis, flammis acribus addictis" - background
song for a Microsoft sponsored "Where do you want to go from here?"
commercial.




----wrtmail--%3423wrt%----

<!DSPAM:133D113DA422445417198307>



------------------------------

Message: 2
Date: Wed, 23 Nov 2011 11:37:48 -0500
From: "Larry G. Wapnitsky" <LWapnitsky at ph.wrtdesign.com>
Subject: RE: [Pymilter] Emulating another MimeDefang function
To: "Stuart D. Gathman" <stuart at bmsi.com>
Cc: pymilter at bmsi.com
Message-ID:
	
<FAE5CA7BEE39AF438CE87BF8DBE1375106AC9844 at ph-svr-exch1.wrtdesign.com>
Content-Type: text/plain;	charset="us-ascii"

Updated code, still not working properly:

#! /usr/bin/env python

import Milter
import StringIO
import email
import email.Message
import hashlib
import mime
import os
import sys
import tempfile
import time
import rfc822


from email import Errors
from email.Message import Message

## ==  IP Information
from socket import AF_INET, AF_INET6
## ==

## === Define multiprocessing == ##

if True:
    from multiprocessing import Process as Thread, Queue
else:
    from threading import Thread
    from Queue import Queue

logq = Queue(maxsize=4)


def background():
    while True:
        t = logq.get()
        if not t: break
        msg,id,ts = t
        print "%s [%d]" % (time.strftime('%Y%b%d
%H:%M:%S',time.localtime(ts)),id),
        # 2005Oct13 02:34:11 [1] msg1 msg2 msg3 ...
        for i in msg: print i,
        print

## === End Define Multiprocesing === ##

class mltr_SaveAttachments(Milter.Base):
    
    def __init__(self):
        self.id = Milter.uniqueID()

    def close(self):
        # always called, even when abort is called.  Clean up
        # any external resources here.
        return Milter.CONTINUE

    def abort(self):
        # client disconnected prematurely
        return Milter.CONTINUE

    def log(self,*msg):
        logq.put((msg,self.id,time.time()))
    
    @Milter.noreply
    def connect(self, IPname, family, hostaddr):
        self.IP = hostaddr[0]
        self.port = hostaddr[1]
        if family == AF_INET6:
            self.flow = hostaddr[2]
            self.scope = hostaddr[3]
        else:
            self.flow = None
            self.scope = None
        self.IPname = IPname # Name from a reverse IP lookup
        self.H = None
        self.fp = None
        self.receiver = self.getsymval('j')
        self.log("connect from %s at %s" % (IPname, hostaddr) )
        return Milter.CONTINUE

    @Milter.noreply
    def header(self, name, hval):
        self.fp.write("%s: %s\n" % (name,hval))	# add header to buffer
        return Milter.CONTINUE
    
    @Milter.noreply
    def eoh(self):
        self.fp.write("\n")				# terminate
headers
        return Milter.CONTINUE
    
    @Milter.noreply
    def body(self,chunk):
        self.fp.write(chunk)
        return Milter.CONTINUE
    

    def envfrom(self,mailfrom,*str):
#        self.log("envfrom")
        # self.F = mailfrom
        # self.R = []
        # self.fromparms = Milter.dictfromlist(str)
        # self.user = self.getsymval('{auth_authen}')
        # self.log("mail from:", mailfrom, *str)
        self.fp = StringIO.StringIO()
        # self.canon_from = '@'.join(parse_addr(mailfrom))
        # self.fp.write('From %s %s\n' % (self.canon_from,time,ctime()))
        return Milter.CONTINUE


    def attachment(self):
        msg = self._msg
        attachDir = attach_dir(msg)
        removedParts = []
        payload = []
        
        
        for part in msg.walk():
            fname = ""
            
            if part.is_multipart():
                continue
            
            dtypes = part.get_params(None, 'Content-Disposition')

            if not dtypes:
                if part.get_content_type() == 'text/plain':
                    payload.append(part)
                    continue
                ctypes = part.getparams()
                if not ctypes:
                    continue
                for key,val in ctypes:
                    if key.lower() == 'name':
                        fname = val
            else:
                for key,val in dtypes:
                    if key.lower() == 'filename':
                        fname = val
                        
            if fname:
                removedParts.append(fname)
                data = part.get_payload(decode=1)
                extract_attachment(data, attachDir, fname)
                part = self.delete_attachments(part, fname)
                payload.append(part)

#        self.log(payload)
        del msg["content-type"]
        del msg["content-disposition"]
        del msg["content-transfer-encoding"]

        msg.set_type('plain/txt')
        msg.set_payload(payload)

        self._msg = msg

        return Milter.CONTINUE
        
    def delete_attachments(self, part,fname):
        for key,value in part.get_params():
            part.del_param(key)
            
        part.set_payload('[DELETED]\n')
        del part["content-type"]
        del part["content-disposition"]
        del part["content-transfer-encoding"]
        part["Content-Type"] = "text/plain, name="+fname+".TXT"
        return part


    def eom(self):
        self.fp.seek(0)
        self._msg = mime.message_from_file(self.fp)
        self.attachment()
        
        self.log("### MESSAGE ###")
        self.log(self._msg)

        return Milter.ACCEPT
#        return Milter.TEMPFAIL
## ===





def attach_dir(msg):
    tempname = fname = tempfile.mktemp(".tmp")
    out = tempfile.TemporaryFile()
    msg.dump(out)
    out.seek(0)
    buf = out.read()
    hashDir = hashit(buf)
    attachDir = dropDir + hashDir
    
    if not os.path.isdir(hashDir):
        os.mkdir(attachDir)

    return attachDir


def extract_attachment(data, attachDir, fname):
    exdir_file = attachDir + "/" + fname
    extracted = open(exdir_file, "wb")
    extracted.write(data)
    extracted.close()
    


def hashit(data):
    sha1 = hashlib.sha1()
    # f = open(filepath, 'rb')
    # try:
    #     sha1.update(f.read())
    # finally:
    #     f.close()
    sha1.update(data)
       
    return sha1.hexdigest()

dropDir = "/dropdir/"

def main():
    bt = Thread(target=background)
    bt.start()
    socketname = "/tmp/py_testmilter.sock"
    timeout = 600
    Milter.factory = mltr_SaveAttachments
    flags = Milter.CHGBODY + Milter.CHGHDRS + Milter.ADDHDRS
    flags += Milter.ADDRCPT
    flags += Milter.DELRCPT
    Milter.set_flags(flags)     # tell Sendmail/Postfix which features
we use
    print "%s milter startup" % time.strftime('%Y%b%d %H:%M:%S')
    sys.stdout.flush()
    Milter.runmilter("py_testmilter",socketname,timeout)
    logq.put(None)
    bt.join()
    print "%s milter shutdown" % time.strftime('%Y%b%d %H:%M:%S')

if __name__ == "__main__":
    main()

Larry G. Wapnitsky MBA, MCSE, MCP+I
IT SUPPORT & ADMINISTRATION COORDINATOR
WRT  |  Wallace Roberts & Todd, LLC
1700 Market Street, 28th Fl
Philadelphia, PA 19103
T  215.430.5068
C  215.713.8635
E  lwapnitsky at WRTdesign.com


-----Original Message-----
From: Larry G. Wapnitsky
Sent: Wednesday, November 23, 2011 10:38 AM
To: 'Stuart D. Gathman'
Cc: pymilter at bmsi.com
Subject: RE: [Pymilter] Emulating another MimeDefang function

So, I was able to remove the attachments in a function I've written, but
when the message gets sent, the attachments are still there rather than
my modified message.  Not sure what I'm missing.

Code below:


#! /usr/bin/env python

import Milter
import StringIO
import email
import email.Message
import hashlib
import mime
import os
import sys
import tempfile
import time
import rfc822


from email import Errors
from email.Message import Message

## ==  IP Information
from socket import AF_INET, AF_INET6
## ==

## === Define multiprocessing == ##

if True:
    from multiprocessing import Process as Thread, Queue
else:
    from threading import Thread
    from Queue import Queue

logq = Queue(maxsize=4)


def background():
    while True:
        t = logq.get()
        if not t: break
        msg,id,ts = t
        print "%s [%d]" % (time.strftime('%Y%b%d
%H:%M:%S',time.localtime(ts)),id),
        # 2005Oct13 02:34:11 [1] msg1 msg2 msg3 ...
        for i in msg: print i,
        print

## === End Define Multiprocesing === ##

class mltr_SaveAttachments(Milter.Base):

    def __init__(self):
        self.id = Milter.uniqueID()

    def log(self,*msg):
        logq.put((msg,self.id,time.time()))

    def connect(self, IPname, family, hostaddr):
        self.IP = hostaddr[0]
        self.port = hostaddr[1]
        if family == AF_INET6:
            self.flow = hostaddr[2]
            self.scope = hostaddr[3]
        else:
            self.flow = None
            self.scope = None
        self.IPname = IPname # Name from a reverse IP lookup
        self.H = None
        self.fp = None
        self.receiver = self.getsymval('j')
        self.log("connect from %s at %s" % (IPname, hostaddr) )
        return Milter.CONTINUE

    @Milter.noreply
    def header(self, name, hval):
        self.fp.write("%s: %s\n" % (name,hval)) # add header to buffer
        return Milter.CONTINUE

    @Milter.noreply
    def eoh(self):
        self.fp.write("\n")                             # terminate
headers
        return Milter.CONTINUE



    def envfrom(self,mailfrom,*str):
#        self.log("envfrom")
        # self.F = mailfrom
        # self.R = []
        # self.fromparms = Milter.dictfromlist(str)
        # self.user = self.getsymval('{auth_authen}')
        # self.log("mail from:", mailfrom, *str)
        self.fp = StringIO.StringIO()
        # self.canon_from = '@'.join(parse_addr(mailfrom))
        # self.fp.write('From %s %s\n' % (self.canon_from,time,ctime()))
        return Milter.CONTINUE


    def attachment(self):
        msg = self._msg
        attachDir = attach_dir(msg)
        removedParts = []

        for part in msg.walk():
            fname = ""

            if part.is_multipart():
                continue

            dtypes = part.get_params(None, 'Content-Disposition')

            if not dtypes:
                if part.get_content_type() == 'text/plain':
                    continue
                ctypes = part.getparams()
                if not ctypes:
                    continue
                for key,val in ctypes:
                    if key.lower() == 'name':
                        fname = val
            else:
                for key,val in dtypes:
                    if key.lower() == 'filename':
                        fname = val

            if fname:
                removedParts.append(fname)
                data = part.get_payload(decode=1)
                extract_attachment(data, attachDir, fname)
                part = delete_attachments(part)

        self.log(removedParts)
#        return Milter.CONTINUE
        return msg


    def eom(self):
        self.fp.seek(0)
        self._msg = mime.message_from_file(self.fp)
        self._msg = self.attachment()

        self.log(self._msg)
        return Milter.ACCEPT
#        return Milter.TEMPFAIL
## ===


def delete_attachments(part):
    for key,value in part.get_params():
        part.del_param(key)

    part.set_payload('[DELETED]\n')
    part.set_type('text/plain')
    del part['Content-Disposition']

    return part



def attach_dir(msg):
    tempname = fname = tempfile.mktemp(".tmp")
    out = tempfile.TemporaryFile()
    msg.dump(out)
    out.seek(0)
    buf = out.read()
    hashDir = hashit(buf)
    attachDir = dropDir + hashDir

    if not os.path.isdir(hashDir):
        os.mkdir(attachDir)

    return attachDir


def extract_attachment(data, attachDir, fname):
    exdir_file = attachDir + "/" + fname
    extracted = open(exdir_file, "wb")
    extracted.write(data)
    extracted.close()



def hashit(data):
    sha1 = hashlib.sha1()
    # f = open(filepath, 'rb')
    # try:
    #     sha1.update(f.read())
    # finally:
    #     f.close()
    sha1.update(data)

    return sha1.hexdigest()

dropDir = "/dropdir/"

def main():
    bt = Thread(target=background)
    bt.start()
    socketname = "/tmp/py_testmilter.sock"
    timeout = 600
    Milter.factory = mltr_SaveAttachments
    flags = Milter.CHGBODY + Milter.CHGHDRS + Milter.ADDHDRS
    flags += Milter.ADDRCPT
    flags += Milter.DELRCPT
    Milter.set_flags(flags)     # tell Sendmail/Postfix which features
we use
    print "%s milter startup" % time.strftime('%Y%b%d %H:%M:%S')
    sys.stdout.flush()
    Milter.runmilter("py_testmilter",socketname,timeout)
    logq.put(None)
    bt.join()
    print "%s bms milter shutdown" % time.strftime('%Y%b%d %H:%M:%S')

if __name__ == "__main__":
    main()



-----Original Message-----
From: Stuart D. Gathman [mailto:stuart at bmsi.com]
Sent: Monday, November 21, 2011 3:43 PM
To: Larry G. Wapnitsky
Cc: pymilter at bmsi.com
Subject: RE: [Pymilter] Emulating another MimeDefang function

On Mon, 21 Nov 2011, Larry G. Wapnitsky wrote:

> Will try this.

If you were curious about the standard place for filenames (as opposed
the many places LookOut checks), look at the get_filename() method in
email.Message (python comes with source).  It looks first in the
Content-Disposition header field for the filename attribute (the
"correct"
place), and secondly in the name attribute of Content-Type.

--
 	      Stuart D. Gathman <stuart at bmsi.com>
     Business Management Systems Inc.  Phone: 703 591-0911 Fax: 703
591-6154 "Confutatis maledictis, flammis acribus addictis" - background
song for a Microsoft sponsored "Where do you want to go from here?"
commercial.




----wrtmail--%3423wrt%----

<!DSPAM:133D14A27F324454233173089>



------------------------------

Message: 3
Date: Wed, 23 Nov 2011 17:59:29 +0100
From: Daniele Nicolodi <daniele at grinta.net>
Subject: Re: [Pymilter] Emulating another MimeDefang function
To: pymilter at bmsi.com
Message-ID: <4ECD2671.1060805 at grinta.net>
Content-Type: text/plain; charset=ISO-8859-1

On 23/11/11 17:37, Larry G. Wapnitsky wrote:
> Updated code, still not working properly:

Where are you telling your SMTP to replace the body of the message?

Cheers,
--
Daniele


------------------------------

_______________________________________________
Pymilter mailing list
Pymilter at bmsi.com
http://www.bmsi.com/mailman/listinfo/pymilter


End of Pymilter Digest, Vol 12, Issue 7
***************************************




----wrtmail--%3423wrt%----


From LWapnitsky at ph.wrtdesign.com  Fri Nov 25 11:15:37 2011
From: LWapnitsky at ph.wrtdesign.com (Larry G. Wapnitsky)
Date: Fri, 25 Nov 2011 11:15:37 -0500
Subject: [Pymilter] RE: PEmulating another MimeDefang function
In-Reply-To: <FAE5CA7BEE39AF438CE87BF8DBE1375106AC9848@ph-svr-exch1.wrtdesign.com>
References: <FAE5CA7BEE39AF438CE87BF8DBE1375106AC9848@ph-svr-exch1.wrtdesign.com>
Message-ID: <FAE5CA7BEE39AF438CE87BF8DBE1375106AC9849@ph-svr-exch1.wrtdesign.com>

Solved the HTML issue...now working on the attachment issue.

Larry G. Wapnitsky MBA, MCSE, MCP+I
IT SUPPORT & ADMINISTRATION COORDINATOR
WRT  |  Wallace Roberts & Todd, LLC
1700 Market Street, 28th Fl
Philadelphia, PA 19103
T  215.430.5068
C  215.713.8635
E  lwapnitsky at WRTdesign.com


-----Original Message-----
From: Larry G. Wapnitsky 
Sent: Friday, November 25, 2011 8:12 AM
To: 'pymilter at bmsi.com'
Subject: RE: PEmulating another MimeDefang function

Update - I've managed two separate things over the past few days, but
neither is accomplishing exactly what I need.

1)  I can get the body back along with the attachments, but no matter
what I define as the attachment name, Outlook (or any other apps) sees
them as ATT#####.txt

2)  I can't seem to keep the formatting of the original message.


Larry G. Wapnitsky MBA, MCSE, MCP+I
IT SUPPORT & ADMINISTRATION COORDINATOR
WRT  |  Wallace Roberts & Todd, LLC
1700 Market Street, 28th Fl
Philadelphia, PA 19103
T  215.430.5068
C  215.713.8635
E  lwapnitsky at WRTdesign.com


-----Original Message-----
From: Larry G. Wapnitsky 
Sent: Wednesday, November 23, 2011 12:04 PM
To: 'pymilter at bmsi.com'
Subject: RE: Pymilter Digest, Vol 12, Issue 7

Daniele-

I think that's what I'm missing (btw - I've moved a little further since
our last conversation...thanks for the tips :))

Larry G. Wapnitsky MBA, MCSE, MCP+I
IT SUPPORT & ADMINISTRATION COORDINATOR
WRT  |  Wallace Roberts & Todd, LLC
1700 Market Street, 28th Fl
Philadelphia, PA 19103
T  215.430.5068
C  215.713.8635
E  lwapnitsky at WRTdesign.com


-----Original Message-----
From: pymilter-bounces at bmsi.com [mailto:pymilter-bounces at bmsi.com] On
Behalf Of pymilter-request at bmsi.com
Sent: Wednesday, November 23, 2011 12:00 PM
To: pymilter at bmsi.com
Subject: Pymilter Digest, Vol 12, Issue 7

Send Pymilter mailing list submissions to
	pymilter at bmsi.com

To subscribe or unsubscribe via the World Wide Web, visit
	http://www.bmsi.com/mailman/listinfo/pymilter
or, via email, send a message with subject or body 'help' to
	pymilter-request at bmsi.com

You can reach the person managing the list at
	pymilter-owner at bmsi.com

When replying, please edit your Subject line so it is more specific than
"Re: Contents of Pymilter digest..."


Today's Topics:

   1. RE: Emulating another MimeDefang function (Larry G. Wapnitsky)
   2. RE: Emulating another MimeDefang function (Larry G. Wapnitsky)
   3. Re: Emulating another MimeDefang function (Daniele Nicolodi)


----------------------------------------------------------------------

Message: 1
Date: Wed, 23 Nov 2011 10:38:22 -0500
From: "Larry G. Wapnitsky" <LWapnitsky at ph.wrtdesign.com>
Subject: RE: [Pymilter] Emulating another MimeDefang function
To: "Stuart D. Gathman" <stuart at bmsi.com>
Cc: pymilter at bmsi.com
Message-ID:
	
<FAE5CA7BEE39AF438CE87BF8DBE1375106AC9843 at ph-svr-exch1.wrtdesign.com>
Content-Type: text/plain;	charset="us-ascii"

So, I was able to remove the attachments in a function I've written, but
when the message gets sent, the attachments are still there rather than
my modified message.  Not sure what I'm missing.

Code below:


#! /usr/bin/env python

import Milter
import StringIO
import email
import email.Message
import hashlib
import mime
import os
import sys
import tempfile
import time
import rfc822


from email import Errors
from email.Message import Message

## ==  IP Information
from socket import AF_INET, AF_INET6
## ==

## === Define multiprocessing == ##

if True:
    from multiprocessing import Process as Thread, Queue
else:
    from threading import Thread
    from Queue import Queue

logq = Queue(maxsize=4)


def background():
    while True:
        t = logq.get()
        if not t: break
        msg,id,ts = t
        print "%s [%d]" % (time.strftime('%Y%b%d
%H:%M:%S',time.localtime(ts)),id),
        # 2005Oct13 02:34:11 [1] msg1 msg2 msg3 ...
        for i in msg: print i,
        print

## === End Define Multiprocesing === ##

class mltr_SaveAttachments(Milter.Base):

    def __init__(self):
        self.id = Milter.uniqueID()

    def log(self,*msg):
        logq.put((msg,self.id,time.time()))

    def connect(self, IPname, family, hostaddr):
        self.IP = hostaddr[0]
        self.port = hostaddr[1]
        if family == AF_INET6:
            self.flow = hostaddr[2]
            self.scope = hostaddr[3]
        else:
            self.flow = None
            self.scope = None
        self.IPname = IPname # Name from a reverse IP lookup
        self.H = None
        self.fp = None
        self.receiver = self.getsymval('j')
        self.log("connect from %s at %s" % (IPname, hostaddr) )
        return Milter.CONTINUE

    @Milter.noreply
    def header(self, name, hval):
        self.fp.write("%s: %s\n" % (name,hval)) # add header to buffer
        return Milter.CONTINUE

    @Milter.noreply
    def eoh(self):
        self.fp.write("\n")                             # terminate
headers
        return Milter.CONTINUE



    def envfrom(self,mailfrom,*str):
#        self.log("envfrom")
        # self.F = mailfrom
        # self.R = []
        # self.fromparms = Milter.dictfromlist(str)
        # self.user = self.getsymval('{auth_authen}')
        # self.log("mail from:", mailfrom, *str)
        self.fp = StringIO.StringIO()
        # self.canon_from = '@'.join(parse_addr(mailfrom))
        # self.fp.write('From %s %s\n' % (self.canon_from,time,ctime()))
        return Milter.CONTINUE


    def attachment(self):
        msg = self._msg
        attachDir = attach_dir(msg)
        removedParts = []

        for part in msg.walk():
            fname = ""

            if part.is_multipart():
                continue

            dtypes = part.get_params(None, 'Content-Disposition')

            if not dtypes:
                if part.get_content_type() == 'text/plain':
                    continue
                ctypes = part.getparams()
                if not ctypes:
                    continue
                for key,val in ctypes:
                    if key.lower() == 'name':
                        fname = val
            else:
                for key,val in dtypes:
                    if key.lower() == 'filename':
                        fname = val

            if fname:
                removedParts.append(fname)
                data = part.get_payload(decode=1)
                extract_attachment(data, attachDir, fname)
                part = delete_attachments(part)

        self.log(removedParts)
#        return Milter.CONTINUE
        return msg


    def eom(self):
        self.fp.seek(0)
        self._msg = mime.message_from_file(self.fp)
        self._msg = self.attachment()

        self.log(self._msg)
        return Milter.ACCEPT
#        return Milter.TEMPFAIL
## ===


def delete_attachments(part):
    for key,value in part.get_params():
        part.del_param(key)

    part.set_payload('[DELETED]\n')
    part.set_type('text/plain')
    del part['Content-Disposition']

    return part



def attach_dir(msg):
    tempname = fname = tempfile.mktemp(".tmp")
    out = tempfile.TemporaryFile()
    msg.dump(out)
    out.seek(0)
    buf = out.read()
    hashDir = hashit(buf)
    attachDir = dropDir + hashDir

    if not os.path.isdir(hashDir):
        os.mkdir(attachDir)

    return attachDir


def extract_attachment(data, attachDir, fname):
    exdir_file = attachDir + "/" + fname
    extracted = open(exdir_file, "wb")
    extracted.write(data)
    extracted.close()



def hashit(data):
    sha1 = hashlib.sha1()
    # f = open(filepath, 'rb')
    # try:
    #     sha1.update(f.read())
    # finally:
    #     f.close()
    sha1.update(data)

    return sha1.hexdigest()

dropDir = "/dropdir/"

def main():
    bt = Thread(target=background)
    bt.start()
    socketname = "/tmp/py_testmilter.sock"
    timeout = 600
    Milter.factory = mltr_SaveAttachments
    flags = Milter.CHGBODY + Milter.CHGHDRS + Milter.ADDHDRS
    flags += Milter.ADDRCPT
    flags += Milter.DELRCPT
    Milter.set_flags(flags)     # tell Sendmail/Postfix which features
we use
    print "%s milter startup" % time.strftime('%Y%b%d %H:%M:%S')
    sys.stdout.flush()
    Milter.runmilter("py_testmilter",socketname,timeout)
    logq.put(None)
    bt.join()
    print "%s bms milter shutdown" % time.strftime('%Y%b%d %H:%M:%S')

if __name__ == "__main__":
    main()



-----Original Message-----
From: Stuart D. Gathman [mailto:stuart at bmsi.com]
Sent: Monday, November 21, 2011 3:43 PM
To: Larry G. Wapnitsky
Cc: pymilter at bmsi.com
Subject: RE: [Pymilter] Emulating another MimeDefang function

On Mon, 21 Nov 2011, Larry G. Wapnitsky wrote:

> Will try this.

If you were curious about the standard place for filenames (as opposed
the many places LookOut checks), look at the get_filename() method in
email.Message (python comes with source).  It looks first in the
Content-Disposition header field for the filename attribute (the
"correct"
place), and secondly in the name attribute of Content-Type.

--
 	      Stuart D. Gathman <stuart at bmsi.com>
     Business Management Systems Inc.  Phone: 703 591-0911 Fax: 703
591-6154 "Confutatis maledictis, flammis acribus addictis" - background
song for a Microsoft sponsored "Where do you want to go from here?"
commercial.




----wrtmail--%3423wrt%----

<!DSPAM:133D113DA422445417198307>



------------------------------

Message: 2
Date: Wed, 23 Nov 2011 11:37:48 -0500
From: "Larry G. Wapnitsky" <LWapnitsky at ph.wrtdesign.com>
Subject: RE: [Pymilter] Emulating another MimeDefang function
To: "Stuart D. Gathman" <stuart at bmsi.com>
Cc: pymilter at bmsi.com
Message-ID:
	
<FAE5CA7BEE39AF438CE87BF8DBE1375106AC9844 at ph-svr-exch1.wrtdesign.com>
Content-Type: text/plain;	charset="us-ascii"

Updated code, still not working properly:

#! /usr/bin/env python

import Milter
import StringIO
import email
import email.Message
import hashlib
import mime
import os
import sys
import tempfile
import time
import rfc822


from email import Errors
from email.Message import Message

## ==  IP Information
from socket import AF_INET, AF_INET6
## ==

## === Define multiprocessing == ##

if True:
    from multiprocessing import Process as Thread, Queue
else:
    from threading import Thread
    from Queue import Queue

logq = Queue(maxsize=4)


def background():
    while True:
        t = logq.get()
        if not t: break
        msg,id,ts = t
        print "%s [%d]" % (time.strftime('%Y%b%d
%H:%M:%S',time.localtime(ts)),id),
        # 2005Oct13 02:34:11 [1] msg1 msg2 msg3 ...
        for i in msg: print i,
        print

## === End Define Multiprocesing === ##

class mltr_SaveAttachments(Milter.Base):
    
    def __init__(self):
        self.id = Milter.uniqueID()

    def close(self):
        # always called, even when abort is called.  Clean up
        # any external resources here.
        return Milter.CONTINUE

    def abort(self):
        # client disconnected prematurely
        return Milter.CONTINUE

    def log(self,*msg):
        logq.put((msg,self.id,time.time()))
    
    @Milter.noreply
    def connect(self, IPname, family, hostaddr):
        self.IP = hostaddr[0]
        self.port = hostaddr[1]
        if family == AF_INET6:
            self.flow = hostaddr[2]
            self.scope = hostaddr[3]
        else:
            self.flow = None
            self.scope = None
        self.IPname = IPname # Name from a reverse IP lookup
        self.H = None
        self.fp = None
        self.receiver = self.getsymval('j')
        self.log("connect from %s at %s" % (IPname, hostaddr) )
        return Milter.CONTINUE

    @Milter.noreply
    def header(self, name, hval):
        self.fp.write("%s: %s\n" % (name,hval))	# add header to buffer
        return Milter.CONTINUE
    
    @Milter.noreply
    def eoh(self):
        self.fp.write("\n")				# terminate
headers
        return Milter.CONTINUE
    
    @Milter.noreply
    def body(self,chunk):
        self.fp.write(chunk)
        return Milter.CONTINUE
    

    def envfrom(self,mailfrom,*str):
#        self.log("envfrom")
        # self.F = mailfrom
        # self.R = []
        # self.fromparms = Milter.dictfromlist(str)
        # self.user = self.getsymval('{auth_authen}')
        # self.log("mail from:", mailfrom, *str)
        self.fp = StringIO.StringIO()
        # self.canon_from = '@'.join(parse_addr(mailfrom))
        # self.fp.write('From %s %s\n' % (self.canon_from,time,ctime()))
        return Milter.CONTINUE


    def attachment(self):
        msg = self._msg
        attachDir = attach_dir(msg)
        removedParts = []
        payload = []
        
        
        for part in msg.walk():
            fname = ""
            
            if part.is_multipart():
                continue
            
            dtypes = part.get_params(None, 'Content-Disposition')

            if not dtypes:
                if part.get_content_type() == 'text/plain':
                    payload.append(part)
                    continue
                ctypes = part.getparams()
                if not ctypes:
                    continue
                for key,val in ctypes:
                    if key.lower() == 'name':
                        fname = val
            else:
                for key,val in dtypes:
                    if key.lower() == 'filename':
                        fname = val
                        
            if fname:
                removedParts.append(fname)
                data = part.get_payload(decode=1)
                extract_attachment(data, attachDir, fname)
                part = self.delete_attachments(part, fname)
                payload.append(part)

#        self.log(payload)
        del msg["content-type"]
        del msg["content-disposition"]
        del msg["content-transfer-encoding"]

        msg.set_type('plain/txt')
        msg.set_payload(payload)

        self._msg = msg

        return Milter.CONTINUE
        
    def delete_attachments(self, part,fname):
        for key,value in part.get_params():
            part.del_param(key)
            
        part.set_payload('[DELETED]\n')
        del part["content-type"]
        del part["content-disposition"]
        del part["content-transfer-encoding"]
        part["Content-Type"] = "text/plain, name="+fname+".TXT"
        return part


    def eom(self):
        self.fp.seek(0)
        self._msg = mime.message_from_file(self.fp)
        self.attachment()
        
        self.log("### MESSAGE ###")
        self.log(self._msg)

        return Milter.ACCEPT
#        return Milter.TEMPFAIL
## ===





def attach_dir(msg):
    tempname = fname = tempfile.mktemp(".tmp")
    out = tempfile.TemporaryFile()
    msg.dump(out)
    out.seek(0)
    buf = out.read()
    hashDir = hashit(buf)
    attachDir = dropDir + hashDir
    
    if not os.path.isdir(hashDir):
        os.mkdir(attachDir)

    return attachDir


def extract_attachment(data, attachDir, fname):
    exdir_file = attachDir + "/" + fname
    extracted = open(exdir_file, "wb")
    extracted.write(data)
    extracted.close()
    


def hashit(data):
    sha1 = hashlib.sha1()
    # f = open(filepath, 'rb')
    # try:
    #     sha1.update(f.read())
    # finally:
    #     f.close()
    sha1.update(data)
       
    return sha1.hexdigest()

dropDir = "/dropdir/"

def main():
    bt = Thread(target=background)
    bt.start()
    socketname = "/tmp/py_testmilter.sock"
    timeout = 600
    Milter.factory = mltr_SaveAttachments
    flags = Milter.CHGBODY + Milter.CHGHDRS + Milter.ADDHDRS
    flags += Milter.ADDRCPT
    flags += Milter.DELRCPT
    Milter.set_flags(flags)     # tell Sendmail/Postfix which features
we use
    print "%s milter startup" % time.strftime('%Y%b%d %H:%M:%S')
    sys.stdout.flush()
    Milter.runmilter("py_testmilter",socketname,timeout)
    logq.put(None)
    bt.join()
    print "%s milter shutdown" % time.strftime('%Y%b%d %H:%M:%S')

if __name__ == "__main__":
    main()

Larry G. Wapnitsky MBA, MCSE, MCP+I
IT SUPPORT & ADMINISTRATION COORDINATOR
WRT  |  Wallace Roberts & Todd, LLC
1700 Market Street, 28th Fl
Philadelphia, PA 19103
T  215.430.5068
C  215.713.8635
E  lwapnitsky at WRTdesign.com


-----Original Message-----
From: Larry G. Wapnitsky
Sent: Wednesday, November 23, 2011 10:38 AM
To: 'Stuart D. Gathman'
Cc: pymilter at bmsi.com
Subject: RE: [Pymilter] Emulating another MimeDefang function

So, I was able to remove the attachments in a function I've written, but
when the message gets sent, the attachments are still there rather than
my modified message.  Not sure what I'm missing.

Code below:


#! /usr/bin/env python

import Milter
import StringIO
import email
import email.Message
import hashlib
import mime
import os
import sys
import tempfile
import time
import rfc822


from email import Errors
from email.Message import Message

## ==  IP Information
from socket import AF_INET, AF_INET6
## ==

## === Define multiprocessing == ##

if True:
    from multiprocessing import Process as Thread, Queue
else:
    from threading import Thread
    from Queue import Queue

logq = Queue(maxsize=4)


def background():
    while True:
        t = logq.get()
        if not t: break
        msg,id,ts = t
        print "%s [%d]" % (time.strftime('%Y%b%d
%H:%M:%S',time.localtime(ts)),id),
        # 2005Oct13 02:34:11 [1] msg1 msg2 msg3 ...
        for i in msg: print i,
        print

## === End Define Multiprocesing === ##

class mltr_SaveAttachments(Milter.Base):

    def __init__(self):
        self.id = Milter.uniqueID()

    def log(self,*msg):
        logq.put((msg,self.id,time.time()))

    def connect(self, IPname, family, hostaddr):
        self.IP = hostaddr[0]
        self.port = hostaddr[1]
        if family == AF_INET6:
            self.flow = hostaddr[2]
            self.scope = hostaddr[3]
        else:
            self.flow = None
            self.scope = None
        self.IPname = IPname # Name from a reverse IP lookup
        self.H = None
        self.fp = None
        self.receiver = self.getsymval('j')
        self.log("connect from %s at %s" % (IPname, hostaddr) )
        return Milter.CONTINUE

    @Milter.noreply
    def header(self, name, hval):
        self.fp.write("%s: %s\n" % (name,hval)) # add header to buffer
        return Milter.CONTINUE

    @Milter.noreply
    def eoh(self):
        self.fp.write("\n")                             # terminate
headers
        return Milter.CONTINUE



    def envfrom(self,mailfrom,*str):
#        self.log("envfrom")
        # self.F = mailfrom
        # self.R = []
        # self.fromparms = Milter.dictfromlist(str)
        # self.user = self.getsymval('{auth_authen}')
        # self.log("mail from:", mailfrom, *str)
        self.fp = StringIO.StringIO()
        # self.canon_from = '@'.join(parse_addr(mailfrom))
        # self.fp.write('From %s %s\n' % (self.canon_from,time,ctime()))
        return Milter.CONTINUE


    def attachment(self):
        msg = self._msg
        attachDir = attach_dir(msg)
        removedParts = []

        for part in msg.walk():
            fname = ""

            if part.is_multipart():
                continue

            dtypes = part.get_params(None, 'Content-Disposition')

            if not dtypes:
                if part.get_content_type() == 'text/plain':
                    continue
                ctypes = part.getparams()
                if not ctypes:
                    continue
                for key,val in ctypes:
                    if key.lower() == 'name':
                        fname = val
            else:
                for key,val in dtypes:
                    if key.lower() == 'filename':
                        fname = val

            if fname:
                removedParts.append(fname)
                data = part.get_payload(decode=1)
                extract_attachment(data, attachDir, fname)
                part = delete_attachments(part)

        self.log(removedParts)
#        return Milter.CONTINUE
        return msg


    def eom(self):
        self.fp.seek(0)
        self._msg = mime.message_from_file(self.fp)
        self._msg = self.attachment()

        self.log(self._msg)
        return Milter.ACCEPT
#        return Milter.TEMPFAIL
## ===


def delete_attachments(part):
    for key,value in part.get_params():
        part.del_param(key)

    part.set_payload('[DELETED]\n')
    part.set_type('text/plain')
    del part['Content-Disposition']

    return part



def attach_dir(msg):
    tempname = fname = tempfile.mktemp(".tmp")
    out = tempfile.TemporaryFile()
    msg.dump(out)
    out.seek(0)
    buf = out.read()
    hashDir = hashit(buf)
    attachDir = dropDir + hashDir

    if not os.path.isdir(hashDir):
        os.mkdir(attachDir)

    return attachDir


def extract_attachment(data, attachDir, fname):
    exdir_file = attachDir + "/" + fname
    extracted = open(exdir_file, "wb")
    extracted.write(data)
    extracted.close()



def hashit(data):
    sha1 = hashlib.sha1()
    # f = open(filepath, 'rb')
    # try:
    #     sha1.update(f.read())
    # finally:
    #     f.close()
    sha1.update(data)

    return sha1.hexdigest()

dropDir = "/dropdir/"

def main():
    bt = Thread(target=background)
    bt.start()
    socketname = "/tmp/py_testmilter.sock"
    timeout = 600
    Milter.factory = mltr_SaveAttachments
    flags = Milter.CHGBODY + Milter.CHGHDRS + Milter.ADDHDRS
    flags += Milter.ADDRCPT
    flags += Milter.DELRCPT
    Milter.set_flags(flags)     # tell Sendmail/Postfix which features
we use
    print "%s milter startup" % time.strftime('%Y%b%d %H:%M:%S')
    sys.stdout.flush()
    Milter.runmilter("py_testmilter",socketname,timeout)
    logq.put(None)
    bt.join()
    print "%s bms milter shutdown" % time.strftime('%Y%b%d %H:%M:%S')

if __name__ == "__main__":
    main()



-----Original Message-----
From: Stuart D. Gathman [mailto:stuart at bmsi.com]
Sent: Monday, November 21, 2011 3:43 PM
To: Larry G. Wapnitsky
Cc: pymilter at bmsi.com
Subject: RE: [Pymilter] Emulating another MimeDefang function

On Mon, 21 Nov 2011, Larry G. Wapnitsky wrote:

> Will try this.

If you were curious about the standard place for filenames (as opposed
the many places LookOut checks), look at the get_filename() method in
email.Message (python comes with source).  It looks first in the
Content-Disposition header field for the filename attribute (the
"correct"
place), and secondly in the name attribute of Content-Type.

--
 	      Stuart D. Gathman <stuart at bmsi.com>
     Business Management Systems Inc.  Phone: 703 591-0911 Fax: 703
591-6154 "Confutatis maledictis, flammis acribus addictis" - background
song for a Microsoft sponsored "Where do you want to go from here?"
commercial.




----wrtmail--%3423wrt%----

<!DSPAM:133D14A27F324454233173089>



------------------------------

Message: 3
Date: Wed, 23 Nov 2011 17:59:29 +0100
From: Daniele Nicolodi <daniele at grinta.net>
Subject: Re: [Pymilter] Emulating another MimeDefang function
To: pymilter at bmsi.com
Message-ID: <4ECD2671.1060805 at grinta.net>
Content-Type: text/plain; charset=ISO-8859-1

On 23/11/11 17:37, Larry G. Wapnitsky wrote:
> Updated code, still not working properly:

Where are you telling your SMTP to replace the body of the message?

Cheers,
--
Daniele


------------------------------

_______________________________________________
Pymilter mailing list
Pymilter at bmsi.com
http://www.bmsi.com/mailman/listinfo/pymilter


End of Pymilter Digest, Vol 12, Issue 7
***************************************




----wrtmail--%3423wrt%----


From LWapnitsky at ph.wrtdesign.com  Fri Nov 25 11:27:39 2011
From: LWapnitsky at ph.wrtdesign.com (Larry G. Wapnitsky)
Date: Fri, 25 Nov 2011 11:27:39 -0500
Subject: [Pymilter] RE: PEmulating another MimeDefang function
In-Reply-To: <FAE5CA7BEE39AF438CE87BF8DBE1375106AC9849@ph-svr-exch1.wrtdesign.com>
References: <FAE5CA7BEE39AF438CE87BF8DBE1375106AC9848@ph-svr-exch1.wrtdesign.com>
	<FAE5CA7BEE39AF438CE87BF8DBE1375106AC9849@ph-svr-exch1.wrtdesign.com>
Message-ID: <FAE5CA7BEE39AF438CE87BF8DBE1375106AC984A@ph-svr-exch1.wrtdesign.com>

Further update - it's Outlook that's causing the issue...Thunderbird and
Gmail web show it properly.

Larry G. Wapnitsky MBA, MCSE, MCP+I
IT SUPPORT & ADMINISTRATION COORDINATOR
WRT  |  Wallace Roberts & Todd, LLC
1700 Market Street, 28th Fl
Philadelphia, PA 19103
T  215.430.5068
C  215.713.8635
E  lwapnitsky at WRTdesign.com


-----Original Message-----
From: Larry G. Wapnitsky 
Sent: Friday, November 25, 2011 11:16 AM
To: 'pymilter at bmsi.com'
Subject: RE: PEmulating another MimeDefang function

Solved the HTML issue...now working on the attachment issue.

Larry G. Wapnitsky MBA, MCSE, MCP+I
IT SUPPORT & ADMINISTRATION COORDINATOR
WRT  |  Wallace Roberts & Todd, LLC
1700 Market Street, 28th Fl
Philadelphia, PA 19103
T  215.430.5068
C  215.713.8635
E  lwapnitsky at WRTdesign.com


-----Original Message-----
From: Larry G. Wapnitsky 
Sent: Friday, November 25, 2011 8:12 AM
To: 'pymilter at bmsi.com'
Subject: RE: PEmulating another MimeDefang function

Update - I've managed two separate things over the past few days, but
neither is accomplishing exactly what I need.

1)  I can get the body back along with the attachments, but no matter
what I define as the attachment name, Outlook (or any other apps) sees
them as ATT#####.txt

2)  I can't seem to keep the formatting of the original message.


Larry G. Wapnitsky MBA, MCSE, MCP+I
IT SUPPORT & ADMINISTRATION COORDINATOR
WRT  |  Wallace Roberts & Todd, LLC
1700 Market Street, 28th Fl
Philadelphia, PA 19103
T  215.430.5068
C  215.713.8635
E  lwapnitsky at WRTdesign.com






----wrtmail--%3423wrt%----