[Pymilter] general gossip questions

[Stuart D. Gathman]

On Mon, 22 Feb 2010, Todd Lyons wrote:

> don't have a need for, so now I'll see about getting pygossip up and
> running on a test mail system.

Beware the unsolved problem of REUSE_ADDR not working for TCPServer.

> 1. Is there any more recent version/code in git or any other VCS?  Bug
> fixes?  Feature additions?

I check in work to CVS on sourceforge.

> 2. I don't know much about python, but Shelves just seems to be a
> version of berkeleyDB.  Is this right?

Correct.  Easy, and sufficient for my servers.  My server with 
heaviest usage (100000 msgs/day) uses 630M for the shelve database
with no problem.  I think there is a way to plug in another database.

> 3. What about manual adjustments to the data?  For example, completely
> purge one sender's reputation?

The 'R' command resets the reputation for a domain:qual.  The tc.py
script (inexplicably not installed by package - should at least go in
/usr/lib/pymilter) provides a simple command line interface to query, feedback,
and reset reputations.

> 4. Any kind of summary or data mining scripts?  Bosses like pretty graphs.

No.  pygossip_purge.py deletes old unused records.  It could collect
stats while doing its thing.   (It could run daily automatically
when the REUSE_ADDR bug is fixed.)

> I don't quite have a grasp of the UMIS yet, but I'm studying to see
> how it differs in function from the unique identifier that is the
> IP:domain tuple.

The UMIS identifies a message - as opposed to a domain qual.  The
UMIS is stored in an email header, and can be used to update how that
message "votes" for the reputation of the domain.  The UMIS is used
so that each message gets at most one vote.  pygossip doesn't currently worry
too much about losing votes.  :-)

Note that IP has been generalized to qualifier.  In the milter package,
domains that pass SPF have the SPF qualifier.  Softfail gets the
SOFTFAIL qualifier, and so on for NEUTRAL, FAIL, PERMERROR, GUESS (gets
a best guess pass).  (Yes, some braindead domains routinely send out mail with
SPF fail, and need a special policy to accept their braindamage - but the
reputation system will still block them if enough spam comes in with their
domain.)  IP is used when SPF result is NONE.

Thus, the NEUTRAL reputation of a domain may have a different (and likely
much worse) reputation than the SPF reputation.

-- 
	      Stuart D. Gathman <stuart at bmsi.com>
    Business Management Systems Inc.  Phone: 703 591-0911 Fax: 703 591-6154
"Confutatis maledictis, flammis acribus addictis" - background song for
a Microsoft sponsored "Where do you want to go from here?" commercial.