From stuart at gathman.org Fri Mar 9 08:03:06 2018 From: stuart at gathman.org (Stuart Gathman) Date: Fri, 9 Mar 2018 08:03:06 -0500 Subject: [Pymilter] Issue with spfmilter? In-Reply-To: References: Message-ID: <87c51eb1-00e0-7855-74ec-37b2842595c7@gathman.org> On 03/09/2018 03:28 AM, Harald Hannelius wrote: > > sorry if I'm beeing rude and contacting you. I'm not used to github > and don't know how to discuss problems I'm seeing with spfmilter.py. > Please give me advice if this is the wrong way. It would help more people if you used the mailing list.? I am subscribing you.? > > I have a domain in sweden, that when sending e-mail a few e-mails pass > the milter's SPF-check but then I get one sender that gets a Void > lookup error. That is almost certainly correct.? The sender policy has a non-existent domain.? Per RFC7208, the default limit for void lookups is 2.? However, this is an optional test, and your other tools may not implement this or may use a different limit. RFC7208 4.6.4/7 As described at the end of Section 11.1, there may be cases where it is useful to limit the number of "terms" for which DNS queries return either a positive answer (RCODE 0) with an answer count of 0, or a "Name Error" (RCODE 3) answer. These are sometimes collectively referred to as "void lookups". SPF implementations SHOULD limit "void lookups" to two. An implementation MAY choose to make such a limit configurable. In this case, a default of two is RECOMMENDED. Exceeding the limit produces a "permerror" result. We cannot look at the sender policy in question to point? out the null lookup, because you did not mention the domain! There is currently no config option for the void lookup limit (I should add it, it would be simple).? You can add a line to spfmilter.py to change the "constant"? after importing spf: ? spf.MAX_VOID_LOOKUPS = 2? # default is 2 - make it bigger if you insist.? However, the sender should fix his policy.? The void lookups are limited to prevent DNS amplification attacks. From harald at iki.fi Fri Mar 9 08:14:00 2018 From: harald at iki.fi (Harald Hannelius) Date: Fri, 9 Mar 2018 15:14:00 +0200 (EET) Subject: [Pymilter] Issue with spfmilter? In-Reply-To: <87c51eb1-00e0-7855-74ec-37b2842595c7@gathman.org> References: <87c51eb1-00e0-7855-74ec-37b2842595c7@gathman.org> Message-ID: On Fri, 9 Mar 2018, Stuart Gathman wrote: > On 03/09/2018 03:28 AM, Harald Hannelius wrote: >> >> sorry if I'm beeing rude and contacting you. I'm not used to github >> and don't know how to discuss problems I'm seeing with spfmilter.py. >> Please give me advice if this is the wrong way. > It would help more people if you used the mailing list.? I am > subscribing you.? Thank You, and hello all. >> I have a domain in sweden, that when sending e-mail a few e-mails pass >> the milter's SPF-check but then I get one sender that gets a Void >> lookup error. > That is almost certainly correct.? The sender policy has a non-existent > domain.? Per RFC7208, the default limit for void lookups is 2.? However, > this is an optional test, and your other tools may not implement this or > may use a different limit. Sorry, the domain is mdh.se and they have an SPF-record. It validates OK on both mxtoolbox and kitterman.com tests. I then noted that if I telnet the IPv6-address of my SMTP-gateway, and pretend to send from mdh.se I get the "Void lookup" error every time. If I telnet the IPv4-address, the test succeeds. Also, please consider this log-excerpt; Mar 9 09:59:37 gateway sm-mta[12962]: w297xYG3012962: Milter: from=, reject=550-5.5.2 SPF Permanent Error: Void lookup limit of 2 exceeded\r\n550-5.5.2 There is a fatal syntax error in the SPF record for mdh.se\r\n550 5.5.2 We cannot accept mail from mdh.se until this is corrected. Mar 9 09:59:37 gateway sm-mta[12962]: w297xYG3012962: from=, size=94241, class=0, nrcpts=0, proto=ESMTPS, daemon=MTA, relay=mail-ve1eur01on0728.outbound.protection.outlook.com [IPv6:2a01:111:f400:fe1f:0:0:0:728] The first try to deliver failed. Mar 9 10:00:59 gateway sm-mta[13277]: w2980uun013277: from=, size=86456, class=0, nrcpts=1, msgid=, proto=ESMTPS, daemon=MTA, relay=mail-eopbgr50092.outbound.protection.outlook.com [40.107.5.92] Mar 9 10:01:00 gateway sm-mta[13277]: w2980uun013277: Milter insert (0): header: Received-SPF: Pass (gateway.arcada.fi: domain of mdh.se designates 40.107.5.92 as permitted sender) client-ip=40.107.5.92; envelope-from="some.one at mdh.se"; helo=EUR03-VE1-obe.outbound.protection.outlook.com; receiver=gateway.arcada.fi; mechanism="include:spf.protection.outlook.com"; identity=mailfrom One and a half minute later, the SPF-record validates and the mail is delivered. The difference I see is that the first connections was over IPv6, and the second over IPv4. There seems to be an issue with IPv6. I also posted a bug-report for Debian: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=892477 > > RFC7208 4.6.4/7 > > As described at the end of Section 11.1, there may be cases where it > is useful to limit the number of "terms" for which DNS queries return > either a positive answer (RCODE 0) with an answer count of 0, or a > "Name Error" (RCODE 3) answer. These are sometimes collectively > referred to as "void lookups". SPF implementations SHOULD limit > "void lookups" to two. An implementation MAY choose to make such a > limit configurable. In this case, a default of two is RECOMMENDED. > Exceeding the limit produces a "permerror" result. > > > We cannot look at the sender policy in question to point? out the null > lookup, because you did not mention the domain! My bad, that is mdh.se. > There is currently no config option for the void lookup limit (I should > add it, it would be simple).? You can add a line to spfmilter.py > to change the "constant"? after importing spf: > > ? spf.MAX_VOID_LOOKUPS = 2? # default is 2 - make it bigger if you insist.? > > However, the sender should fix his policy.? The void lookups are limited > to prevent DNS amplification attacks. I think the void-limit is a good thing, and I'd rather not change that. I hope I can fix the reason why the milter is dropping the mail. -- Harald Hannelius | harald at iki.fi | +358505941020 From stuart at gathman.org Fri Mar 9 08:25:47 2018 From: stuart at gathman.org (Stuart D. Gathman) Date: Fri, 9 Mar 2018 08:25:47 -0500 (EST) Subject: [Pymilter] Issue with spfmilter? In-Reply-To: References: <87c51eb1-00e0-7855-74ec-37b2842595c7@gathman.org> Message-ID: On Fri, 9 Mar 2018, Harald Hannelius wrote: > Sorry, the domain is mdh.se and they have an SPF-record. It validates OK on > both mxtoolbox and kitterman.com tests. You apparently only tested with IP4. > I then noted that if I telnet the IPv6-address of my SMTP-gateway, and > pretend to send from mdh.se I get the "Void lookup" error every time. If I > telnet the IPv4-address, the test succeeds. That's because their record has way more than 2 void lookups for IPv6. > One and a half minute later, the SPF-record validates and the mail is > delivered. The difference I see is that the first connections was over IPv6, > and the second over IPv4. > > There seems to be an issue with IPv6. I also posted a bug-report for Debian: Yes, they shouldn't be trying to connect via IPv6 when their sender policy is full of void lookups for IPv6. That is why the void lookup limit was added in RFC7208. If you want to be lenient on void lookups only for IPv6, that should be simple to add via code. When I get around to adding a config for void lookups, I should probably have separate configs for IP4 and IPv6. -- Stuart D. Gathman "Confutatis maledictis, flamis acribus addictis" - background song for a Microsoft sponsored "Where do you want to go from here?" commercial. From harald at iki.fi Fri Mar 9 08:35:23 2018 From: harald at iki.fi (Harald Hannelius) Date: Fri, 9 Mar 2018 15:35:23 +0200 (EET) Subject: [Pymilter] Issue with spfmilter? In-Reply-To: References: <87c51eb1-00e0-7855-74ec-37b2842595c7@gathman.org> Message-ID: On Fri, 9 Mar 2018, Stuart D. Gathman wrote: > On Fri, 9 Mar 2018, Harald Hannelius wrote: > >> Sorry, the domain is mdh.se and they have an SPF-record. It validates OK on >> both mxtoolbox and kitterman.com tests. > > You apparently only tested with IP4. Well yes, I was unable to enter an IPv6-address in the mxtoolbox.com test. > >> I then noted that if I telnet the IPv6-address of my SMTP-gateway, and >> pretend to send from mdh.se I get the "Void lookup" error every time. If I >> telnet the IPv4-address, the test succeeds. > > That's because their record has way more than 2 void lookups for IPv6. Oh, I have missed that. Sorry. I have to look closer on their record. This succeeds; $ spfquery --sender=some.user at mdh.se --ip=2a01:0111:f400:fe1e::0712 pass spfquery: domain of mdh.se designates 2a01:111:f400:fe1e::712 as permitted sender Received-SPF: pass (spfquery: domain of mdh.se designates 2a01:111:f400:fe1e::712 as permitted sender) client-ip=2a01:111:f400:fe1e::712; envelope-from=some.user at mdh.se; I can't figure out why there are more than 2 void lookups in their record. Maybe I just don't get it :) $?host -t txt mdh.se mdh.se descriptive text "v=spf1 a:f-medby.ita.mdh.se a:www.netigate.se a:smtp.quicknet.se ip4:130.238.186.19 ip4:130.238.7.172 ip4:130.238.7.173 ip4:130.238.7.174 ip4:130.238.7.175 ip4:130.238.7.176 ip4:130.238.7.177 " "include:spf.protection.outlook.com include:spf-eu.exlibrisgroup.com ~all" mdh.se descriptive text "JaV+SpuwwSQ7VgkHegHs4triOGA+YaEYbk5Ta20BNd9q8BlQdeyFEqZrmR7bTHJbRF6Vi1bbw+KcRJ1iHRBE0A==" mdh.se descriptive text "MS=ms19563410" $?host f-medby.ita.mdh.se f-medby.ita.mdh.se has address 130.243.84.223 $?host www.netigate.se www.netigate.se has address 89.46.81.172 $?host smtp.quicknet.se smtp.quicknet.se has address 193.41.214.32 $?host -t txt spf.protection.outlook.com spf.protection.outlook.com descriptive text "v=spf1 ip4:207.46.100.0/24 ip4:207.46.163.0/24 ip4:65.55.169.0/24 ip4:157.56.110.0/23 ip4:157.55.234.0/24 ip4:213.199.154.0/24 ip4:213.199.180.128/26 include:spfa.protection.outlook.com -all" $?host -t txt spfa.protection.outlook.com spfa.protection.outlook.com descriptive text "v=spf1 ip4:157.56.112.0/24 ip4:207.46.51.64/26 ip4:64.4.22.64/26 ip4:40.92.0.0/14 ip4:40.107.0.0/17 ip4:40.107.128.0/17 ip4:134.170.140.0/24 include:spfb.protection.outlook.com ip6:2001:489a:2202::/48 -all" $?host -t txt spfb.protection.outlook.com spfb.protection.outlook.com descriptive text "v=spf1 ip6:2a01:111:f400::/48 ip4:23.103.128.0/19 ip4:23.103.198.0/23 ip4:65.55.88.0/24 ip4:104.47.0.0/17 ip4:23.103.200.0/21 ip4:23.103.208.0/21 ip4:23.103.191.0/24 ip4:216.32.180.0/23 ip4:94.245.120.64/26 -all" $?host -t txt spf-eu.exlibrisgroup.com spf-eu.exlibrisgroup.com descriptive text "v=spf1 ip4:95.172.90.143 ip4:95.172.90.156 -all" $? > Yes, they shouldn't be trying to connect via IPv6 when their sender > policy is full of void lookups for IPv6. I agree! I just don't get it then. > If you want to be lenient on void lookups only for IPv6, that should > be simple to add via code. When I get around to adding a config for > void lookups, I should probably have separate configs for IP4 and IPv6. No need to be lenient. People should fix their -- Harald Hannelius | harald at iki.fi | +358505941020 From stuart at gathman.org Fri Mar 9 08:44:49 2018 From: stuart at gathman.org (Stuart D. Gathman) Date: Fri, 9 Mar 2018 08:44:49 -0500 (EST) Subject: [Pymilter] Issue with spfmilter? In-Reply-To: References: <87c51eb1-00e0-7855-74ec-37b2842595c7@gathman.org> Message-ID: On Fri, 9 Mar 2018, Harald Hannelius wrote: > $?host f-medby.ita.mdh.se > f-medby.ita.mdh.se has address 130.243.84.223 $ host -t aaaa f-medby.ita.mdh.se # first void lookup f-medby.ita.mdh.se has no AAAA record $ host -t aaaa aaaa www.netigate.se # second void lookup www.netigate.se has no AAAA record $ host -t aaaa smtp.quicknet.se # third void lookup smtp.quicknet.se has no AAAA record Three strikes - theeeiiiirrrrr OUT! (Sorry, American baseball.) -- Stuart D. Gathman "Confutatis maledictis, flamis acribus addictis" - background song for a Microsoft sponsored "Where do you want to go from here?" commercial. From harald at iki.fi Fri Mar 9 08:51:37 2018 From: harald at iki.fi (Harald Hannelius) Date: Fri, 9 Mar 2018 15:51:37 +0200 (EET) Subject: [Pymilter] Issue with spfmilter? In-Reply-To: References: <87c51eb1-00e0-7855-74ec-37b2842595c7@gathman.org> Message-ID: On Fri, 9 Mar 2018, Stuart D. Gathman wrote: > On Fri, 9 Mar 2018, Harald Hannelius wrote: > >> $?host f-medby.ita.mdh.se >> f-medby.ita.mdh.se has address 130.243.84.223 > > $ host -t aaaa f-medby.ita.mdh.se # first void lookup > f-medby.ita.mdh.se has no AAAA record > $ host -t aaaa aaaa www.netigate.se # second void lookup > www.netigate.se has no AAAA record > $ host -t aaaa smtp.quicknet.se # third void lookup > smtp.quicknet.se has no AAAA record Thanks for the clarification. I was confused by thinking that 'a' corresponds to 'IN A' and that there would have to be an 'aaaa' also, corresponding to 'IN AAAA'. Funny enough, I'm running another milter 'spfmilter-2.001', and this one doesn't reject mail because of this. It doesn't log anything though, only "reject". > Three strikes - theeeiiiirrrrr OUT! (Sorry, American baseball.) Hahaa :) Many thanks! Have a nice weekend -- Harald Hannelius | harald at iki.fi | +358505941020 From stuart at gathman.org Fri Mar 9 09:08:06 2018 From: stuart at gathman.org (Stuart Gathman) Date: Fri, 9 Mar 2018 09:08:06 -0500 Subject: [Pymilter] Issue with spfmilter? In-Reply-To: References: <87c51eb1-00e0-7855-74ec-37b2842595c7@gathman.org> Message-ID: On 03/09/2018 08:51 AM, Harald Hannelius wrote: > > Thanks for the clarification. I was confused by thinking that 'a' > corresponds to 'IN A' and that there would have to be an 'aaaa' also, > corresponding to 'IN AAAA'. > > Funny enough, I'm running another milter 'spfmilter-2.001', and this > one doesn't reject mail because of this. It doesn't log anything > though, only "reject". The limit is RECOMMENDED, not MUSTard in RFC7208.? You can add a policy override in /etc/mail/access for mdh.se: SPF-PermError:mdh.se ?????? OK The more complex milter also has a DSN policy which sends a DSN (MAILFROM <>) to the sender explaining the problem.? In the decade I've been running python milter, however, not *once* has anyone except a mail admin read a DSN from me or from anyone else.? Not once has an end user ever forwarded a DSN to their mail admin.? In fact, I have never encountered an end user who had any clue who their mail admin was. (or actually, "What's a mail admin?")? So the only thing sending a DSN accomplishes is verifying that the Return-Path works.? :-) Here is the template for a PermError DSN: To: %(sender)s From: postmaster@%(receiver)s Subject: Critical SPF configuration error Auto-Submitted: auto-generated (configuration error) This is an automatically generated Delivery Status Notification. THIS IS A WARNING MESSAGE ONLY. YOU DO *NOT* NEED TO RESEND YOUR MESSAGE. Delivery to the following recipients has been delayed. ??? %(rcpt)s Subject: %(subject)s Received-SPF: %(spf_result)s Your spf record has a permanent error.? The error was: ??? %(perm_error)s We will reinterpret your record using "lax" processing heuristics which may result in your mail being accepted anyway.? But you or your mail administrator need to fix your SPF record as soon as possible. We are sending you this message to alert you to the fact that you have problems with your email configuration. If you need further assistance, please do not hesitate to contact me again. Kind regards, postmaster@%(receiver)s