From ietf-dane at dukhovni.org Tue Nov 3 21:10:19 2015 From: ietf-dane at dukhovni.org (Viktor Dukhovni) Date: Tue, 3 Nov 2015 20:10:19 +0000 Subject: Deployment news (comcast.net publishes TLSA RRs) Message-ID: <20151103201019.GP18315@mournblade.imrryr.org> Today comcast.net published TLSA records for their MX hosts. comcast.net. IN MX 5 mx1.comcast.net. comcast.net. IN MX 5 mx2.comcast.net. _25._tcp.mx1.comcast.net. IN TLSA 3 1 1 90e2f742b459860c0bbf1343b5a36bc5842a3f45056d30bf25dbb475a62eca47 _25._tcp.mx2.comcast.net. IN TLSA 3 1 1 c8cb2faa4c0b92cb3fd37e61eb4671744055f123c14c0dd31e8d92c379f9f8a3 $ posttls-finger -c -Lsummary -o inet_protocols=ipv4 "[mx1.comcast.net]" posttls-finger: Verified TLS connection established to mx1.comcast.net[96.114.157.80]:25: TLSv1.2 with cipher DHE-RSA-AES256-SHA (256/256 bits) $ posttls-finger -c -Lsummary -o inet_protocols=ipv4 "[mx2.comcast.net]" posttls-finger: Verified TLS connection established to mx2.comcast.net[68.87.20.5]:25: TLSv1.2 with cipher DHE-RSA-AES256-SHA (256/256 bits) Congratulations and thanks to Comcast. They are the first major US email provider to do so. Let's hope their lead will be followed by many others. My ongoing survey has now found 9389 working DANE domains. Most of these are served by a few domain hosting providers: 5230 udmedia.de 955 nederhost.net 354 transip.email 47 mediaweb-it.net 45 mailbox.org 36 gr-webdesign.de 32 core-networks.de 32 wk-serv.net 30 set-hosting.de 30 dotplex.de The actual numbers of DANE-enabled hosted domains is much larger, for example udmedia alone reportedly has over 25 thousand. My lists of candidate domains to test are far from complete. Of these 9389, there are now 28 domains (up from 27 yesterday now that comcast.net is live) that are "large enough" to be listed in Google's email transparency report: conjur.com.br jpberlin.de comcast.net freebsd.org mypst.com.br lrz.de rrpproxy.net ietf.org registro.br posteo.de t-2.net isc.org societe.com ruhr-uni-bochum.de aanbodpagina.nl netbsd.org t-2.com tum.de xs4all.nl openssl.org bayern.de unitymedia.de debian.org samba.org bund.de lepartidegauche.fr eu.org torproject.org On the "problem" front. The following DNS hosters still have some issues with correct DNSSEC "denial of existence": #Domains Provider -------- ---------- 33 binero.se (resolution in progress) 28 isphuset.no (issue acknowledged) 15 axc.nl (notified) 11 papaki.gr (notified) 5 forpsi.net (notified) And 10 "small" domains currently publish incorrect TLSA records: bebidaliberada.com.br solucoesglobais.com.br nevodnet.com zx.com 1post.de geekify.de wx0.de tsimnet.eu konundrum.org www.co.tt If anyone reading this happens to know a usable contact for the above, please let them know their TLSA records need updates. Finally, I have a list of ~97000 domains that have DNSSEC and at least one "primary" MX host has DNSSEC, but no TLSA records are published as yet. These domains are good candidates for DANE deployment, it is just a matter of deciding out of whether to use "3 1 1" end-entity records or "2 0 1" trust-anchor records, and documenting a key/cert rotation procedure: https://tools.ietf.org/html/rfc7671#section-5.1 https://tools.ietf.org/html/rfc7671#section-5.2 As always, don't forget: https://dane.sys4.de/common_mistakes#3 https://dane.sys4.de/common_mistakes#6 https://tools.ietf.org/html/rfc7671#section-8.1 https://tools.ietf.org/html/rfc7671#section-8.4 -- Viktor. From ietf-dane at dukhovni.org Fri Nov 13 17:52:37 2015 From: ietf-dane at dukhovni.org (Viktor Dukhovni) Date: Fri, 13 Nov 2015 16:52:37 +0000 Subject: Deployment news (comcast.net publishes TLSA RRs) In-Reply-To: <20151103201019.GP18315@mournblade.imrryr.org> References: <20151103201019.GP18315@mournblade.imrryr.org> Message-ID: <20151113165237.GI24882@mournblade.imrryr.org> On Tue, Nov 03, 2015 at 08:10:19PM +0000, Viktor Dukhovni wrote: > On the "problem" front. The following DNS hosters still have some > issues with correct DNSSEC "denial of existence": > > #Domains Provider > -------- ---------- > 33 binero.se (resolution in progress) > 28 isphuset.no (issue acknowledged) > 15 axc.nl (notified) > 11 papaki.gr (notified) > 5 forpsi.net (notified) I'm told isphuset.no are expecting to resolve the issue this month. Also papaki.gr have upgraded their PowerDNS servers, and the issue is resolved for their domains. -- Viktor. From ietf-dane at dukhovni.org Mon Nov 16 16:52:16 2015 From: ietf-dane at dukhovni.org (Viktor Dukhovni) Date: Mon, 16 Nov 2015 15:52:16 +0000 Subject: Deployment news (comcast.net publishes TLSA RRs) In-Reply-To: <1447688517.894513.441153417.326D57E6@webmail.messagingengine.com> References: <20151103201019.GP18315@mournblade.imrryr.org> <1447688517.894513.441153417.326D57E6@webmail.messagingengine.com> Message-ID: <20151116155216.GJ18315@mournblade.imrryr.org> On Mon, Nov 16, 2015 at 09:41:57AM -0600, Mark Felder wrote: > > My ongoing survey has now found 9389 working DANE domains. Most > > of these are served by a few domain hosting providers: > > > > 5230 udmedia.de > > 955 nederhost.net > > 354 transip.email > > 47 mediaweb-it.net > > 45 mailbox.org > > 36 gr-webdesign.de > > 32 core-networks.de > > 32 wk-serv.net > > 30 set-hosting.de > > 30 dotplex.de > > > > Your numbers seem much higher than Verisign's. I wonder what Verisign's > secspider is missing? > > http://secspider.verisignlabs.com/stats.html They are counting zones with TLSA records. I'm counting domains whose MX hosts have TLSA records. Those 5230 domains for udmedia.de all resolve to MX hosts in just 1 zone. My dataset has 1265 MX hosts in 1185 zones, but the number of *domains* with SMTP DANE TLS is 9513. -- Viktor. From ietf-dane at dukhovni.org Fri Nov 20 01:58:53 2015 From: ietf-dane at dukhovni.org (Viktor Dukhovni) Date: Fri, 20 Nov 2015 00:58:53 +0000 Subject: Please take care when deploying Let's Encrypt certificates... Message-ID: <20151120005852.GF18315@mournblade.imrryr.org> If you've published DANE TLSA records for your current certificate chain, and are considering switch to Let's Encrypt issued certificates, please do not forget: https://dane.sys4.de/common_mistakes#3 https://tools.ietf.org/html/rfc7671#section-8.1 I've seen more than one of the early adopters of LE certificates neglect to update their TLSA records (a few TTLs) *before* deploying the new LE certificate chain. -- Viktor. From the.lists at mgm51.com Fri Nov 20 18:21:12 2015 From: the.lists at mgm51.com (Mike) Date: Fri, 20 Nov 2015 12:21:12 -0500 Subject: Please take care when deploying Let's Encrypt certificates... In-Reply-To: <20151120005852.GF18315@mournblade.imrryr.org> References: <20151120005852.GF18315@mournblade.imrryr.org> Message-ID: <564F5688.7010300@mgm51.com> On 11/19/2015 7:58 PM, Viktor Dukhovni wrote: > > If you've published DANE TLSA records for your current certificate > chain, and are considering switch to Let's Encrypt issued certificates, > please do not forget: > > https://dane.sys4.de/common_mistakes#3 > > https://tools.ietf.org/html/rfc7671#section-8.1 > > I've seen more than one of the early adopters of LE certificates > neglect to update their TLSA records (a few TTLs) *before* deploying > the new LE certificate chain. > Something else to keep in mind with the Let's Encrypt certificates is that they have a 90-day lifetime with the automatic renewal process starting at sixty days. Using a Let's Encrypt certificate with DANE TLSA will require an alert sysadmin. https://community.letsencrypt.org/t/maximum-and-minimum-certificate-lifetimes/264/9 From ietf-dane at dukhovni.org Fri Nov 20 19:50:02 2015 From: ietf-dane at dukhovni.org (Viktor Dukhovni) Date: Fri, 20 Nov 2015 18:50:02 +0000 Subject: Please take care when deploying Let's Encrypt certificates... In-Reply-To: <564F5688.7010300@mgm51.com> References: <20151120005852.GF18315@mournblade.imrryr.org> <564F5688.7010300@mgm51.com> Message-ID: <20151120185002.GM18315@mournblade.imrryr.org> On Fri, Nov 20, 2015 at 12:21:12PM -0500, Mike wrote: > > https://dane.sys4.de/common_mistakes#3 > > > > https://tools.ietf.org/html/rfc7671#section-8.1 > > > > I've seen more than one of the early adopters of LE certificates > > neglect to update their TLSA records (a few TTLs) *before* deploying > > the new LE certificate chain. > > Something else to keep in mind with the Let's Encrypt certificates is > that they have a 90-day lifetime with the automatic renewal process > starting at sixty days. Automated replacement might make them entirely unsuitable for DANE-EE(3). That is, assuming the automation neglects the necessary DNS update precondition. One of the most important features of DANE-EE(3) is that certificates DO NOT EXPIRE with DANE-EE(3). You replace it when you are ready to do it, not when the certificate goes up in smoke. The expiration is in the RRSIG end time, not in the certificate. If you lose that with EE, DO NOT switch to LE. For port 25 SMTP it'll do more harm than good. By all means use LE for port 587 (different certs for the MTA and MSA). The only way LE for port 25 with DANE can work is if renewal is possible with the same private key, and the TLSA records are "3 1 1", making certificate replacement a non-event. The other way, is to publish "2 0 1" records for the LE root CA (which MUST then appear in the server's chain) or "2 1 1" records for the LE intermediate CA (which must appear in the server's chain, but that's more typically true anyway). Using "3 0 1" with LE 90 day certificates that are rotated automatically, sounds like a recipe for disaster, unless deployment of the new certificate can be delayed (after it is obtained) and the required DNS updates automated, with the certificate deployed only once the DNS records have been fielded sufficiently long. > Using a Let's Encrypt certificate with DANE TLSA will require an alert > sysadmin. > > https://community.letsencrypt.org/t/maximum-and-minimum-certificate-lifetimes/264/9 This does not discuss whether a new key is used for each renewal. Can anyone using LE automated rotation check whether the key stays the same or not? -- Viktor. From ietf-dane at dukhovni.org Sat Nov 21 03:06:53 2015 From: ietf-dane at dukhovni.org (Viktor Dukhovni) Date: Sat, 21 Nov 2015 02:06:53 +0000 Subject: Deployment news (comcast.net publishes TLSA RRs) In-Reply-To: <20151103201019.GP18315@mournblade.imrryr.org> References: <20151103201019.GP18315@mournblade.imrryr.org> Message-ID: <20151121020653.GK24882@mournblade.imrryr.org> On Tue, Nov 03, 2015 at 08:10:19PM +0000, Viktor Dukhovni wrote: > #Domains Provider > -------- ---------- > 33 binero.se (resolution in progress) > 28 isphuset.no (issue acknowledged) > 15 axc.nl (notified) > 5 forpsi.net (notified) DNS at binero.se is now resolved. With ongoing scans in the mean-time, the number of affected domains I managed to find was briefly more than eighty, but now it is zero. The solution was actually a software update at neustar.biz (also known as UltraDNS.net) so this also addresses the same issue for all other ultradns.net customers (only one such domain in my scans, but my surveys are far from comprehensive). Progress continues. -- Viktor. From patrickdk at patrickdk.com Sat Nov 21 03:12:49 2015 From: patrickdk at patrickdk.com (Patrick Domack) Date: Fri, 20 Nov 2015 21:12:49 -0500 Subject: Deployment news (comcast.net publishes TLSA RRs) In-Reply-To: <20151121020653.GK24882@mournblade.imrryr.org> References: <20151103201019.GP18315@mournblade.imrryr.org> <20151121020653.GK24882@mournblade.imrryr.org> Message-ID: <20151120211249.Horde.2P1jdKMbBgUQadb2woK2kUB@mail.patrickdk.com> I have been attempting to push more people to use dane, but it is hard. More and more server admins keep asking to not send email to their domains without tls verification or certificate pinning, but none of them have heard of dane. Most don't even have dnssec even. Quoting Viktor Dukhovni : > On Tue, Nov 03, 2015 at 08:10:19PM +0000, Viktor Dukhovni wrote: > >> #Domains Provider >> -------- ---------- >> 33 binero.se (resolution in progress) >> 28 isphuset.no (issue acknowledged) >> 15 axc.nl (notified) >> 5 forpsi.net (notified) > > DNS at binero.se is now resolved. With ongoing scans in the > mean-time, the number of affected domains I managed to find was > briefly more than eighty, but now it is zero. > > The solution was actually a software update at neustar.biz (also > known as UltraDNS.net) so this also addresses the same issue for > all other ultradns.net customers (only one such domain in my scans, > but my surveys are far from comprehensive). Progress continues. > > -- > Viktor. From ietf-dane at dukhovni.org Sat Nov 21 03:42:01 2015 From: ietf-dane at dukhovni.org (Viktor Dukhovni) Date: Sat, 21 Nov 2015 02:42:01 +0000 Subject: Deployment news (comcast.net publishes TLSA RRs) In-Reply-To: <20151120211249.Horde.2P1jdKMbBgUQadb2woK2kUB@mail.patrickdk.com> References: <20151103201019.GP18315@mournblade.imrryr.org> <20151121020653.GK24882@mournblade.imrryr.org> <20151120211249.Horde.2P1jdKMbBgUQadb2woK2kUB@mail.patrickdk.com> Message-ID: <20151121024200.GR18315@mournblade.imrryr.org> On Fri, Nov 20, 2015 at 09:12:49PM -0500, Patrick Domack wrote: > I have been attempting to push more people to use dane, but it is hard. > > More and more server admins keep asking to not send email to their domains > without tls verification or certificate pinning, but none of them have heard > of dane. Most don't even have dnssec even. Thanks for helping get the message out. Yes, it is difficult to get initial momentum. Still, certificate pinning scales exceedingly poorly, so it is worth trying. This is still fairly early in the adoption cycle. I need to get DANE TLS adopted into OpenSSL. Some initial code is awaiting internal team review... It will be important to see DANE support in more than Postfix and early adopter releases of Exim. The folks publishing TLSA records need only DNSSEC and some operational discipline, no need (at the same time) to deploy an MTA that can verify such records. They may need better DNSSEC tools. IIRC Microsoft significantly enhanced DNSSEC support in recent releases of ActiveDirectory. Adoption has grown from a hundred or so domains last summer to thousands now, but mostly small domains. I hope momentum will pick up once web.de and gmx.de go live. -- Viktor. From patrickdk at patrickdk.com Sat Nov 21 03:48:43 2015 From: patrickdk at patrickdk.com (Patrick Domack) Date: Fri, 20 Nov 2015 21:48:43 -0500 Subject: Deployment news (comcast.net publishes TLSA RRs) In-Reply-To: <20151121024200.GR18315@mournblade.imrryr.org> References: <20151103201019.GP18315@mournblade.imrryr.org> <20151121020653.GK24882@mournblade.imrryr.org> <20151120211249.Horde.2P1jdKMbBgUQadb2woK2kUB@mail.patrickdk.com> <20151121024200.GR18315@mournblade.imrryr.org> Message-ID: <20151120214843.Horde.ZNVhkqp6iwurS5DqgnTr2Ng@mail.patrickdk.com> Quoting Viktor Dukhovni : > On Fri, Nov 20, 2015 at 09:12:49PM -0500, Patrick Domack wrote: > >> I have been attempting to push more people to use dane, but it is hard. >> >> More and more server admins keep asking to not send email to their domains >> without tls verification or certificate pinning, but none of them have heard >> of dane. Most don't even have dnssec even. > > Thanks for helping get the message out. Yes, it is difficult to > get initial momentum. Still, certificate pinning scales exceedingly > poorly, so it is worth trying. > > This is still fairly early in the adoption cycle. I need to get > DANE TLS adopted into OpenSSL. Some initial code is awaiting > internal team review... It will be important to see DANE support > in more than Postfix and early adopter releases of Exim. > > The folks publishing TLSA records need only DNSSEC and some > operational discipline, no need (at the same time) to deploy an > MTA that can verify such records. They may need better DNSSEC > tools. IIRC Microsoft significantly enhanced DNSSEC support in > recent releases of ActiveDirectory. > > Adoption has grown from a hundred or so domains last summer to > thousands now, but mostly small domains. I hope momentum will pick > up once web.de and gmx.de go live. Yes, I have noticed it is a big movement in germany. Have had a lot of people asking for help on setting up dane the last few months from there. But can't get any movement that is noticable here in the usa. From ietf-dane at dukhovni.org Sat Nov 21 04:28:11 2015 From: ietf-dane at dukhovni.org (Viktor Dukhovni) Date: Sat, 21 Nov 2015 03:28:11 +0000 Subject: Deployment news (comcast.net publishes TLSA RRs) In-Reply-To: <20151120214843.Horde.ZNVhkqp6iwurS5DqgnTr2Ng@mail.patrickdk.com> References: <20151103201019.GP18315@mournblade.imrryr.org> <20151121020653.GK24882@mournblade.imrryr.org> <20151120211249.Horde.2P1jdKMbBgUQadb2woK2kUB@mail.patrickdk.com> <20151121024200.GR18315@mournblade.imrryr.org> <20151120214843.Horde.ZNVhkqp6iwurS5DqgnTr2Ng@mail.patrickdk.com> Message-ID: <20151121032811.GS18315@mournblade.imrryr.org> On Fri, Nov 20, 2015 at 09:48:43PM -0500, Patrick Domack wrote: > Yes, I have noticed it is a big movement in germany. Have had a lot of > people asking for help on setting up dane the last few months from there. > But can't get any movement that is noticable here in the usa. I think that what's needed is getting software support for DANE into OpenSSL, mTLS and GnuTLS, plus adoption by the SMTP major appliance vendors, Ironport, Proofpoint, ... and of course Microsoft Exchange. There's still some work to do. Making it easier to update the DNS with the right records would also help, sadly there's no satisfactory and standard management interface with a decent access control model. So automating publication of TLSA records is difficult. Perhaps we need a new protocol by which a TLS server can securely pre-publish the next certificate without activating it (say include it in a new TLS extension), thus allowing the DNS server operator to automate TLSA record updates by querying the SMTP server (authenticated via the current records). If anyone has better ideas to automate coordination of DNS updates and key rotation, I'm all ears... -- Viktor. From patrickdk at patrickdk.com Sat Nov 21 04:38:14 2015 From: patrickdk at patrickdk.com (Patrick Domack) Date: Fri, 20 Nov 2015 22:38:14 -0500 Subject: Deployment news (comcast.net publishes TLSA RRs) In-Reply-To: <20151121032811.GS18315@mournblade.imrryr.org> References: <20151103201019.GP18315@mournblade.imrryr.org> <20151121020653.GK24882@mournblade.imrryr.org> <20151120211249.Horde.2P1jdKMbBgUQadb2woK2kUB@mail.patrickdk.com> <20151121024200.GR18315@mournblade.imrryr.org> <20151120214843.Horde.ZNVhkqp6iwurS5DqgnTr2Ng@mail.patrickdk.com> <20151121032811.GS18315@mournblade.imrryr.org> Message-ID: <20151120223814.Horde.6bIEC7PZ_UuMyOmB8ys01rU@mail.patrickdk.com> Quoting Viktor Dukhovni : > On Fri, Nov 20, 2015 at 09:48:43PM -0500, Patrick Domack wrote: > >> Yes, I have noticed it is a big movement in germany. Have had a lot of >> people asking for help on setting up dane the last few months from there. >> But can't get any movement that is noticable here in the usa. > > Perhaps we need a new protocol by which a TLS server can securely > pre-publish the next certificate without activating it (say include > it in a new TLS extension), thus allowing the DNS server operator > to automate TLSA record updates by querying the SMTP server > (authenticated via the current records). That sounds pretty difficult to adjust for, and would need a lot of changes. I like the current dnssec method, where we can publish multiple keys. I will generally publish a new key a month ahead of time for my ksk rollover, then rotate it, and then a month later remove the old key. The same method could be done for tlsa, by publishing multiple records. I have not tested if any software accepts this or not, but just publishing the new one a week ahead of time, rotating it, and removing the old one at the same or later time (in case of failback), to me sounds like the perferred method. From ietf-dane at dukhovni.org Sat Nov 21 04:49:35 2015 From: ietf-dane at dukhovni.org (Viktor Dukhovni) Date: Sat, 21 Nov 2015 03:49:35 +0000 Subject: Deployment news (comcast.net publishes TLSA RRs) In-Reply-To: <20151120223814.Horde.6bIEC7PZ_UuMyOmB8ys01rU@mail.patrickdk.com> References: <20151103201019.GP18315@mournblade.imrryr.org> <20151121020653.GK24882@mournblade.imrryr.org> <20151120211249.Horde.2P1jdKMbBgUQadb2woK2kUB@mail.patrickdk.com> <20151121024200.GR18315@mournblade.imrryr.org> <20151120214843.Horde.ZNVhkqp6iwurS5DqgnTr2Ng@mail.patrickdk.com> <20151121032811.GS18315@mournblade.imrryr.org> <20151120223814.Horde.6bIEC7PZ_UuMyOmB8ys01rU@mail.patrickdk.com> Message-ID: <20151121034935.GT18315@mournblade.imrryr.org> On Fri, Nov 20, 2015 at 10:38:14PM -0500, Patrick Domack wrote: > >Perhaps we need a new protocol by which a TLS server can securely > >pre-publish the next certificate without activating it (say include > >it in a new TLS extension), thus allowing the DNS server operator > >to automate TLSA record updates by querying the SMTP server > >(authenticated via the current records). > > That sounds pretty difficult to adjust for, and would need a lot of changes. Well, it requires IETF action and software support on the server side, yes. > I like the current dnssec method, where we can publish multiple keys. I will > generally publish a new key a month ahead of time for my ksk rollover, then > rotate it, and then a month later remove the old key. This works today, but automating the publication of the "future" TLSA record is the part that some will struggle with. > The same method could be done for tlsa, by publishing multiple records. Not could, is. See: https://tools.ietf.org/html/rfc7671#section-8.1 ... > I have not tested if any software accepts this or not, but just publishing the > new one a week ahead of time, rotating it, and removing the old one at the > same or later time (in case of failback), to me sounds like the perferred > method. Yes, that's what section 8 of RFC7671 describes. And of course this is supported by Postfix and Exim, and will soon(ish) be supported by OpenSSL. The problem is that if certificate deployment is fully automated (see today's Let's Encrypt thread on this list), then prepublishing the next TLSA record becomes difficult. The work-arounds, are: * Keep the key the same while getting new certificates for it, publish "DANE-EE(3) SPKI(1) SHA2-256(1)" records, change keys manually, and go through the usual prepublication process for the key digest. * Switch to using "DANE-TA(2) SPKI(1) SHA2-256(1)" records, with a selector of SPKI rather that Cert, because with a third-party issuing CA, you have less control over when the CA cert might get re-issued to extend its life, or cross-sign it, ... Then you're good while the same CA (private key) continues to issue your certificates. The CA cert needs to appear in your chain even if it is a root CA. https://tools.ietf.org/html/rfc7671#section-5.2 -- Viktor.