PowerDNS Authoritative Server 4.0.6 & 4.1.5 and Recursor 4.0.9 & 4.1.5 Released

We’ve released PowerDNS Authoritative Server 4.0.6 & 4.1.5 and Recursor 4.0.9 & 4.1.5.

These are security releases with additional minor improvements and bug fixes.

Minimal patches for the releases are available at https://downloads.powerdns.com/patches/.

The changelogs look as follows:

Authoritative Server 4.1.5

This release fixes the following security advisories:

  • PowerDNS Security Advisory 2018-03 (CVE-2018-10851)
  • PowerDNS Security Advisory 2018-05 (CVE-2018-14626)


  • Apply alias scopemask after chasing (#6976)
  • Release memory in case of error in the openssl ecdsa constructor (#6917)
  • Switch to devtoolset 7 for el6 (#7118, #7040)

Bug Fixes

  • Crafted zone record can cause a denial of service (CVE-2018-10851, #7149)
  • Packet cache pollution via crafted query (CVE-2018-14626, #7149)
  • Fix compilation with libressl 2.7.0+ (#6948, #6943)
  • Actually truncate truncated responses (#6913)

Authoritative Server 4.0.6

This release fixes PowerDNS Security Advisory 2018-03 (CVE-2018-10851).

Bug fixes

  • Crafted zone record can cause a denial of service (CVE-2018-10851, #7150)
  • Skip v6-dependent test when pdns_test_no_ipv6 is set in environment (#6013)
  • Fix el6 builds (#7135)


  • Prevent cname + other data with dnsupdate (#6315)
  • Switch to devtoolset 7 for el6 (#7119)

Recursor 4.1.5

This release fixes the following security advisories:

  • PowerDNS Security Advisory 2018-04 (CVE-2018-10851)
  • PowerDNS Security Advisory 2018-06 (CVE-2018-14626)
  • PowerDNS Security Advisory 2018-07 (CVE-2018-14644)


  • Add pdnslog to lua configuration scripts (Chris Hofstaedtler) (#6919, #6848)
  • Fix compilation with libressl 2.7.0+ (#6948, #6943)
  • Export outgoing ECS value and server ID in protobuf (if any) (#7004, #6991, #6989)
  • Switch to devtoolset 7 for el6 (#7122, #7040)
  • Allow the signature inception to be off by a number of seconds (Kees Monshouwer) (#7125, #7081)

Bug Fixes

  • Crafted answer can cause a denial of service (CVE-2018-10851, #7151)
  • Packet cache pollution via crafted query (CVE-2018-14626, #7151)
  • Crafted query for meta-types can cause a denial of service (CVE-2018-14644, #7151)
  • Delay the creation of rpz threads until we have dropped privileges (#6984 #6792)
  • Cleanup the netmask trees used for the ecs index on removals (#6961 #6960)
  • Make sure that the ecs scope from the auth is < to the source (#6963, #6605)
  • Authority records in aa=1 cname answer are authoritative (#6980, #6979)
  • Avoid a memory leak in catch-all exception handler (#7073)
  • Don’t require authoritative answers for forward-recurse zones (#6741, #6340)
  • Release memory in case of error in the openssl ecdsa constructor (#6917)
  • Convert a few uses to toLogString to print DNSName’s that may be empty in a safer manner (#6925, #6924)
  • Avoid a crash on DEC Alpha systems (#6945)
  • Clear all caches on (N)TA changes (#6951, #6949)

Recursor 4.0.9

This release fixes the following security advisories:

  • PowerDNS Security Advisory 2018-04 (CVE-2018-10851)
  • PowerDNS Security Advisory 2018-06 (CVE-2018-14626)
  • PowerDNS Security Advisory 2018-07 (CVE-2018-14644)

Bug fixes

  • Crafted answer can cause a denial of service (CVE-2018-10851, #7152)
  • Packet cache pollution via crafted query (CVE-2018-14626, #7152)
  • Crafted query for meta-types can cause a denial of service (CVE-2018-14644, #7152)

The tarballs and signatures are available at downloads.powerdns.com and packages for CentOS 6 and 7, Debian Jessie and Stretch, Ubuntu Bionic, Trusty and Xenial are available from repo.powerdns.com.  Raspberry PI packages will follow tomorrow.

Please send us all feedback and issues you might have via the mailing list, or in case of a bug, via GitHub.

FOSDEM 2019 DNS devroom CfP

Hello DNS enthusiasts and other developers,

After a successful and packed half-day DNS devroom at FOSDEM 2018, we are happy to announce a full-day DNS devroom at FOSDEM 2019.

As with last year, we hope to host talks anywhere from hardcore protocol stuff, to practical sessions for programmers that are not directly involved with DNS but may have to deal with DNS in their day to day coding or system administrators responsible for DNS infrastructure.

We have been allotted a room on Sunday 3 February 2019. We expect to schedule 30 minutes per talk, including questions, but if you need more or less time, we can discuss this.

If you have something you’d like to share with your fellow developers, please head to pentabarf at https://penta.fosdem.org/submission/FOSDEM19. Examples of topics are measuring, monitoring, DNS libraries, and anecdotes on how you’ve (ab)used the DNS. Here’s the 2018 schedule, for your inspiration: https://archive.fosdem.org/2018/schedule/track/dns/ .

The deadline for submission is December 1st. If you have a FOSDEM pentabarf account from a previous year, please use that account. Reach out to dns-devroom-manager@fosdem.org if you run into any trouble.

See you there!

Peter van Dijk, Shane Kerr, Pieter Lexis, and Kees Monshouwer

PowerDNS and the ICANN KSK roll

The root KSK rollover is currently planned for 1600 UTC on the 11th of October 2018 – a few days from now. If you are using PowerDNS Recursor for DNSSEC validation, please keep reading!

During the KSK rollover, the root zone will stop using the old root Key Signing Key, known as KSK-2010 or 19036, and will start using the new Key Signing Key, known as KSK-2017 or 20326. Your Recursor needs to be aware of both keys to make sure validation keeps working after the rollover event.

If you are running Recursor 4.0.5 or up, both keys come preconfigured. If you are running an older 4.0.x version, it is possible your distribution has added the key for you.

In case of any doubt, verify you are ready:

# rec_control --socket-dir=. get-tas
Configured Trust Anchors:
19036 8 2 49aac11d7b6f6446702e54a1607371607a1a41855200fd2ce1cdde32f24e8fb5
20326 8 2 e06d44b80b8f1d39a95c0b0d7c65d08458e880409bbc683457104237c7f8ec8d

The output should have both the 19036 and 20326 lines. If 20326 is missing, please upgrade your Recursor. If for some reason upgrading is not feasible for you right now, please follow the PowerDNS Recursor instructions that ICANN published. Those instructions involve a restart; if you want to avoid a restart this week, please see Runtime Configuration of Trust Anchors in the PowerDNS documentation.

In case of panic (in the unlikely event ICANN botches the roll, or the roll finds a bug in our software), you can run rec_control add-nta . DNSSEC on root is broken to disable DNSSEC immediately without restarting your daemon.

Should you have any trouble: if you are a supported customer, please reach out through the usual channels. Otherwise, contact us via our community channels.

Spoofing DNS with fragments

With some care, it turns out to be possible to spoof fake DNS responses using fragmented datagrams. While preparing a presentation for XS4ALL back in 2009, I found out how this could be done, but I never got round to formally publishing the technique. The presentation was however made available.

Update: this “discovery” has now been dated back to at least 2008 when Florian Weimer knew about it & tells us it was communicated clearly and widely back then.

In 2013, Amir Herzberg & Haya Shulman (while at Bar Ilan University) published a paper called Fragmentation Considered Poisonous. In this paper they explain how fragmented DNS responses can be used for cache poisoning. Later that year CZNIC presented about this paper and its techniques at RIPE 67.

A stunning 72 papers cite the original article, but as of 2018 not too many people know about this cache poisoning method.

More recently, The Register reported that another team, also involving Dr Shulman (now at Fraunhofer Institute for Secure Information Technology), has been able to use fragmented DNS responses to acquire certificates for domain names whose nameservers they do not control. They were able to demonstrate this in real life, which is a remarkable achievement. Incidentally, this team includes Amit Klein who in 2008 discovered & reported a weakness in PowerDNS.

Full details will be presented at the ACM Conference on Computer and Communications Security in Toronto, October 18. This presentation will also propose countermeasures.

Meanwhile, in this post, I hope to explain a (likely) part of their technique.

Whole datagram DNS spoofing

To match bona fide DNS responses to their corresponding queries, resolvers and operating system check:

  • Name of the query
  • Type of the query
  • Source/destination address
  • Destination port (16 bits)
  • DNS transaction ID (16 bits)

The first three items can be predictable, the last two aren’t supposed to be. To spoof in a false response therefore means we need to guess 32 bits of random. To do so, the attacker needs to send the resolver lots and lots of fake answers with guesses for destination port and the transaction ID. Over (prolonged) time, their chosen response arrives ahead of the authentic response, is accepted, and they are able to spoof a domain name. Profit.

Screenshot from 2018-09-10 22-12-57

In practice this turns out to be very hard to do. The 32 bit requirement plus the short timeframe in which to send false responses means that as far as I know, this has been demonstrated in a lab setting just once. Anecdotal reports of blindly spoofing a fully randomized source port resolver have not been substantiated.


DNS queries and responses can be carried in UDP datagrams. A UDP datagram can be many kilobytes in size – far larger than most UDP packets. This means that a sufficiently large UDP response datagram can get split up into multiple packets. These are then called fragments.

Such fragments travel the network separately, to be joined together again on receipt.

Fragmented DNS responses happen occasionally with DNSSEC, for example in this case:

$ dig -t mx  isc.org @ams.sns-pb.isc.org +dnssec -4 +bufsize=16000
43.028963 IP > 20903+ [1au] MX? isc.org. (48)
43.035379 IP > 20903*- 3/5/21 
  MX mx.ams1.isc.org. 20, MX mx.pao1.isc.org. 10, RRSIG (1472)
43.035391 IP > ip-proto-17

The final line represents a fragment, which only notes it is UDP (protocol 17).

Matching fragments together is quite comparable to matching DNS queries to responses. Every IP packet, even a fragment, carries a 16 bit number called an IPID. This IPID is not copied from the query to the response, it is picked by the DNS responder.

Screenshot from 2018-09-10 22-13-30

On receipt, fragments are grouped by IPID, after which the checksum of the reassembled datagram is checked. If correct, the DNS response gets forwarded to the resolver process.

If we want to spoof a DNS response, we could pick a DNS query that leads to a fragmented datagram, and then try to spoof only the second fragment. On first sight, this does not appear to be much easier as we now need to guess the IPID (16 bits) and we also need to make sure the checksum of the whole datagram matches (another 16 bits). This then also requires a 32 bit guess to succeed.

However, if we send a server a DNS query, it will most of the time send the same DNS response to everyone who asks (also for fragmented answers). In other words, if the attacker wants to spoof a certain response, it will know exactly what that response looks like – with the exception of the destination port and the DNS transaction ID (32 bits).

But note that both of these unpredictable parts are in the first fragment. The second fragment is completely static, except for the IPID. Now for the clever bit.

The ‘internet checksum’ is literally .. a sum. So the checksum of the entire datagram consists of the checksum of the first fragment plus the checksum of the second fragment (modulo 16 bits).

Screenshot from 2018-09-10 22-13-49

This means that to make sure the whole reassembled datagram passes the checksum test, all we have to do is make sure that our fake second fragment has the same known partial checksum as the original. We can pick the checksum of our fake second segment easily through the TTL of the our chosen response record.

This leaves us with only 16 bits to guess, which given the birthday paradox is not that hard.


Randomness of the IPID

So how random is the IPID, does it even represent a 16-bit challenge? According to the 2013 paper, some operating systems pick the IPID from a global counter. This means an attacker can learn the currently used IPID and predict the one used for the next response with pretty good accuracy.

Other operating systems use an IPID that increments per destination which means we can’t remotely guess the IPID. It turns out however that through clever use of multiple fragments, this still allows an attacker to “capture” one of these. See the original paper for details.

Is that it?

Definitely not. In order to get a certificate issued falsely using this technique requires several additional elements. First we must be able to force many questions. Secondly, we must make sure that the original authoritative server fragments the answer just right. There are ways to do both, but they are not easy.

I await the presentation at the ACM conference in October eagerly – but I’m pretty sure it will build on the technique outlined above.


In the meantime, DNSSEC does actually protect against this vulnerability, but it does require that your domain is signed and that your CA validates. This may not yet be the case.

On Firefox moving DNS to a third party

DNS lookups occur for every website visited. The processor of DNS requests gets a complete picture of what a household or phone is doing on the internet. In addition, DNS can be used to block sites or to discover if devices are accessing malware or are part of a botnet.

(for the tl;dr, please skip right to the summary at the end)

Recently, we’ve seen Cloudflare (rumoured to be heading to IPO soon) get interested in improving your DNS privacy. Through a collaboration with Mozilla, Cloudflare is offering to move Firefox DNS lookups from the subscriber’s service provider straight onto its own systems. From a variety of blog posts it appears that Mozilla is aiming to make this the new default, although we also hear the decision has not yet been taken and that other organizations beyond Cloudflare may be involved. This new DNS service will be encrypted, using a protocol called DNS over HTTPS.

We are currently living in strange times where companies are willing to offer us services for “free” in return for access to our data. This data can then be used for profiling purposes (targeted advertising) or competitive analysis (market intelligence, for example what kinds of people visit what sites etc). In this way, if you are getting something for free, you frequently aren’t the customer, you are the product.

In addition, once our data flows through a third party, it is possible for that third party to influence what we see or how well things work: Gmail moving your school newsletter to the never opened ‘Promotional’ tab, Facebook suddenly no longer displaying your updates to users unless you pay up, Outlook.com deciding that most independent email providers should end up in the spam folder.

At Open-Xchange and PowerDNS, we think further centralization of the internet is a bad thing in and of itself, so we are not happy about the idea of moving DNS to a large, central, third party. Centralization means permissionless innovation becomes harder, when it was this very permissionless innovation that gave us the internet as we know it today.

We do of course applaud giving users a choice of encrypted DNS providers. Our worry is about the mulled plan to switch users over by default, or asking users to make an uninformed choice to switch to “better, more private DNS”, without making sure consumers know what is going on. Because that ‘OK, Got It’ button will frequently just get clicked.

Good thing it is encrypted and secure

Beyond our worries about centralization however there are concrete reasons to think twice before changing the DNS trust model & moving queries to a third party by default.

What will change?

When a user wants to visit ‘www.wikipedia.org’, the browser first looks up the IP address for this site. As it stands, by default, the service provider nameserver is consulted for this purpose. The setting for this is hidden in the Cable/DSL/FTTH-modem or phone. In the newly proposed world, the browser would ask Cloudflare for the IP address of ‘www.wikipedia.org’. Cloudflare says it takes your privacy more seriously than telecommunication service providers do because this DNS query will be encrypted, unlike regular DNS. They also promise not to sell your data or engage in user profiling.

Cloudflare and Mozilla have set out a privacy policy that rules out any form of customer profiling. Their story is that many ISPs are doing user profiling and marketing, and that moving your DNS to Cloudflare is therefore a win for your privacy.

Interestingly, this claim cannot be true in Europe.The EU GDPR and telecom regulations greatly limit what ISPs could do with the data. Selling it on is absolutely forbidden. Service providers would be risking 4% revenue fines because doing this secretly would be in stark violation of the GDPR, Europe’s privacy regulation.

In other countries, service providers do indeed study and use their user’s traffic patterns for marketing purposes.

So given this, under what circumstances would it be ok for Cloudflare (or any other third party) to take over our DNS by default?


Cloudflare is a Content Delivery Network (CDN). CDNs serve website content & videos from servers across the globe, so that content is closer to the end-user. As it stands, large scale CDNs like Akamai, Fastly, Google, Level3 and Cloudflare cooperate and coordinate intimately with service providers, to the point of co-locating caches within ISP networks to guarantee rapid delivery of content. When connecting to ‘www.whitehouse.gov’ for example, it is entirely possible to end up on an Akamai server hosted within your own service provider in the city you live in.  Only two companies were then involved in delivering that page to you: your ISP and Akamai. Neither your request, nor the response ever left your own country.

In the proposed future where Cloudflare does our DNS, all queries go through their networks first before we reach content hosted by them, or their competitors. We can legitimately wonder if Cloudflare will diligently work to protect the interests of its competitors and deliver the best service it can.

Interestingly enough, as of today, at least for KPN (a national service provider in The Netherlands) and www.whitehouse.gov this is not true: the IP address we mostly get from the KPN servers is 20% closer in terms of latency, and is reached through Internet peering. The IP address we get via Cloudflare is slower and additionally reached through IP transit, which is more expensive for both KPN and Akamai. Cloudflare is therefore slowing down access to an Akamai hosted website, at higher cost for everyone involved. Cloudflare, incidentally, explains that this is because of privacy reasons.

Any new default DNS provider should commit to working with all its competitors to deliver service that is as good as would have been provided through the service providers’ DNS.


Any chokepoint of communications is susceptible to government blocking orders and legal procedures. In some countries the government shows up with a (long) list of what domains to block, in other countries this happens only after a series of long-winded lawsuits. In addition, child pornography researchers (& law enforcement organizations) frequently provide lists of domains they think should be blocked, and these often are.  

Local service providers typically fight attempts to block popular content, since their subscribers don’t like it. Once an international DNS provider is the default for lookups, it can also expect government orders and other legal efforts aimed to get domain names blocked.

A new default DNS provider should document its policies on how it will deal with lawsuits and government orders commanding it to block traffic. At the very least, blocks should be constrained regionally. It should also document what content they would block out of their own accord.

Government interception

Without going all “Snowden” on this subject, many governments grant themselves rights to intercept foreign communications with far less oversight than if they were intercepting national traffic. In other words, citizens of country X enjoy far less privacy protection in country Y. This is not a controversial statement and is explicitly written out in many countries’ interception laws and regulations. But the upshot is that for privacy, it pays to keep DNS within the country where you are a citizen.

In addition, most countries have legislated that communications service providers can and must break their own contracts, terms and conditions to comply with government interception orders. In other words, even though a company has committed in writing to not share your data with anyone, if the government shows up, they can be forced to do so anyhow.

It may well be that a third party DNS provider operates under a regime that has an interest in the DNS traffic that gets sent to it from all over the world.

New centralised DNS providers should document which governments have interception powers over them and be honest about their chances of standing up to such interception.

Losing control

DNS is currently under control of your network provider – which could be your employer, your coffee shop or frequently, your (Internet) service provider. Enterprise environments often filter DNS for malware related traffic, blocking requests for known harmful domain names. They will also use query logs to spot infected devices. Increasingly, large scale service providers are also offering DNS based malware filtering, especially in the UK.

When moving DNS to a centralised provider, such local filtering no longer functions. Enterprise network administrators will also lose visibility into what traverses their network. From the standpoint of the individual employee this may be great but it is not what the network operator wanted.

Interestingly enough, DNS over HTTPS has specifically been designed to be hard to block, as the designers envisioned that network operators would attempt to use firewall rules to disable forms of DNS they could not monitor or control.

When asking users if they should move their DNS to a new provider, they should be reminded they may be losing protection that was previously provided to them by their service provider or employer network administrators.

Is your service provider actually spying on you?

If we want to assess the benefit of moving DNS to a third party by default, it is important to know if we are being spied upon in the first place. In some cases and in some countries, this is definitely true. In Russia and China, DNS is routinely intercepted and even changed. Also, some providers replace ‘this domain does not exist’ DNS answers by the IP address of a ‘search page’ with advertisements.

But in many places, local service providers are bound by stringent rules that forbid any spying or profiling, mostly countries that fall under the European GDPR or GDPR inspired legislation.

Screenshot from 2018-09-04 17-15-20

A non-scientific Twitter poll

It has been argued that users are not sophisticated enough to reason about this subject and that the DNS move should happen by default, with an opt-out for those that care. Another idea that has been raised is a startup dialogue that proposes a more secure internet experience and a ‘Got it!’ button. This clearly does not go far enough in educating users about the change they will be authorizing.

Before moving DNS to a third party, users should be surveyed if they feel their current provider is spying on them or not, and if they think the new third party DNS provider would be an improvement. The outcome will likely be different per region. This survey could then lead to a well-designed, localized, opt-in procedure.


Having a choice of (encrypted) DNS providers is good. Mozilla is pondering moving DNS resolution to a third party by default, initially Cloudflare. Before doing so, any third party should commit to:

  • Network neutrality: promise to work with competitors to ensure performance for other CDNs does not deteriorate compared to when the service provider DNS was used
  • A policy on blocking: how will the provider deal with government blocking requests or lawsuits demanding that content will be blocked.
  • Warning users the new DNS may not offer safety features they got from the network DNS provider
  • Being clear about the legislations it operates under: which governments could force it into large scale interception?

Finally, Mozilla should survey its users to find out their attitudes towards moving DNS from their current service provider to Cloudflare. To do so, those users must first be well informed about what such a move would mean. Based on the survey results, an honest consent page can be generated that makes sure users know what they are agreeing to.

We want to thank Rudolf van der Berg and Remco van Mook for their comments & input for this post. These opinions are ours alone though. 

PowerDNS Recursor 4.1.4 released

We’ve just released PowerDNS Recursor version 4.1.4, this is a maintenance release with no major changes.

One new setting was added, this is the max-udp-queries-per-round, which controls the maximum amount of messages the Recursor will handle before other mthreads are scheduled. Its default should be high enough for nearly all users.

The changelog looks as follows:


  • #6436: Split pdns_enable_unit_tests. (Chris Hofstaedtler)
  • #6518: Add a new max-udp-queries-per-round setting.
  • #6590: Fix warnings reported by gcc 8.1.0.
  • #6809: Tests: replace awk command by perl.
  • #6720: Allow the snmp thread to retrieve statistics.

Bug Fixes

  • #6462, #6465: Don’t account chained queries more than once.
  • #6536, #6557: Make rec_control respect include-dir.
  • #6567, #6812: Load lua scripts only in worker threads.
  • #6873: Purge all auth/forward zone data including subtree. (@phonedph1)

The source tarball is available from downloads.powerdns.com (sig) and packages for Centos 6 and 7, Ubuntu Xenial, Trusty and Bionic and Debian Jessie and Stretch are available from our repositories.

PowerDNS Authoritative Server 4.1.4 released

We’re happy to announce the release of the PowerDNS Authoritative server version 4.1.4. This maintenance release features several bug fixes. The two most important being a fix in the handling of child-zones with DNS UPDATE and one where orphaned record comments would mess up records returned by the API.

Here is the list of changes, the full changelog is available in the documentation:


  • #6590: Fix warnings reported by gcc 8.1.0.
  • #6632, #6844, #6842, #6848: Make the gmysql backend future-proof. (Chris Hofstaedtler)
  • #6685, #6686: Initialize some missed qtypes. (Chris Hofstaedtler)

Bug Fixes

  • #6780: Avoid concurrent records/comments iteration from running out of sync.
  • #6816: Fix a crash in the API when adding records.
  • #4457, #6691: pdns_control notify: handle slave without renotify properly. (Chris Hofstaedtler)
  • #6736, #6738: Reset the TSIG state between queries.
  • #6857: Remove SOA-check backoff on incoming notify and fix lock handling. (Klaus Darilion)
  • #6858: Fix an issue where updating a record via DNS-UPDATE in a child zone that also exists in the parent zone, we would incorrectly apply the update to the parent zone.
  • #6676, #6677: Geoipbackend: check geoip_id_by_addr_gl and geoip_id_by_addr_v6_gl return value. (Aki Tuomi)

The tarball is here (sig) and packages for CentOS 6 and 7, Ubuntu Trusty, Xenial and Bionic and Debian Jessie and Stretch are available from the repositories.