Category: Uncategorized

PowerDNS Recursor 4.1.11 Released

Since Spectre / Meltdown, system calls have become more expensive. In
addition, relevant versions of glibc turn out to implement pthread_cond_wait
and pthread_cond_signal in such a way that they use multiple system calls always.
There is an optimization in glibc to improve this but it is disabled.

This new setup changes our protobuf logging so it amortizes system calls so we perform
far less than one call per message.

Note that our previous RemoteLogger was configured in terms of how many
messages it would buffer. Our new code is configured in terms of how many
bytes. I have multiplied the configured numbers by 100 elsewhere (recursor
config, dnsdist config) to sort of maintain parity.

In addition, the old RemoteLogger would buffer messages while there was no
connection available. We no longer do this.

Finally new, every ‘reconnectTimeout’ seconds we will flush our buffers
opportunistically to not keep people waiting.

The changelog:

  • #7434: Add an option to export only responses over protobuf
  • #7430: Reduce systemcall usage in protobuf logging

The tarball (signature) is available at downloads.powerdns.com and packages for CentOS 6 and 7, Debian Jessie and Stretch, Ubuntu Bionic, Trusty and Xenial are available from repo.powerdns.com.

Please send us all feedback and issues you might have via the mailing list, or in case of a bug, via GitHub.

PowerDNS Authoritative Server 4.1.6 Released

This release fixes a single issue: the PowerDNS API would accept more than one CNAME record for the same name attribute and would return 204 instead of refusing and returning a 4xx error.

The changelog:

  • #7279: Prevent more than one CNAME/SOA record in the same RRset

The tarball (signature) is available at downloads.powerdns.com and packages for CentOS 6 and 7, Debian Jessie and Stretch, Ubuntu Bionic, Trusty and Xenial are available from repo.powerdns.com.

Please send us all feedback and issues you might have via the mailing list, or in case of a bug, via GitHub.

PowerDNS Recursor 4.1.10 Released

This release is only relevant when building PowerDNS Recursor from source with protobuf support disabled.

Release 4.1.10 fixes a bug where the recursor would not build when one had protobuf support disabled.

The changelog:

  • #7403: Fix compilation in handleRunningTCPQuestion without protobuf support

The tarball (signature) is available at downloads.powerdns.com and packages for CentOS 6 and 7, Debian Jessie and Stretch, Ubuntu Bionic, Trusty and Xenial are available from repo.powerdns.com.

Please send us all feedback and issues you might have via the mailing list, or in case of a bug, via GitHub.

Domain security outside of DNS: Getting hacked administratively

This is a brief blogpost on the news that has been sent to us by many people, namely that there is a suspected Iranian group that is “hijacking DNS”. I was about to be interviewed on this subject but sadly that fell through. I did however prepare notes already, so please find some possibly useful things on the subject here.

Briefly: the weakest part of your DNS security currently likely isn’t actually DNS. It is the login (and password reset mailbox) where you manage your domains & nameserver settings. 

In general, if an attacker wants to take over a service you provide (a website, email or whatever), this requires them to change or redirect traffic between users and the targeted service.

There are four “gates” that determine how information flows from/to a named service:

  1. The nameserver configuration for a domain name (“the names of the nameservers”)
  2. What those configured nameserver names respond with.
  3. Which cables those IP addresses are routed to: the Border Gateway Protocol
  4. The actual servers and the software they run

Many attacks have historically focused on item 4, hacking either servers or software. For decades this was the easiest way. Recently however, the most used pieces of software have become more secure, and operators also update their software far more faithfully. Often this is done on their behalf by cloud providers.

Meanwhile, we are seeing a lot more attacks that involve BGP hijacks to route the (correct) IP addresses to incorrect locations.

The currently discussed attack involves items one and two. Why attack a server if you can reroute all traffic with a simple login? Once an attacker is logged in to a domain management solution they can change whatever they want.

So, how are these systems attacked? In the simplest case, an important domain is hosted at a registrar and protected only by a weak or leaked password. We may wonder how this is possible but it happens a lot. The “most important domain” for many companies was frequently registered by the founder ages ago. And unlike all new domains, it still languishes at a relatively unknown provider where it was registered back in the 1990s. This for example is the case for our own ‘powerdns.com’ domain.

And this original founder may not have been a security professional and picked ‘password123’ as his password. Or perhaps he did pick a good password, but it has now leaked in one of the massive breaches over the past decades.

Secondly, almost all DNS control panels come with a password reset solution that typically sends a reset mail.. to a preconfigured email address. This email address again might not be a well secured Gmail account but some ancient Yahoo mailbox that hasn’t been touched in years – again likely with a 1990s-era  password.

If that doesn’t work, an attacker has further options. If gaining access to the control panel of ‘importantdomain.com’ fails on the first try, and the password recovery email address is ‘john@companyfounder.com’, we can repeat the whole process over at ‘companyfounder.com’!

Perhaps someone over time improved the password for ‘importantdomain.com’ but not for the control panel of ‘companyfounder.com’. And if we control the ‘companyfounder.com’ domain, we can hijack the account recovery email, and thence take over ‘importantdomain.com’.

Beyond, even if that fails, we can repeat the whole process for the accounts not of ‘importantdomain.com’, but for the domain that contains the names of the nameservers themselves. Once an atttacker changes those, they can substitute nameservers that give answers that are implement the hijack.

The options to attack a domain administratively go on and on and on. It is therefore indeed very plausible that attackers have been able to acquire control of large numbers of domains.

So, what should operators do? The standing recommendation is of course to enable two-factor authentication for all control panels and to make sure that any remaining account recovery mailboxes are very well secured. Despite our very low opinions of Google’s stance on privacy, currently almost nothing beats a GMail account that itself is two-factor secured.

In addition, some domains can be “registry locked”, which is also highly recommended for high-profile domains.

But the most important recommendation is to audit each and every domain name of the company to see if these security measures have actually been taken. Because through the hops as described above, ‘importantdomain.com’ may in fact be hijacked via the nameserver of the domain of the mailbox of the company founder.

As a final note, it is often claimed that DNSSEC and TLS will protect against these attacks. While adding cryptography does raise the bar, and sometimes significantly, the control panels we have discussed so far include options to disable DNSSEC, while a DNS hijack enables an attacker to get fresh all new TLS certificates under their control within seconds. Using DNSSEC and TLS judiciously requires attackers to work harder, but it is no guarantee.

I hope this has been helpful!

PowerDNS Authoritative Server 4.2.0-alpha1: Lua records, ixfrdist, swagger

We’re proud to release the first alpha version of the PowerDNS Authoritative Server 4.2 series. While some users have already deployed this version straight from our package builders or master repositories, this is still a very fresh release.
4.2 represents almost a year of development over 4.1 and contains some major new features and improvements, while deprecating some functionality you may have been relying on (autoserial, for example).

Lua records

An important new feature is the support for Lua Records, which make the following possible, from any backend (even BIND!):

@ IN LUA A "ifportup(443, {'52.48.64.3', '45.55.10.200'})"

This will poll the named IP addresses (in the background) and only serve up hosts that are available. Far more powerful constructs are possible, for example to pick servers from regional pools close to the user, except if all servers in that pool are down. It is also possible to do traffic engineering based on subnets or AS numbers. A simple example:
@    IN   LUA A ( "ifportup(443, {'52.48.64.3', '45.55.10.200'}, "
                  "{selector='closest'})
For more about this feature, please head to the documentation.

Deprecations

4.2 will see the removal of the poorly documented ‘autoserial’ feature. This removal decision was not taken lightly but as noted, its removal allows us to fix other bugs. Autoserial was holding us back. We realise it is no fun when a feature disappears, but since Authoritative Server 4.1 is still around, you can still use that if you require ‘autoserial’.
Following RFC6986 and anticipating the publication of Algorithm Implementation Requirements and Usage Guidance for DNSSEC, support for both ECC-GOST signing and GOST DS digests have been removed.

ixfrdist

A new tool ixfrdist transfers zones from an authoritative server and re-serves these zones over AXFR and IXFR. It checks the SOA serial for all configured domains and downloads new versions to disk. This makes it possible for hundreds of PowerDNS Recursors (or authoritative servers) to slave an (RPZ) zone from a single server, without overwhelming providers like our friends over at Spamhaus/Deteque and Farsight.
Inspired by our Open-Xchange colleagues our API is now described by a Swagger spec!

Log-log histograms

Over at PowerDNS, we love statistics. Making sense of DNS performance is not that easy however – most queries get answered very quickly, but it is the outliers that determine how users “experience the internet”. It turns out that log-log histograms make it possible to fully capture the quality of a DNS service. As explained in this blog post, PowerDNS now comes with tooling to make such histograms:
log-full-avg

Note that this tooling is not specific to PowerDNS Authoritative or even PowerDNS: it will analyse any PCAP file with DNS in there.

Improvements, fixes

Much more

The changelog lists many more improvements and bug fixes.

Important Changes in PowerDNS Authoritative Server 4.2.0

Hello PowerDNS user,

as the year draws to an end, we are preparing for the release of a new version of the Authoritative Server (and the Recursor – stay tuned for more news in that area). This release (4.2.0) will see some changes that may affect your usage of PowerDNS. Please read, or at least skim, this article to make sure you will not be surprised when you upgrade.

If, after reading this, you have questions, please reach out to us via our mailing list or IRC channel. If you feel you’ve found a bug or other problem, we’d love a ticket on our GitHub repository.

Autoserial

Since forever, PowerDNS has supported a badly-documented feature called ‘autoserial’ which allowed a database backend to generate a serial on demand, when the serial inside the SOA record was set to zero. This feature was never documented properly and most deployments we have seen of it turned out to be broken in key areas. Supporting this feature also means we have bugs around the handling of the valid serial number zero in zones that do not want autoserial.

Over the last few releases, PowerDNS has gained robust HTTP REST support, and a mature implementation of RFC 2136 DNS Update. For the command line user, pdnsutil increase-serial is a simple way to increase serials on zones after updates.

Because autoserial was hard to use, rarely did what people wanted, and because its presence prevents us from fixing other bugs, we have decided to retire the feature now that several alternatives are available.

Builder/packaging changes

Between 4.1 and 4.2, we have replaced our old package building infrastructure with a new one, based on pdns-builder. While the idea is that most users will not notice this transition, there may be unintended changes.

Our Debian/Ubuntu packages no longer use ucf.

Our ./configure script inconsistently used --enable and --with. This has been fixed; downstream packagers may need to adjust their packaging.

Lua

Around the development of the LUA record type (more on that in a separate post), some of the Lua handling in the Authoritative Server has been refactored. If you have any existing Lua scripts in your Auth server, please make sure they still work correctly after upgrading to 4.2.

GOST

Following RFC6986 which deprecates the usage of GOST R 34.11-2012 generally, and anticipating the publication of Algorithm Implementation Requirements and Usage Guidance for DNSSEC which intends to move DNSSEC ECC-GOST support in signers to the ‘MUST NOT’ category, support for both ECC-GOST signing and GOST DS digests has been removed. In the unlikely situation that you have domains signed with ECC-GOST, you will need to roll their algorithms before upgrading to PowerDNS Authoritative Server 4.2.

 

PowerDNS Recursor 4.1.8 Released

We’ve released PowerDNS Recursor 4.1.8.

This release fixes Security Advisory 2018-09 that we recently discovered, affecting PowerDNS Recursor from 4.1.0 up to and including 4.1.7.  PowerDNS Recursor 4.0.x and below are not affected.

The issue is that a remote attacker can trigger an out-of-bounds memory read via a crafted query, while computing the hash of the query for a packet cache lookup, possibly leading to a crash.

When the PowerDNS Recursor is run inside a supervisor like supervisord or systemd, a crash will lead to an automatic restart, limiting the impact to a somewhat degraded service.

A minimal patch is available at https://downloads.powerdns.com/patches/2018-09/.

The changelog:

  • #7221: Crafted query can cause a denial of service (CVE-2018-16855)

The tarball (signature) is available at downloads.powerdns.com and packages for CentOS 6 and 7, Debian Jessie and Stretch, Ubuntu Bionic, Trusty and Xenial are available from repo.powerdns.com.

Please send us all feedback and issues you might have via the mailing list, or in case of a bug, via GitHub.