Quad9 uses PowerDNS to provide a worldwide encrypted DNS Service, a privacy-friendly public DNS resolving service for everyone. We talk to John Todd, General Manager - Quad9 Recursive Resolver, about why PowerDNS is its go-to solution.
John, you and your team have been very successfully running the Quad9 DNS service for about five years now. Please tell us about Quad9. Who are you and what services do you provide?
The Quad9 Foundation is a Switzerland-based not-for-profit organization. We provide our global community with a privacy-friendly DNS service. The service uses encrypted DNS and includes anti-malware and anti-phishing protections to guarantee the privacy of personal data across all transactions.
Quad9 users include consumers, educational organizations, Internet Service Providers (ISPs) local and regional governments, and small/medium enterprises. They often use Quad9’s resolving service instead of their internet provider’s DNS resolver because it delivers added security and increased performance.
To offer users the benefits of a high performing DNS resolver, combined with added security and privacy, Quad9 leverages several PowerDNS products, as well as other DNS components.
What does Quad9 use PowerDNS for?
Quad9 uses ‘anycast’ for DNS delivery, which lets it deliver equivalent configurations in more than 180 cities worldwide. A user’s ISP determines the geographically ‘closest’ Quad9 location to connect to. Once the DNS query reaches Quad9, it is distributed via a local routing protocol across several different instances, which allows Quad9 to share loads between instances.
Upon delivery of a DNS request to a node, PowerDNS DNSdist comes into play. DNSdist evaluates the traffic for denial of service or other anomalous behavior counters. If DNSdist classifies queries as abusive or malformed, it will be handled accordingly by, for example, rate-limiting the origin or dropping queries.
Why have you decided to use DNSdist in particular?
DNSdist’s extensive ability to track and respond to unusual packet types or volumes is extremely valuable to Quad9. In large networks, with a wide range of equipment, misconfigured devices or operating systems with legacy DNS code often cause outages due to inadvertent flooding or bad response behaviors. DNSdist allows more granular responses and handling of this type of fault, as well as extensive flexibility in handling and quenching intentionally abusive behaviors. If the volume of traffic that is ‘bogus’ is quite high, even if it is not malicious, DNSdist protects the backend systems from much of this ‘noise’ by caching negative answers or replying directly.
The DNSdist packetcache contains answers from prior lookups, and responses will be generated from that packetcache, resulting in extremely rapid response times, while not consuming any further resources on the recursive resolvers, as requests are never transmitted to the ‘backend’ resolver instances. This has helped us keep query latencies low and allows us to separate recursive processing load from client response load. This is especially important, as we continue to see more DNSSEC implementation and, in the future, encryption between the authoritative and recursive servers will further contribute to the need to separate the ‘frontend’ and ‘backend’ processes.
What happens after DNSdist handled incoming traffic?
DNSdist sends queries to one of Quad9’s three recursive resolver systems. There are multiple instances of each recursive resolver system in each set of nodes, adding a third layer of redundancy.
DNSdist spreads the query load to the recursive resolvers in a load sharing/load shedding model across various DNS resolvers, including Unbound, BIND and the PowerDNS Recursor, with PowerDNS seeing and serving a large portion of Quad9’s overall queries.
Why is it that the PowerDNS Recursor is dealing with a comparably large portion of queries?
Due to the PowerDNS Recursor’s ability to quickly determine if it can resolve a request, DNSdist tends to favor Quad9’s PowerDNS Recursor instances over others. The PowerDNS Recursor’s packetcache makes it able to answer more quickly than other resolver stacks for responses that happen to be in the resolver’s packetcache, but which have timed out from DNSdist.
If there is a situation where one resolver is unstable or causes a failure in certain response types, DNSdist will automatically push the traffic to the remaining resolver(s) that are able to better answer the query stream.
Why did you choose DNSdist to handle encrypted traffic?
To minimize complexity and increase the ability to scale our offering, Quad9 needs to be able to ingest and respond to queries from all major encryption models within one platform. DNSdist provides this and allows a more centralized management concept, where all queries are measured in one interface, instead of multiple ingress locations that may have differing models of abuse control or traffic management.
Using DNSdist, DNS over TLS (DoT), DNS over HTTPS (DoH) and DNSCrypt are all integrated into the same tool that also responds to standard UDP/TCP DNS queries, which greatly decreases the number of instrumentation and configuration points in the query stack.
Do you have any final remarks?
Quad9 owes a significant portion of our success to DNSdist, and we use PowerDNS as one of our primary recursive resolvers as well. We’ve been very pleased with the security profiles of DNSdist and PowerDNS Recursor, and in the rare instances new security or standards compliance issues are raised within the DNS community, the PowerDNS team has responded quickly and effectively. The ability to centralize much of our policy, traffic management, redundancy, and monitoring into DNSdist, while actually improving response times, is a major contributor in our ability to deliver our security services to a worldwide audience, while maintaining reliability and speed.
We would like to thank John again for his input and time. We appreciate the work John and his colleagues are putting in day to day to provide a public, open-source-based alternative to other DNS services. It makes us glad that PowerDNS forms an integral part in their setup, and we will continue to support Quad9’s efforts.
Please reach out to us or your OX account manager if you want to learn more about OX PowerDNS Recursor or DNSdist.
