Related to recent DoS attacks: Recursor configuration file guidance

Hi everybody,

Over the past week we’ve been contacted by a few users reporting their PowerDNS Recursor became unresponsive under a moderate denial of service attack, one which PowerDNS should be expected to weather without issues.

In the course of investigating this issue, we’ve found that many PowerDNS installations on Linux are configured to consume (far) more filedescriptors than are actually available, wasting resources, potentially leading to unresponsiveness.

To check if this is the case for you, multiply the ‘max-mthreads’ setting by the ‘threads’ setting. Default values are 2048 and 2, leading to a theoretical FD consumption of 4096. Many Linux distributions default to 1024. So, our defaults exceed the Linux defaults by a large margin!

(FreeBSD defaults are far higher, and should not pose an issue).

To fix, there are four options:

  1. Reduce max-mthreads to 512 (or threads to 1 and max-mthreads to 1024) (max-mthreads was introduced in Recursor 3.2; but if you are running a version that old, please upgrade it!)
  2. Run ‘ulimit -n 32768’ before starting (perhaps put this in /etc/init.d/ script). There’s little reason to skip on this number.
  3. Investigate defaults in /etc/security/limits.conf
  4. Apply the patch in https://github.com/Habbie/pdns/commit/e24b124a4c7b49f38ff8bcf6926cd69077d16ad8 (this patch is in our 3.6.0 release. We recommend just upgrading!)

The patch automates 1 and 2, either raising the limit if possible, or  reducing max-mthreads until “it fits”.

Thank you for your attention, and if you have results to report to us on previous or current DoS attacks, please contact us privately.

6 comments

  1. ZaphodB

    Please note that defaults in /etc/security/limits.conf won’t actually work at boot time for Debian squeeze/wheezy because the relevant pam libraries are not loaded at that time.
    Hence it is advisable to also specify the limits in /etc/initscript ( man 5 initscript ).

    • Diman

      It works, fine, but you need to write rule for root too:
      root soft nofile 100000
      root soft nofile 100000
      * soft nofile 100000
      * hard nofile 100000

      I have problem on linux with flood too. PDNS trops requests, and was forces to go to unbound.

  2. Pingback: Further DoS guidance, packages and patches available | PowerDNS Blog
  3. Pingback: Recursor 3.6.0 Release Candidate 1 | PowerDNS Blog
  4. Pingback: Recursor 3.6.0 released | PowerDNS Blog
  5. Pingback: DoS flaw in PowerDNS default configuration | Web Security Watch

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s