Hi everybody,
Over the past week we’ve been contacted by a few users reporting their PowerDNS Recursor became unresponsive under a moderate denial of service attack, one which PowerDNS should be expected to weather without issues.
In the course of investigating this issue, we’ve found that many PowerDNS installations on Linux are configured to consume (far) more filedescriptors than are actually available, wasting resources, potentially leading to unresponsiveness.
To check if this is the case for you, multiply the ‘max-mthreads’ setting by the ‘threads’ setting. Default values are 2048 and 2, leading to a theoretical FD consumption of 4096. Many Linux distributions default to 1024. So, our defaults exceed the Linux defaults by a large margin!
(FreeBSD defaults are far higher, and should not pose an issue).
To fix, there are four options:
- Reduce max-mthreads to 512 (or threads to 1 and max-mthreads to 1024) (max-mthreads was introduced in Recursor 3.2; but if you are running a version that old, please upgrade it!)
- Run ‘ulimit -n 32768’ before starting (perhaps put this in /etc/init.d/ script). There’s little reason to skip on this number.
- Investigate defaults in /etc/security/limits.conf
- Apply the patch in https://github.com/Habbie/pdns/commit/e24b124a4c7b49f38ff8bcf6926cd69077d16ad8 (this patch is in our 3.6.0 release. We recommend just upgrading!)
The patch automates 1 and 2, either raising the limit if possible, or reducing max-mthreads until “it fits”.
Thank you for your attention, and if you have results to report to us on previous or current DoS attacks, please contact us privately.