As we gear up for the release of PowerDNS Recursor 4.0.0, we are doing a series of posts describing new cool features which you can try out today. Many deployments are already running with 4.x alphas or snapshots, and now is a great time to familiarize yourself with the new and upcoming possibilities.
In this post, we’ll explain how to efficiently use the PowerDNS Recursor to optionally block certain domains for some or all of your users. This could be to stop users being tracked, to block advertisements or to protect against malware. The simple scripts below scale to millions of domain names and millions of users, all with acceptable startup times (seconds).
It starts with getting a good data set, and the good people from Mozilla have us covered here. The Mozilla Focus project through their partner Disconnect have a list of trackers and advertising servers which we can retrieve like this:
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| $ git clone https://github.com/mozilla/focus.git | |
| $ cd focus | |
| $ ./checkout.sh | |
| $ cd Lists |
(Note, the actual Focus project performs blocking more cleverly than can be achieved purely with DNS. Focus also takes into account the URL of the page containing the advertisement).
To make these lists usable by PowerDNS, we need a little bit of conversion using the most excellent ‘jq’ tool:
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| ( | |
| echo 'return{' | |
| for a in $(jq '.[].trigger["url-filter"]' disconnect-advertising.json | | |
| cut -f3 -d? | sed 's:\\\\.:.:g' | sed s:\"::) | |
| do | |
| echo \"$a\", | |
| done | |
| echo '}' | |
| ) > blocklist.lua |
This delivers a file which looks like ‘return{“a.com”, “b.com”, “c.com”}’, which is how to rapidly import data into Lua. To also block trackers, rerun this script for ‘disconnect-analytics.json’ and store as ‘trackers.lua’.
Next up, we use a small script ‘adblock.lua’ to tell PowerDNS to use this list to deny the existence of the filtered domains:
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| adservers=newDS() | |
| adservers:add(dofile("blocklist.lua")) | |
| function preresolve(dq) | |
| if(not adservers:check(dq.qname) or (dq.qtype ~= pdns.A and dq.qtype ~= pdns.AAAA)) then | |
| return false | |
| end | |
| dq:addRecord(pdns.SOA, | |
| "fake."..dq.qname:toString().." fake."..dq.qname:toString().." 1 7200 900 1209600 86400", | |
| 2) | |
| return true | |
| end |
Finally, put ‘lua-dns-script=adblock.lua’ in recursor.conf to tell PowerDNS to load this list. And for the basic functionality, we are done! A few words about the script. With ‘newDS()’ we defined a new Domain Set. In the second line, we load the list of advertisement related domain names.
Next we define the function preresolve() which gets called by PowerDNS to determine what to do with a domain. If we see that a query name is not part of one of the advertisement domains, or the query is not for an IP(v6) address, we return false and the normal resolution process continues.
Otherwise if it is a domain to be blocked, we insert a SOA record that says “this domain name exists, but the type you queried for doesn’t”.
Making it optional
Not everyone will want their advertisements filtered like this. And some people just want to be tracked online. To make this optional, first we make a list of IP addresses that want their DNS to be filtered:
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| (echo return{; | |
| for z in {1..10} | |
| do for a in {1..255} | |
| do for b in {1..255} | |
| do echo \"10.$z.$a.$b\", | |
| done ; done; done | |
| echo } ) > filtercustomers.lua |
This delivers a file with around 650000 IP addresses, which we can import in under a second in our Lua script like this:
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| adservers=newDS() | |
| adservers:add(dofile("blocklist.lua")) | |
| — optionally: adservers:add(dofile("trackers.lua")) | |
| filterset=newCAS() | |
| filterset:add(dofile("filtercustomers.lua")) | |
| function preresolve(dq) | |
| if(not adservers:check(dq.qname) or (dq.qtype ~= pdns.A and dq.qtype ~= pdns.AAAA)) then | |
| return false | |
| end | |
| dq.variable=true | |
| if(not filterset:check(dq.remoteaddr)) then | |
| return false | |
| end | |
| dq:addRecord(pdns.SOA, | |
| "fake."..dq.qname:toString().." fake."..dq.qname:toString().." 1 7200 900 1209600 86400", | |
| 2) | |
| return true | |
| end |
New line number 13 informs PowerDNS that this domain name will have variable responses depending on who asks. The following new lines check if this user wants to be filtered or not.
This script can easily be expanded to have different lists for users that want advertising or perhaps only trackers to be blocked.
Finally, to reload the script with new data, issue ‘rec_control reload-lua-script’.
Take away
Using the elementary scripts shown above, we can optionally provide very large userbases with optional advertisement or tracker filtering. To learn more about the PowerDNS Recursor dynamic abilities, head to the documentation, where you can also find how to retrieve user and domain status in real time from external servers.
