Converting a Hostfile Adblock into Unbound DNS records

There are lots of places where you can download ad blacklists in the form of simple host files, but they require manual installation on every device. Instead we will make records for our DNS server so we can block ads network wide. I used a blacklist from somewhocares. This github page contains a range of different filters and their stats. If you wish to learn more about setting up Unbound, I have an example configuration fileHere is my script to do it. It also allows you to specifiy a different address than "0.0.0.0". This is useful if you have a web server you wish to point blocked domains (e.g. pihole)

#! /bin/bash

BlocklistURL=${1:-"https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts"}
EndpointIP=${2:-"0.0.0.0"}
GeneratedOutput=${3:-"ads.conf"}

wget -qO- ${BlocklistURL} | grep '^0\.0\.0\.0' | awk '{print "local-zone: \""$2"\" redirect\nlocal-data: \""$2" A '${EndpointIP}'\""}' > ${GeneratedOutput};

If no arguments are provided, it will use default values.

Now we point /etc/unbound.conf to the file

    include: /etc/unbound/ads.conf

We run unbound-checkconf to verify the data.

Otherwise, any errors will cause unbound to fail to start

 $ sudo unbound-checkconf && dig +short statsfe2.ws.microsoft.com
 unbound-checkconf: no errors in /etc/unbound/unbound.conf
 192.168.255.254

All done. I'll be doing a few more tutorials on Unbound.

Appendix: Running a blank page webserver

While in my experience, DNS blocking on its own works pretty well, you can maximize its effectiveness by running a webserver on your local address. Without a web server running, sometimes scripts will stall and other little glitches. We will create a lighttpd server running on the address 192.168.255.254
It will redirect any .js file to a script that throws a new error, any images to a blank image, and anything else to a blank response.

/etc/lighttpd/lighttpd.conf
(the bottom section is the saliant one)

var.log_root    = "/var/log/lighttpd"
var.server_root = "/var/www"
var.state_dir   = "/var/run"
var.home_dir    = "/var/lib/lighttpd"
var.conf_dir    = "/etc/lighttpd"
var.cache_dir   = "/var/cache/lighttpd"
var.socket_dir  = state_dir + "/sockets"

include "modules.conf"

server.port = 80
server.use-ipv6 = "disable"
server.bind = "192.168.255.254"

server.username  = "lighttpd"
server.groupname = "lighttpd"
server.document-root = server_root + "/lighttpd"
server.pid-file = state_dir + "/lighttpd.pid"
server.errorlog             = log_root + "/error.log"
include "conf.d/access_log.conf"

server.event-handler = "linux-sysepoll"
server.network-backend = "sendfile"
server.stat-cache-engine = "simple"
server.max-connections = 1024

$SERVER["socket"] == "192.168.255.254:80" {
        url.rewrite = ( ".*\.js$" => "/adblock.js", ".*\.(jpeg|jpg|png|gif)$" =>"/adblock.png", ".*"=>"/adblock.txt")
}

Now we configure the adblock files to serve,

$ cd /var/www/lighttpd
$ touch adblock.txt
$ wget -qOadblock.png http://placehold.it/1x1
$ echo "throw new Error();" >adblock.js

Now let's test out images

ad_image="images.outbrain.com/transform/v2/eyJpdSI6IjhlMzdiY2FjMDk3MTQ3Nzc0Njk1ZjI4MTVmZWI5NmNhZmM1MWVjNTRiM2YyNzRmNDljYTk3YzU3ZTA5Yzc1ZTEiLCJ3IjoxNDUsImgiOjExMCwiZCI6MS41LCJjcyI6MCwiZiI6MH0.jpg"

$ wget -qO- $ad_image | diff - /var/www/lighttpd/adblock.png

Next, we test javascript

ad_javascript="widgets.outbrain.com/outbrain.js"

$ wget -qO- $ad_javascript
throw new Error();