adblock: update 4.4.0-3

* checked and fixed the kresd and smartdns support
* fixed another ETAG issue
* changed the enabled feeds in default config to certpl, aguard and adguard_tracking
* various other small fixes
* update the readme

Signed-off-by: Dirk Brenken <dev@brenken.org>
This commit is contained in:
Dirk Brenken 2025-04-16 21:36:47 +02:00
parent 7b2daf0f0a
commit 3e99991067
No known key found for this signature in database
GPG Key ID: 9D71CD547BFAE684
5 changed files with 61 additions and 52 deletions

View File

@ -7,7 +7,7 @@ include $(TOPDIR)/rules.mk
PKG_NAME:=adblock
PKG_VERSION:=4.4.0
PKG_RELEASE:=2
PKG_RELEASE:=3
PKG_LICENSE:=GPL-3.0-or-later
PKG_MAINTAINER:=Dirk Brenken <dev@brenken.org>

View File

@ -14,14 +14,14 @@ A lot of people already use adblocker plugins within their desktop browsers, but
| :------------------ | :-----: | :--- | :--------------- | :-------------------------------------------------------------------------------- |
| 1Hosts | | VAR | compilation | [Link](https://github.com/badmojr/1Hosts) |
| adaway | | S | mobile | [Link](https://github.com/AdAway/adaway.github.io) |
| adguard | x | L | general | [Link](https://adguard.com) |
| adguard_tracking | | L | tracking | [Link](https://github.com/AdguardTeam/cname-trackers) |
| adguard | x | L | general | [Link](https://adguard.com) |
| adguard_tracking | x | L | tracking | [Link](https://github.com/AdguardTeam/cname-trackers) |
| android_tracking | | S | tracking | [Link](https://github.com/Perflyst/PiHoleBlocklist) |
| andryou | | L | compilation | [Link](https://gitlab.com/andryou/block/-/blob/master/readme.md) |
| anti_ad | | L | compilation | [Link](https://github.com/privacy-protection-tools/anti-AD/blob/master/README.md) |
| anudeep | | M | compilation | [Link](https://github.com/anudeepND/blacklist) |
| bitcoin | | S | mining | [Link](https://github.com/hoshsadiq/adblock-nocoin-list) |
| certpl | | L | phishing | [Link](https://cert.pl/en/warning-list/) |
| certpl | x | L | phishing | [Link](https://cert.pl/en/warning-list/) |
| cpbl | | XL | compilation | [Link](https://github.com/bongochong/CombinedPrivacyBlockLists) |
| disconnect | | S | general | [Link](https://disconnect.me) |
| doh_blocklist | | S | doh_server | [Link](https://github.com/dibdot/DoH-IP-blocklists) |
@ -95,7 +95,7 @@ A lot of people already use adblocker plugins within their desktop browsers, but
* Additional local blocklist for manual overrides, located in '/etc/adblock/adblock.blocklist'
* Quality checks during blocklist update to ensure a reliable DNS backend service
* Minimal status & error logging to syslog, enable debug logging to receive more output
* Procd based init system support ('start', 'stop', 'restart', 'reload', 'enable', 'disable', 'running', 'status', 'suspend', 'resume', 'query', 'report')
* Procd based init system support ('start', 'stop', 'restart', 'reload', 'enable', 'disable', 'running', 'status', 'suspend', 'resume', 'query', 'report')
* Auto-Startup via procd network interface trigger or via classic time based startup
* Suspend & Resume adblock temporarily without blocklist re-processing
* Provides comprehensive runtime information
@ -110,26 +110,27 @@ A lot of people already use adblocker plugins within their desktop browsers, but
<a id="prerequisites"></a>
## Prerequisites
* [OpenWrt](https://openwrt.org), tested with the stable release series and with the latest snapshot releases.
<b>Please note:</b> Devices with less than 128 MByte RAM are _not_ supported!
<b>Please note:</b> For performance reasons, adblock depends on gnu awk (gawk) by default.
If you insist to use the slow busybox awk implementation, remove the gawk package afterwards (_opkg remove gawk --force-depends_) or install adblock without any dependency checks/installation (_opkg install adblock --nodeps_). Both installation variants are officially unsupported.
* A usual setup with an enabled DNS backend at minimum - dumb AP modes without a working DNS backend are _not_ supported
* **[OpenWrt](https://openwrt.org)**, latest stable release 24.x or a development snapshot
* A usual setup with a working DNS backend
* A download utility with SSL support: 'wget', 'uclient-fetch' with one of the 'libustream-*' ssl libraries or 'curl' is required
* A certificate store such as 'ca-bundle' or 'ca-certificates', as adblock checks the validity of the SSL certificates of all download sites by default
* Optional E-Mail notification support: for E-Mail notifications you need to install the additional 'msmtp' package
* Optional DNS Query Report support: for DNS reporting you need to install the additional package 'tcpdump-mini' or 'tcpdump'
* For E-Mail notifications you need to install and setup the additional 'msmtp' package
* For DNS reporting you need to install the additional package 'tcpdump-mini' or 'tcpdump'
**Please note:**
* Devices with less than 128MB of RAM are **_not_** supported
* For performance reasons, adblock depends on gnu sort and gawk
<a id="installation-and-usage"></a>
## Installation & Usage
* Update your local opkg repository (_opkg update_)
* Install 'adblock' (_opkg install adblock_). The adblock service is enabled by default
* Install the LuCI companion package 'luci-app-adblock' (_opkg install luci-app-adblock_)
* Update your local opkg/apk repository
* Install the LuCI companion package 'luci-app-adblock' which also installs the main 'adblock' package as a dependency
* It's strongly recommended to use the LuCI frontend to easily configure all aspects of adblock, the application is located in LuCI under the 'Services' menu
* It's also recommended to configure at least a 'Startup Trigger Interface' to depend on WAN ifup events during boot or restart of your router
<a id="adblock-cli-interface"></a>
## Adblock CLI interface
* All important adblock functions are accessible via CLI as well.
* The most important adblock functions are accessible via CLI as well.
```
~# /etc/init.d/adblock
@ -162,14 +163,14 @@ Available commands:
| adb_enabled | 1, enabled | set to 0 to disable the adblock service |
| adb_feedfile | /etc/adblock/adblock.feeds | full path to the used adblock feed file |
| adb_dns | -, auto-detected | 'dnsmasq', 'unbound', 'named', 'kresd', 'smartdns' or 'raw' |
| adb_fetchutil | -, auto-detected | 'uclient-fetch', 'wget' or 'curl' |
| adb_fetchcmd | -, auto-detected | 'uclient-fetch', 'wget' or 'curl' |
| adb_fetchparm | -, auto-detected | manually override the config options for the selected download utility |
| adb_fetchinsecure | 0, disabled | don't check SSL server certificates during download |
| adb_trigger | -, not set | trigger network interface or 'not set' to use a time-based startup |
| adb_triggerdelay | 2 | additional trigger delay in seconds before adblock processing begins |
| adb_debug | 0, disabled | set to 1 to enable the debug output |
| adb_nice | 0, standard prio. | valid nice level range 0-19 of the adblock processes |
| adb_forcedns | 0, disabled | set to 1 to force DNS requests to the local resolver |
| adb_dnsforce | 0, disabled | set to 1 to force DNS requests to the local resolver |
| adb_dnsdir | -, auto-detected | path for the generated blocklist file 'adb_list.overall' |
| adb_dnstimeout | 10 | timeout in seconds to wait for a successful DNS backend restart |
| adb_dnsinstance | 0, first instance | set to the relevant dns backend instance used by adblock (dnsmasq only) |
@ -198,6 +199,7 @@ Available commands:
<a id="examples"></a>
## Examples
**Change the DNS backend to 'unbound':**
No further configuration is needed, adblock deposits the final blocklist 'adb_list.overall' in '/var/lib/unbound' by default.
To preserve the DNS cache after adblock processing please install the additional package 'unbound-control'.
@ -221,8 +223,7 @@ and at the end of the file add:
```
**Change the DNS backend to 'kresd':**
Adblock deposits the final blocklist 'adb_list.overall' in '/etc/kresd', no further configuration needed.
<b>Please note:</b> The knot-resolver (kresd) is only available on Turris devices and does not support the SafeSearch functionality yet.
Adblock deposits the final blocklist 'adb_list.overall' in '/tmp/kresd', no further configuration needed.
**Change the DNS backend to 'smartdns':**
No further configuration is needed, adblock deposits the final blocklist 'adb_list.overall' in '/tmp/smartdns' by default.
@ -259,6 +260,14 @@ password xxx
</code></pre>
Finally enable E-Mail support and add a valid E-Mail receiver address in LuCI.
**Send status E-Mails and update the adblock lists via cron job**
For a regular, automatic status mailing and update of the used lists on a daily basis set up a cron job, e.g.
```
55 03 * * * /etc/init.d/adblock report mail
00 04 * * * /etc/init.d/adblock reload
```
**Service status output:**
In LuCI you'll see the realtime status in the 'Runtime' section on the overview page.
To get the status in the CLI, just call _/etc/init.d/adblock status_ or _/etc/init.d/adblock status\_service_:

View File

@ -2,8 +2,11 @@
config adblock 'global'
option adb_enabled '1'
option adb_debug '0'
option adb_forcedns '0'
option adb_dnsforce '0'
option adb_dnsshift '0'
option adb_safesearch '0'
option adb_mail '0'
option adb_report '0'
list adb_feed 'adguard'
list adb_feed 'adguard_tracking'
list adb_feed 'certpl'

View File

@ -8,10 +8,6 @@
# set (s)hellcheck exceptions
# shellcheck disable=all
LC_ALL=C
PATH="/usr/sbin:/usr/bin:/sbin:/bin"
[ -r "/lib/functions.sh" ] && . "/lib/functions.sh"
[ -r "/usr/bin/adblock.sh" ] && . "/usr/bin/adblock.sh" "mail"
adb_debug="$(uci_get adblock global adb_debug "0")"
@ -27,7 +23,7 @@ adb_mailhead="From: ${adb_mailsender}\nTo: ${adb_mailreceiver}\nSubject: ${adb_m
# info preparation
#
sys_info="$("${adb_stringscmd}" /etc/banner 2>/dev/null; "${adb_ubuscmd}" call system board | "${adb_awkcmd}" 'BEGIN{FS="[{}\"]"}{if($2=="kernel"||$2=="hostname"||$2=="system"||$2=="model"||$2=="description")printf " + %-12s: %s\n",$2,$4}' 2>/dev/null)"
sys_info="$("${adb_catcmd}" /etc/banner 2>/dev/null; "${adb_ubuscmd}" call system board | "${adb_awkcmd}" 'BEGIN{FS="[{}\"]"}{if($2=="kernel"||$2=="hostname"||$2=="system"||$2=="model"||$2=="description")printf " + %-12s: %s\n",$2,$4}' 2>/dev/null)"
adb_info="$(/etc/init.d/adblock status 2>/dev/null)"
rep_info="${1}"
if [ -x "${adb_logreadcmd}" ]; then

View File

@ -202,7 +202,7 @@ f_char() {
# load dns backend config
#
f_dns() {
local util utils dns_section dns_info mem_free
local util utils dns_section dns_info mem_free dir
mem_free="$("${adb_awkcmd}" '/^MemAvailable/{printf "%s",int($2/1000)}' "/proc/meminfo" 2>/dev/null)"
if [ "${adb_action}" = "start" ] && [ -z "${adb_trigger}" ]; then
@ -292,11 +292,11 @@ f_dns() {
adb_dnscachecmd="-"
adb_dnsinstance="${adb_dnsinstance:-"0"}"
adb_dnsuser="${adb_dnsuser:-"root"}"
adb_dnsdir="${adb_dnsdir:-"/etc/kresd"}"
adb_dnsdir="${adb_dnsdir:-"/tmp/kresd"}"
adb_dnsheader="${adb_dnsheader:-"\$TTL 2h\n@ IN SOA localhost. root.localhost. (1 6h 1h 1w 2h)\n"}"
adb_dnsdeny="${adb_dnsdeny:-"${adb_awkcmd} '{print \"\"\$0\" CNAME .\\n*.\"\$0\" CNAME .\"}'"}"
adb_dnsallow="${adb_dnsallow:-"${adb_awkcmd} '{print \"\"\$0\" CNAME rpz-passthru.\\n*.\"\$0\" CNAME rpz-passthru.\"}'"}"
adb_dnssafesearch="${adb_dnssafesearch:-"${adb_awkcmd} -v item=\"\$item\" '{type=\"AAAA\";if(match(item,/^([0-9]{1,3}\.){3}[0-9]{1,3}$/)){type=\"A\"}}{print \"\"\$0\" \"type\" \"item\"\"}'"}"
adb_dnssafesearch="${adb_dnssafesearch:-"${adb_awkcmd} -v item=\"\$item\" '{print \"\"\$0\" CNAME \"item\".\\n*.\"\$0\" CNAME \"item\".\"}'"}"
adb_dnsstop="${adb_dnsstop:-"* CNAME ."}"
;;
"smartdns")
@ -329,9 +329,9 @@ f_dns() {
adb_finaldir="${adb_backupdir}"
fi
if [ "${adb_action}" != "stop" ]; then
[ ! -d "${adb_backupdir}" ] && mkdir -p "${adb_backupdir}"
[ ! -d "${adb_finaldir}" ] && mkdir -p "${adb_finaldir:-"/tmp"}"
[ "${adb_jail}" = "1" ] && [ ! -d "${adb_jaildir}" ] && mkdir -p "${adb_jaildir:-"/tmp"}"
for dir in "${adb_dnsdir:-"/tmp"}" "${adb_backupdir:-"/tmp"}" "${adb_jaildir:-"/tmp"}"; do
[ ! -d "${dir}" ] && mkdir -p "${dir}"
done
if [ "${adb_dnsflush}" = "1" ] || [ "${mem_free}" -lt "64" ]; then
printf "%b" "${adb_dnsheader}" >"${adb_finaldir}/${adb_dnsfile}"
f_dnsup
@ -490,21 +490,22 @@ f_extconf() {
"kresd")
config="resolver"
if [ "${adb_enabled}" = "1" ] &&
! uci_get ${config} kresd rpz_file | "${adb_grepcmd}" -q "${adb_finaldir}/${adb_dnsfile}"; then
uci -q add_list ${config}.kresd.rpz_file="${adb_finaldir}/${adb_dnsfile}"
! uci_get ${config} kresd rpz_file | "${adb_grepcmd}" -q "${adb_dnsdir}/${adb_dnsfile}"; then
uci -q add_list ${config}.kresd.rpz_file="${adb_dnsdir}/${adb_dnsfile}"
elif [ "${adb_enabled}" = "0" ] &&
uci_get ${config} kresd rpz_file | "${adb_grepcmd}" -q "${adb_finaldir}/${adb_dnsfile}"; then
uci -q del_list ${config}.kresd.rpz_file="${adb_finaldir}/${adb_dnsfile}"
uci_get ${config} kresd rpz_file | "${adb_grepcmd}" -q "${adb_dnsdir}/${adb_dnsfile}"; then
uci -q del_list ${config}.kresd.rpz_file="${adb_dnsdir}/${adb_dnsfile}"
fi
;;
"smartdns")
config="smartdns"
if [ "${adb_enabled}" = "1" ] &&
! uci_get ${config} @${config}[${adb_dnsinstance}] conf_files | "${adb_grepcmd}" -q "${adb_finaldir}/${adb_dnsfile}"; then
uci -q add_list ${config}.@${config}[${adb_dnsinstance}].conf_files="${adb_finaldir}/${adb_dnsfile}"
! uci_get ${config} @${config}[${adb_dnsinstance}] conf_files | "${adb_grepcmd}" -q "${adb_dnsdir}/${adb_dnsfile}"; then
uci -q add_list ${config}.@${config}[${adb_dnsinstance}].conf_files="${adb_dnsdir}/${adb_dnsfile}"
elif [ "${adb_enabled}" = "0" ] &&
uci_get ${config} @${config}[${adb_dnsinstance}] conf_files | "${adb_grepcmd}" -q "${adb_finaldir}/${adb_dnsfile}"; then
uci -q del_list ${config}.@${config}[${adb_dnsinstance}].conf_files="${adb_finaldir}/${adb_dnsfile}"
uci_get ${config} @${config}[${adb_dnsinstance}] conf_files | "${adb_grepcmd}" -q "${adb_dnsdir}/${adb_dnsfile}"; then
uci -q del_list ${config}.@${config}[${adb_dnsinstance}].conf_files="${adb_dnsdir}/${adb_dnsfile}"
fi
;;
esac
@ -517,7 +518,7 @@ f_extconf() {
for port in ${adb_portlist}; do
if ! printf "%s" "${fwcfg}" | "${adb_grepcmd}" -q "adblock_${zone}${port}"; then
config="firewall"
if "${adb_lookupcmd}" "localhost" "127.0.0.1:${port}" >/dev/null 2>&1; then
if "${adb_lookupcmd}" "localhost." "127.0.0.1:${port}" >/dev/null 2>&1; then
uci -q batch <<-EOC
set firewall."adblock_${zone}${port}"="redirect"
set firewall."adblock_${zone}${port}".name="Adblock DNS (${zone}, ${port})"
@ -601,7 +602,7 @@ f_dnsup() {
break
fi
cnt="$((cnt + 1))"
sleep 1
sleep 2
done
if [ "${out_rc}" = "0" ] && [ "${adb_dns}" = "unbound" ]; then
if [ -x "${adb_dnscachecmd}" ] && [ -d "${adb_tmpdir}" ] && [ -s "${adb_tmpdir}/adb_cache.dump" ]; then
@ -628,17 +629,17 @@ f_etag() {
if [ -z "${etag_id}" ]; then
etag_id="$(printf "%s" "${http_head}" | "${adb_awkcmd}" 'tolower($0)~/^[[:space:]]*last-modified: /{gsub(/[Ll]ast-[Mm]odified:|[[:space:]]|,|:/,"");printf "%s\n",$1}')"
fi
etag_cnt="$("${adb_grepcmd}" -c "^${feed}" "${adb_backupdir}/adblock.etag")"
etag_cnt="$("${adb_grepcmd}" -c "^${feed} " "${adb_backupdir}/adblock.etag")"
if [ "${http_code}" = "200" ] && [ "${etag_cnt}" = "${feed_cnt}" ] && [ -n "${etag_id}" ] &&
"${adb_grepcmd}" -q "^${feed}${feed_suffix}[[:space:]]\+${etag_id}\$" "${adb_backupdir}/adblock.etag"; then
"${adb_grepcmd}" -q "^${feed} ${feed_suffix}[[:space:]]\+${etag_id}\$" "${adb_backupdir}/adblock.etag"; then
out_rc="0"
elif [ -n "${etag_id}" ]; then
if [ "${feed_cnt}" -lt "${etag_cnt}" ]; then
"${adb_sedcmd}" -i "/^${feed}/d" "${adb_backupdir}/adblock.etag"
"${adb_sedcmd}" -i "/^${feed} /d" "${adb_backupdir}/adblock.etag"
else
"${adb_sedcmd}" -i "/^${feed}${feed_suffix//\//\\/}/d" "${adb_backupdir}/adblock.etag"
"${adb_sedcmd}" -i "/^${feed} ${feed_suffix//\//\\/}/d" "${adb_backupdir}/adblock.etag"
fi
printf "%-80s%s\n" "${feed}${feed_suffix}" "${etag_id}" >>"${adb_backupdir}/adblock.etag"
printf "%-80s%s\n" "${feed} ${feed_suffix}" "${etag_id}" >>"${adb_backupdir}/adblock.etag"
out_rc="2"
fi
@ -719,7 +720,7 @@ f_list() {
;;
"safesearch")
file_name="${adb_tmpdir}/tmp.safesearch.${src_name}"
if [ "${adb_dns}" = "named" ] || [ "${adb_dns}" = "smartdns" ]; then
if [ "${adb_dns}" = "named" ] || [ "${adb_dns}" = "kresd" ] || [ "${adb_dns}" = "smartdns" ]; then
use_cname="1"
fi
case "${src_name}" in
@ -735,7 +736,7 @@ f_list() {
"${adb_gzipcmd}" -cf "${adb_tmpdir}/tmp.load.safesearch.${src_name}" >"${adb_backupdir}/safesearch.${src_name}.gz"
fi
fi
safe_domains="$("${adb_awkcmd}" "${rset}" "${adb_tmpdir}/tmp.load.safesearch.${src_name}")"
[ -s "${adb_tmpdir}/tmp.load.safesearch.${src_name}" ] && safe_domains="$("${adb_awkcmd}" "${rset}" "${adb_tmpdir}/tmp.load.safesearch.${src_name}")"
;;
"bing")
safe_cname="strict.bing.com"
@ -779,8 +780,8 @@ f_list() {
break
fi
done
out_rc="${?}"
: >"${adb_tmpdir}/tmp.raw.safesearch.${src_name}"
out_rc="0"
fi
;;
"prepare")
@ -847,7 +848,7 @@ f_list() {
if [ "${adb_safesearch}" = "1" ] && [ "${adb_dnssafesearch}" != "0" ]; then
ffiles="${ffiles} -a ! -name safesearch.google.gz"
fi
find "${adb_backupdir}" ${ffiles} -print0 2>/dev/null | xargs -0 rm 2>/dev/null
"${adb_findcmd}" "${adb_backupdir}" ${ffiles} -print0 2>/dev/null | xargs -0 rm 2>/dev/null
"${adb_sortcmd}" ${adb_srtopts} -mu "${adb_tmpfile}".* 2>/dev/null >"${file_name}"
out_rc="${?}"
rm -f "${adb_tmpfile}".*
@ -1480,13 +1481,13 @@ adb_grepcmd="$(f_cmd grep)"
adb_gzipcmd="$(f_cmd gzip)"
adb_pgrepcmd="$(f_cmd pgrep)"
adb_sedcmd="$(f_cmd sed)"
adb_findcmd="$(f_cmd find)"
adb_jsoncmd="$(f_cmd jsonfilter)"
adb_ubuscmd="$(f_cmd ubus)"
adb_loggercmd="$(f_cmd logger)"
adb_lookupcmd="$(f_cmd nslookup)"
adb_dumpcmd="$(f_cmd tcpdump optional)"
adb_mailcmd="$(f_cmd msmtp optional)"
adb_stringscmd="$(f_cmd strings optional)"
adb_logreadcmd="$(f_cmd logread optional)"
# handle different adblock actions