* added a 'DNS Shift' option, where the generated final DNS blocklist is moved to the backup directory and
only a soft link to this file is set in memory. As long as your backup directory is located on an external drive,
you should activate this option to save disk space
* added ETAG-Header support to make sure to download only feeds that has been changed,
use backups otherwise (not supported by uclient-fetch)
* removed aria2 support
* added brave as a new safesearch provider
* removed the racist terminology from the local lists and renamed it to "allowlist" and "blocklist"
* removed the 'list' and 'timer' function from init, use the LuCI feed editor and the standard cron frontend instead
* various code changes and improvements
* major LuCI frontend changes, incl. a custom feed editor (separate commit)
* partial readme update
Signed-off-by: Dirk Brenken <dev@brenken.org>
* optimized procd settings for better performance
* reworked autodetection functions (still broken in master due to apk migration)
* made the tld function optional, set 'adb_tld' accordingly (enabled by default)
* reworked count function
* various code improvements
Signed-off-by: Dirk Brenken <dev@brenken.org>
* fixed adblock status reporting
* optimized the mail template
* removed unanswered DNS requests from reporting
* various small fixes
Signed-off-by: Dirk Brenken <dev@brenken.org>
* added full 1Hosts feed support (4 categories)
* changed the OISD list sources to alternate wildcard domains syntax
* used only the adguard source in default config
* fixed a needless reload delay plus a few cosmetics
Signed-off-by: Dirk Brenken <dev@brenken.org>
* new gawk dependency
* full hagezi support (all 32 categories)
* refine Stevenblack support
* refine whitelist handling
* fixed tcpdump command line for ports other than 53 (see #24685)
Signed-off-by: Dirk Brenken <dev@brenken.org>
* removed an accidentally commited flag of the upcoming adblock 5.x, this fixes a startup regression without trigger interface
Signed-off-by: Dirk Brenken <dev@brenken.org>
* fixed broken/blocked oisd download links (switched to the official github mirror)
* made sure that curl error out on http errors as well
* removed obsolete compatibility stuff from init script
Signed-off-by: Dirk Brenken <dev@brenken.org>
* made the reporting/top statistics flexible, see "top_count" parm in CLI or in LuCI (default 10), fixes#19622
* added the new blocklist source cpbl (provided by PascalCoffeeLake@gmail.com)
* added/separated Easylist/Easyprivacy blocklist sources (provided by PascalCoffeeLake@gmail.com)
* added reg_jp blocklist_source (provided by PascalCoffeeLake@gmail.com)
* removed the easylist addons from the other regional lists
* removed the second/obsolete pl regional list and renamed the first one to "reg_pl"
* updated the readme
Signed-off-by: Dirk Brenken <dev@brenken.org>
* some more cleanups, forgotten with the last update
* optimized unbound syntax ('always_nxdomain' & 'always_transparent')
* optimized oisd download sources (use wilcard variants which are much smaller)
* removed superfluous version information/function
Signed-off-by: Dirk Brenken <dev@brenken.org>
* dnsmasq upstream has changed the code for domain handling
and recommends the 'local' syntax for large blocklists
* remove pipefail command, see #19043 for reference
* removed the unused 'adb_dnsinotify' parameter
* removed the 'adb_maxqueue' parameter,
the queue size will be automatically set by the number of cpu cores
* various cleanups, mostly shellcheck related
Signed-off-by: Dirk Brenken <dev@brenken.org>
* support the RPZ trigger 'RPZ-CLIENT-IP' to always allow/block certain
clients based on their IP (currently only supported by bind!)
* avoid promiscuous mode in tcpdump setup for adblock reporting
* speed up dns report preparation
* support dns report mailing (/etc/init.d/adblock report mail)
* fix bind autodetection
* update LuCI-frontend (separate PR)
* update readme
Signed-off-by: Dirk Brenken <dev@brenken.org>
* major source changes:
* split oisd.nl in basic and full variant
* add swedish regional list
* made archive categories for shallalist and utcapitole selectable
via LuCI
* made all list variants of energized and stevenblack selectable
via LuCI
* removed dns filereset mode
Signed-off-by: Dirk Brenken <dev@brenken.org>
* since openwrt master has merged the depending P/R, the old
extra_help/extra_commands syntax is no longer working, see #13798 for
reference
* removed test.sh script from package
Signed-off-by: Dirk Brenken <dev@brenken.org>
* remove dumb list cache
* start adblock processing after adding/removing
list sources via CLI
* add regional list source for france
Signed-off-by: Dirk Brenken <dev@brenken.org>
* removed 'hphosts' from sources (discontinued)
* fixed a "out of range" bug and another small issue
in the f_dnsup function
* add three new sources: 'anudeep', 'stopforumspam' and 'youtube'
* changed 'list' behaviour, the source file has now
a higher precedence than the archive file (see readme)
* update readme, added missing parameters & more
Signed-off-by: Dirk Brenken <dev@brenken.org>
* new package dependencies: coreultis-sort and
a download util with SSL support
* focus on speed (multicore-support) to handle quite big lists
* include 38 pre-configured blocklist sources in a compressed
json file (/etc/adblock/adblock.sources.gz)
* dynamic SafeSearch support for google, bing, duckduckgo,
yandex, youtube and pixabay (CNAME (bind) & IP (dnsmaq, unbound))
* DNS backend autodetection
* Download Utility autodetection
* Report Interface autodetection
* Easy cron wrapper to set an adblock related auto-timer for
automatic blocklist updates
* raw domain/blocklist support (e.g. for dnscrypt support)
* re-add restrictive Jaillist support
* rework online doc
* Complete LuCI rewrite (migrated to client side JS)
Signed-off-by: Dirk Brenken <dev@brenken.org>
* more startup tweaks
* re-use f_log function in helper scripts
* small fixes / polish up for forthcoming 19.07 release
Signed-off-by: Dirk Brenken <dev@brenken.org>
* use raw procd interface trigger as last resort, if the
adblock config is not available during startup
* fix selective subdomain whitelisting for dnsmasq
* fix a kresd restart issue with 'DNS File Reset'
* fix a suspend/resume cornercase
* disable the tld compression, if the number of blocked domains
is greater than 'adb_maxtld' (default: 100000)
* made the fw portlist configurable (default '53 853 5353')
* preliminary support for inotify-like autoload features
of dns backends like kresd in future Turris OS. If 'adb_dnsinotify'
is set to 'true', all adblock related restarts and the
'DNS File Reset' will be disabled
Signed-off-by: Dirk Brenken <dev@brenken.org>
* add support for 'DNS File Reset', where the final DNS blockfile
will be purged after DNS backend loading (save storage space).
A small background service will be started to trace/handle
dns backend reloads/restarts
* add support for the 'null' blocking variant in dnsmasq
(via addn-hosts), which may provide better response times
in dnsmasq
* enhance the report & search engine to support
the new blocking variants. Search now includes
backups & black-/whitelist as well
* compressed source list backups are now mandatory (default to '/tmp')
* speed up TLD compression
* E-Mail notification setup is now integrated in UCI/LuCI
* update the LuCI frontend to reflect all changes (separate PR)
* drop preliminary dnscrypt-proxy-support (use dnsmasq instead)
* drop additional 'dnsjail' blocklist support (not used by anyone)
* procd cleanups in init
* various shellcheck cleanups
* update readme
Signed-off-by: Dirk Brenken <dev@brenken.org>
* the DNS Report now displays the hostname, MAC-Address or
client IP (CLI & LuCI)
* Filter the DNS Query result set for a particular domain, client or
time frame (CLI & LuCI)
* remove needless XHR.Poll-Events from Reporting page in LuCI
* remove needless 'force sort' option in LuCI
Signed-off-by: Dirk Brenken <dev@brenken.org>
* add adblock dns query reporting via tcpdump (see readme for details)
* fix tld compression on low memory systems (< 64 MB)
* fix various small issues
Signed-off-by: Dirk Brenken <dev@brenken.org>
backend:
* add low priority mode (nice level 10), disabled by default
* enhance 'Force DNS' to redirect ports 53, 853 and 5353
frontend:
* switch to dynamic XHR polling for runtime information and logfile
viewing
* add new 'Refresh' button to reload blocklists
* various cleanups & small fixes
Signed-off-by: Dirk Brenken <dev@brenken.org>
* enable code to support Turris Omnia forthcoming upstream change
(new kresd 'keep_cache' option) to preserve kresd DNS cache
* fix a 'status' race condition while the adblock process
is running in parallel
* various small speed improvements
* rework debug output
* refine blacklist handling
* enable the (empty) blacklist source in the default config
* email notification supports mstmp, even without sendmail symlink
* email notification writes minimal status to log (one-liner)
* LuCI: refine logfile search term
* LuCI: Textarea 'autoscroll down' in logfile view
* LuCI: Left-align blocklist source table plus a more compact design
Signed-off-by: Dirk Brenken <dev@brenken.org>
* preserve DNS cache after adblock processing,
- 'unbound' and 'named' support this (please check readme)
- 'dnsmasq' now uses the 'servers-file' directive
to minimize the reload disruption,
even though the dns cache will be cleared after SIGHUP
- 'kresd' dns cache is persistent by upstream default, anyway
Turris Omnia devices need a small upstream software change
which is not accepted/implemented yet
* email notification in case of an error or domain count < n
(default 0, check readme)
* removed securemecca from default config (service has been closed)
* new separate functions for hash compare and list/overall count
* add missing package dependencies
* various clean-ups
* update documentation
Signed-off-by: Dirk Brenken <dev@brenken.org>
* add kresd & turris omnia support
* add dnscrypt-proxy support
* change start priority to 30, to fix possible trigger issues on slow
booting hardware
* simplify suspend/resume handling (no longer use a hideout directory)
* enhanced LuCI frontend
* many small changes & improvements
* default config change (please update your config!), adblock is now
disabled by default
* documentation update
Signed-off-by: Dirk Brenken <dev@brenken.org>
backend:
* various small fixes & optimizations
LuCI frontend (see luci repo):
* Limit Blacklist/Whitelist Online editing to max. 512 KB, approx.
20.000 domains per list
* Automatically refresh the overview page after button onclick event,
e.g. 'Suspend/Resume' or 'Save & Apply'
* cosmetics
Signed-off-by: Dirk Brenken <dev@brenken.org>
* change start priority to get all interface trigger events, even on
fast hardware
* made default trigger delay more conservative to fix possible start up
issues
Signed-off-by: Dirk Brenken <dev@brenken.org>
* no longer misuse ubus/procd service object for adblock runtime
information, now save all required information directly
in a JSON file/format (/tmp/adb_runtime.json)
* new 'status' init command to print runtime information
* add a configurable interface trigger timeout for
nested or slow modem/router setups,
set 'adb_triggerdelay' accordingly (default 1 second)
* add support for pure http download utilities like wget-nossl
or uclient-fetch without libustream-ssl (http donwloads only!)
* fix stop action
* fix enabled/disabled action
* fix country code in regional list for china
* LuCI update to reflect all changes
Signed-off-by: Dirk Brenken <dev@brenken.org>
* re-order all regional lists in adblocklist/easylist format
* add polish blocklist
* add chinese blocklist
* lists called now 'reg_<ch|pl|ro|ru>'
* changed interface trigger default in config to 'wan' only
* update / sync LuCI frontend (see luci repo for details)
* cosmetics
Signed-off-by: Dirk Brenken <dev@brenken.org>
* revamp LuCI GUI (see luci repo for details)
* integrated runtime statistics in normal adblock ubus service instance
(see readme)
* simplified boot/init
* no longer use raw interface trigger
* use only network interface trigger, pre-configured for
'wan/wwan/lan' interfaces (see readme)
* fixed a reload issue
* removed switch to disable tld compression (seems to be pretty stable)
* muted awk parser if source list contains incorrect records
* raise dns backend timeout to 20 seconds
* documentation update
* cosmetics
Signed-off-by: Dirk Brenken <dev@brenken.org>
* add tld compression,
this new "top level domain compression" removes up to 40 thousand
needless host entries from the block lists and
lowers the memory footprint for the dns backends by 8-10 MByte
* optimize restart behavior in case of an error
* cosmetics
Signed-off-by: Dirk Brenken <dev@brenken.org>
* various optimizations & corner case fixes
* removed no longer needed debug information
* polished up for forthcoming LEDE release ;-)
Signed-off-by: Dirk Brenken <dev@brenken.org>
Dirk Brenken