Skip to content

README.md

Latest commit

 

History

History
806 lines (540 loc) · 25.2 KB

README.md

File metadata and controls

806 lines (540 loc) · 25.2 KB

Dnr-rulesets

Utility to load prebuilt AdGuard DNR rulesets for mv3 extensions.

The list of available filters can be found by filters in the metadata.

Basic usage

Install package.

NOTE: To update filters in time, make sure you have the latest version of the package installed.

npm install --save-dev @adguard/dnr-rulesets

CLI

  1. Add scripts to your package.json to load DNR rulesets and patch extension manifest.
{
    "scripts": {
        "load-dnr-rulesets": "dnr-rulesets load <path-to-output>",
        "patch-manifest": "dnr-rulesets manifest <path-to-manifest> <path-to-filters>"
    }
}

Available commands:

load command

Downloads and saves DNR rulesets to the specified directory.

dnr-rulesets load <path-to-output>

Options for load command:

  • -l, --latest-filters - download latest text filters instead of DNR rulesets (default: false)

manifest command

Patches the extension manifest to include DNR rulesets.

dnr-rulesets manifest <path-to-manifest> <path-to-filters> [options]

Options for manifest command:

  • -f, --force-update - force update rulesets with existing id (default: false)
  • -i, --ids <ids...> - filters ids to append, others will be ignored (default: [] - append all)
  • -e, --enable <ids...> - enable filters by default (default: [])
  • -r, --ruleset-prefix <prefix> - prefix for filters ids (default: "ruleset_")
  • -m, --filters-match <match> - filters files match glob pattern (default: "filter_+([0-9]).txt")

Note about array options: For options that accept multiple values (ids and enable), use please following syntax:

--ids=1,2

watch command

Watches for changes in the filter files and rebuilds DNR rulesets.

dnr-rulesets watch <path-to-manifest> <path-to-resources> [options]

Arguments:

  • <path-to-manifest> - path to the manifest.json file
  • <path-to-resources> - folder with resources to build $redirect rules (can be obtained via @adguard/tswebextension war command)

Options for watch command:

  • -p, --path-to-filters - path to filters and i18n metadata file (default: ./filters relative to manifest folder)
  • -o, --output-path-for-rulesets - output path for rulesets (default: ./filters/declarative relative to manifest folder)
  • -f, --force-update - force update rulesets with existing id (default: true)
  • -i, --ids <ids...> - filters ids to process, others will be ignored (default: [] - process all filters matched via --filters-match)
  • -e, --enable <ids...> - enable filters by default in manifest.json (default: [])
  • -r, --ruleset-prefix <prefix> - prefix for filters ids (default: "ruleset_")
  • -m, --filters-match <match> - filters files match glob pattern (default: "filter_+([0-9]).txt")
  • -l, --latest-filters - download latest text filters on first start before watch (default: false)
  • -d, --debug - enable extended logging during conversion (default: false)
  • -j, --prettify-json - prettify JSON output (default: true)

exclude-unsafe-rules command

Scans rulesets in the specified directory, excludes unsafe rules, and saves excluded unsafe rules to the metadata files, and update rulesets checksums.

dnr-rulesets exclude-unsafe-rules <dir> [options]

Arguments:

  • <dir>: Path to the folder containing rulesets to process.

Options:

  • -j, --prettify-json <bool>: Prettify JSON output (true or false, default: true)
  • -l, --limit <number>: Limit the number of unsafe rules to exclude. If the number of unsafe rules exceeds this limit, the command will throw an error.

Example:

dnr-rulesets exclude-unsafe-rules ./filters/declarative --prettify-json false --limit 100

Note about array options: For options that accept multiple values (ids and enable), use please following syntax:

--ids=1,2
  1. Run the script to load DNR rulesets as part of your build flow.
npm run load-dnr-rulesets
  1. Patch your extension manifest to include DNR rulesets.
npm run patch-manifest

API

You can also integrate functions for downloading and updating the manifest into your build script:

  1. Load DNR rulesets.

    This method copies prebuilt assets to the specified output directory, including:

    • DNR rulesets in JSON format for all available filters
    • filters_i18n.json - translations file with filter names and descriptions
    • local_script_rules.js - local script rules file in JS module format for Manifest v3 extensions where it is highly recommended to provide local script rules. If not provided during build, all script rules (except scriptlets) will not be injected to ensure compliance with Chrome Web Store policies.
    • local_script_rules.json - local script rules in JSON format for Manifest v2. If not provided, all script rules are treated as allowed. Should be provided in Firefox AMO according to their policies.
    import { AssetsLoader } from '@adguard/dnr-rulesets';

const loader = new AssetsLoader();
await loader.load('<path-to-output>');

  2. Copy only local script rules.

    Option A: Copy JavaScript format rules

    import { AssetsLoader } from '@adguard/dnr-rulesets';

const loader = new AssetsLoader();
await loader.copyLocalScriptRulesJs('<path-to-output>');

    This method copies only the local_script_rules.js file to the specified destination directory.

    Option B: Copy JSON format rules

    import { AssetsLoader } from '@adguard/dnr-rulesets';

const loader = new AssetsLoader();
await loader.copyLocalScriptRulesJson('<path-to-output>');

    This method copies only the local_script_rules.json file to the specified destination directory.

  3. Extend local script rules with custom rules.

    Option A: Using extendLocalScriptRulesJs (Manifest V3)

    import { AssetsLoader } from '@adguard/dnr-rulesets';

const loader = new AssetsLoader();
await loader.extendLocalScriptRulesJs(
    '<path-to-local-script-rules.js>',
    [
        'example.com#%#const ad = document.querySelector(".ad"); ad.remove();',
        'example.org#%#console.log("Custom script");'
    ]
);

    This method parses custom filtering rules, extracts JavaScript injection rules from them, and appends them to an existing local_script_rules.js file. It's useful for dynamically adding custom JS rules to your extension at build time.

    Option B: Using extendLocalScriptRulesJson (Manifest V2)

    import { AssetsLoader } from '@adguard/dnr-rulesets';

const loader = new AssetsLoader();
await loader.extendLocalScriptRulesJson(
    '<path-to-local-script-rules.json>',
    [
        'example.com#%#const ad = document.querySelector(".ad"); ad.remove();',
        'example.org,~sub.example.org#%#console.log("Custom script");'
    ]
);

    This method parses custom filtering rules, extracts JavaScript injection rules with their domain configurations (both permitted and restricted domains), and merges them into an existing local_script_rules.json file. Use this method when you need to maintain domain-specific rule associations.

    Note: The extendLocalScriptRulesJs and extendLocalScriptRulesJson methods are only available in the programmatic API and not in the CLI. These methods require programmatic access to parse and manipulate existing local script rule files, which is more suitable for build scripts and custom automation workflows rather than command-line usage.

  4. Patch extension manifest.

    import { ManifestPatcher } from '@adguard/dnr-rulesets';

const patcher = new ManifestPatcher();

patcher.path(
    '<path-to-manifest>',
    '<path-to-output>',
    {
        // Optional: specify filter IDs to include
        ids: ['2', '3'],
        // Optional: specify enabled filter IDs
        enabled: ['2'],
        // Optional: set to true to overwrite existing rulesets
        forceUpdate: true,
        // Optional: set prefix for ruleset paths
        rulesetPrefix: 'ruleset_',
        // Optional: specify filter files matching glob pattern
        filtersMatch: 'filter_+([0-9]).txt',
    },
)

Also you can call to excludeUnsafeRules in your custom build flows or automation scripts.

import { excludeUnsafeRules } from '@adguard/dnr-rulesets';

await excludeUnsafeRules('<path-to-rulesets-dir>', {
    prettifyJson: true, // optional
    limit: 100,         // optional
});

Output structure

/
|
|declarative
|   |
|   |ruleset_<id>
|       |
|       |ruleset_<id>.json // DNR ruleset converted from filter_<id>.txt
|       |metadata.json // Ruleset metadata with source mapping
|       |lazy_Metadata.json // Additional ruleset metadata for lazy loading
|
|filter_<id>.txt // Original filter rules with specified id

Utils

The package provides a set of utility functions for working with DNR rulesets.

getVersion()

Returns the version of the package.

import { getVersion } from '@adguard/dnr-rulesets/utils';

const dnrRulesetsVersion = getVersion();

getVersionTimestampMs()

Returns the timestamp of the dnr-rulesets build, based on the patch version, or current timestamp of the function call if date and time is not present in the patch version.

import { getVersionTimestampMs } from '@adguard/dnr-rulesets/utils';

const dnrRulesetsBuildTimestamp = getVersionTimestampMs();

Advanced usage

Injecting rulesets to the manifest object

We also provide flexible API to apply rulesets to the manifest object. It can be useful if you want to patch to the manifest while bundling.

import { RulesetsInjector } from '@adguard/dnr-rulesets';

const injector = new RulesetsInjector();

const manifest = {
    // Your manifest data
};

const ManifestWithRulesets = injector.applyRulesets(
    (id) => `<path to rulesets>/${id}.json`,
    manifest,
    ['2', '3'],
    {
        // Optional: specify filter IDs to include
        ids: ['2', '3'],
        // Optional: specify enabled filter IDs
        enabled: ['2'],
        // Optional: set to true to overwrite existing rulesets
        forceUpdate: true,
        // Optional: set prefix for ruleset paths
        rulesetPrefix: 'ruleset_',
    },
);

Example

Example of usage: adguard-api-mv3

Included filter lists

Ad blocking

AdGuard Base filter

EasyList + AdGuard English filter. This filter is necessary for quality ad blocking.

  • Filter ID: 2
  • Path: <filters-directory>/declarative/ruleset_2/ruleset_2.json

AdGuard Mobile Ads filter

Filter for all known mobile ad networks. Useful for mobile devices.

  • Filter ID: 11
  • Path: <filters-directory>/declarative/ruleset_11/ruleset_11.json

Privacy

AdGuard Tracking Protection filter

The most comprehensive list of various online counters and web analytics tools. Use this filter if you do not want your actions on the Internet to be tracked.

  • Filter ID: 3
  • Path: <filters-directory>/declarative/ruleset_3/ruleset_3.json

AdGuard URL Tracking filter

Filter that enhances privacy by removing tracking parameters from URLs.

  • Filter ID: 17
  • Path: <filters-directory>/declarative/ruleset_17/ruleset_17.json

Social widgets

AdGuard Social Media filter

Filter for social media widgets such as 'Like' and 'Share' buttons and more.

  • Filter ID: 4
  • Path: <filters-directory>/declarative/ruleset_4/ruleset_4.json

Annoyances

AdGuard Cookie Notices filter

Blocks cookie notices on web pages.

  • Filter ID: 18
  • Path: <filters-directory>/declarative/ruleset_18/ruleset_18.json

AdGuard Popups filter

Blocks all kinds of pop-ups that are not necessary for websites' operation according to our Filter policy.

  • Filter ID: 19
  • Path: <filters-directory>/declarative/ruleset_19/ruleset_19.json

AdGuard Mobile App Banners filter

Blocks irritating banners that promote mobile apps of websites.

  • Filter ID: 20
  • Path: <filters-directory>/declarative/ruleset_20/ruleset_20.json

AdGuard Other Annoyances filter

Blocks irritating elements on web pages that do not fall under the popular categories of annoyances.

  • Filter ID: 21
  • Path: <filters-directory>/declarative/ruleset_21/ruleset_21.json

AdGuard Widgets filter

Blocks annoying third-party widgets: online assistants, live support chats, etc.

  • Filter ID: 22
  • Path: <filters-directory>/declarative/ruleset_22/ruleset_22.json

Security

Online Malicious URL Blocklist

Blocks domains that are known to be used to propagate malware and spyware.

  • Filter ID: 208
  • Path: <filters-directory>/declarative/ruleset_208/ruleset_208.json

Phishing URL Blocklist

Phishing URL blocklist for uBlock Origin (uBO), AdGuard, Vivaldi, Pi-hole, Hosts file, Dnsmasq, BIND, Unbound, Snort and Suricata.

  • Filter ID: 255
  • Path: <filters-directory>/declarative/ruleset_255/ruleset_255.json

Scam Blocklist by DurableNapkin

List for blocking untrustworthy websites.

  • Filter ID: 256
  • Path: <filters-directory>/declarative/ruleset_256/ruleset_256.json

uBlock Origin – Badware risks

Filter for risky sites, warning users of potential threats.

  • Filter ID: 257
  • Path: <filters-directory>/declarative/ruleset_257/ruleset_257.json

Dandelion Sprout's Anti-Malware List

Blocks more malware than most other major anti-malware lists - domains and URL patterns used in malware redirection chains, IP addresses that are solely used by malware, PUP nags, and a few scammers. Already included in Dandelion Sprout's Annoyances List.

  • Filter ID: 259
  • Path: <filters-directory>/declarative/ruleset_259/ruleset_259.json

Other

AdGuard Experimental filter

Filter designed to test certain hazardous filtering rules before they are added to the basic filters.

  • Filter ID: 5
  • Path: <filters-directory>/declarative/ruleset_5/ruleset_5.json

Filter unblocking search ads and self-promotion

Filter that unblocks search ads in Google, DuckDuckGo, Bing, or Yahoo and self-promotion on websites.

  • Filter ID: 10
  • Path: <filters-directory>/declarative/ruleset_10/ruleset_10.json

Language-specific

AdGuard Russian filter

Filter that enables ad blocking on websites in Russian language.

  • Filter ID: 1
  • Path: <filters-directory>/declarative/ruleset_1/ruleset_1.json

AdGuard German filter

EasyList Germany + AdGuard German filter. Filter list that specifically removes ads on websites in German language.

  • Filter ID: 6
  • Path: <filters-directory>/declarative/ruleset_6/ruleset_6.json

AdGuard Japanese filter

Filter that enables ad blocking on websites in Japanese language.

  • Filter ID: 7
  • Path: <filters-directory>/declarative/ruleset_7/ruleset_7.json

AdGuard Dutch filter

EasyList Dutch + AdGuard Dutch filter. Filter list that specifically removes ads on websites in Dutch language.

  • Filter ID: 8
  • Path: <filters-directory>/declarative/ruleset_8/ruleset_8.json

AdGuard Spanish/Portuguese filter

Filter list that specifically removes ads on websites in Spanish, Portuguese, and Brazilian Portuguese languages.

  • Filter ID: 9
  • Path: <filters-directory>/declarative/ruleset_9/ruleset_9.json

AdGuard Turkish filter

Filter list that specifically removes ads on websites in Turkish language.

  • Filter ID: 13
  • Path: <filters-directory>/declarative/ruleset_13/ruleset_13.json

AdGuard French filter

Liste FR + AdGuard French filter. Filter list that specifically removes ads on websites in French language.

  • Filter ID: 16
  • Path: <filters-directory>/declarative/ruleset_16/ruleset_16.json

AdGuard Ukrainian filter

Filter that enables ad blocking on websites in Ukrainian language.

  • Filter ID: 23
  • Path: <filters-directory>/declarative/ruleset_23/ruleset_23.json

Bulgarian list

Additional filter list for websites in Bulgarian.

  • Filter ID: 103
  • Path: <filters-directory>/declarative/ruleset_103/ruleset_103.json

EasyList Czech and Slovak

Additional filter list for websites in Czech and Slovak.

  • Filter ID: 105
  • Path: <filters-directory>/declarative/ruleset_105/ruleset_105.json

EasyList Hebrew

Additional filter list for websites in Hebrew.

  • Filter ID: 108
  • Path: <filters-directory>/declarative/ruleset_108/ruleset_108.json

EasyList Italy

Additional filter list for websites in Italian.

  • Filter ID: 109
  • Path: <filters-directory>/declarative/ruleset_109/ruleset_109.json

EasyList Lithuania

Additional filter list for websites in Lithuanian.

  • Filter ID: 110
  • Path: <filters-directory>/declarative/ruleset_110/ruleset_110.json

Latvian List

Additional filter list for websites in Latvian.

  • Filter ID: 111
  • Path: <filters-directory>/declarative/ruleset_111/ruleset_111.json

Liste AR

Additional filter list for websites in Arabic.

  • Filter ID: 112
  • Path: <filters-directory>/declarative/ruleset_112/ruleset_112.json

AdBlockID

Additional filter list for websites in Indonesian.

  • Filter ID: 120
  • Path: <filters-directory>/declarative/ruleset_120/ruleset_120.json

EasyList Thailand

Filter that blocks ads on Thai sites.

  • Filter ID: 202
  • Path: <filters-directory>/declarative/ruleset_202/ruleset_202.json

Hungarian filter

Hufilter. Filter list that specifically removes ads on websites in the Hungarian language.

  • Filter ID: 203
  • Path: <filters-directory>/declarative/ruleset_203/ruleset_203.json

ABPVN List

Vietnamese adblock filter list.

  • Filter ID: 214
  • Path: <filters-directory>/declarative/ruleset_214/ruleset_214.json

Official Polish filters for AdBlock, uBlock Origin & AdGuard

Additional filter list for websites in Polish.

  • Filter ID: 216
  • Path: <filters-directory>/declarative/ruleset_216/ruleset_216.json

Polish GDPR-Cookies Filters

Polish filter list for cookies blocking.

  • Filter ID: 217
  • Path: <filters-directory>/declarative/ruleset_217/ruleset_217.json

Estonian List

Filter for ad blocking on Estonian sites.

  • Filter ID: 218
  • Path: <filters-directory>/declarative/ruleset_218/ruleset_218.json

AdGuard Chinese filter

EasyList China + AdGuard Chinese filter. Filter list that specifically removes ads on websites in Chinese language.

  • Filter ID: 224
  • Path: <filters-directory>/declarative/ruleset_224/ruleset_224.json

List-KR

Filter that removes ads and various scripts from websites with Korean content. Combined and augmented with AdGuard-specific rules for enhanced filtering. This filter is expected to be used alongside with AdGuard Base filter.

  • Filter ID: 227
  • Path: <filters-directory>/declarative/ruleset_227/ruleset_227.json

Adblock List for Finland

Finnish ad blocking filter list.

  • Filter ID: 233
  • Path: <filters-directory>/declarative/ruleset_233/ruleset_233.json

Persian Blocker

Filter list for blocking ads and trackers on websites in Persian.

  • Filter ID: 235
  • Path: <filters-directory>/declarative/ruleset_235/ruleset_235.json

Polish Anti Adblock Filters

Official Polish filters against Adblock alerts.

  • Filter ID: 238
  • Path: <filters-directory>/declarative/ruleset_238/ruleset_238.json

Frellwit's Swedish Filter

Filter that aims to remove regional Swedish ads, tracking, social media, annoyances, sponsored articles etc.

  • Filter ID: 243
  • Path: <filters-directory>/declarative/ruleset_243/ruleset_243.json

Dandelion Sprout's Nordic Filters

This list covers websites for Norway, Denmark, Iceland, Danish territories, and the Sami indigenous population.

  • Filter ID: 249
  • Path: <filters-directory>/declarative/ruleset_249/ruleset_249.json

Dandelion Sprout's Serbo-Croatian List

A filter list for websites in Serbian, Montenegrin, Croatian, and Bosnian.

  • Filter ID: 252
  • Path: <filters-directory>/declarative/ruleset_252/ruleset_252.json

IndianList

Additional filter list for websites in Hindi, Tamil and other Dravidian and Indic languages.

  • Filter ID: 253
  • Path: <filters-directory>/declarative/ruleset_253/ruleset_253.json

Macedonian adBlock Filters

Blocks ads and trackers on various Macedonian websites.

  • Filter ID: 254
  • Path: <filters-directory>/declarative/ruleset_254/ruleset_254.json

Development

build:assets

Downloads original rules, converts it to DNR rule sets via TSUrlFilter declarative-converter and generates extension manifest with predefined rules resources.

pnpm build:assets

build:lib

Builds SDK to load DNR rule sets to the specified directory.

pnpm build:lib

build:cli

Builds CLI utility to load DNR rule sets to the specified directory, inject rulesets to the manifest object and can be used for local development for DNR rulesets.

pnpm build:cli

build:docs

Generates Included filter lists section.

pnpm build:docs

build

Clears dist folder and runs build:assets, build:cli and build:lib scripts.

pnpm build

watch

Watches for changes in the dist/filters folder and rebuilds DNR rulesets.

pnpm watch