Skip to content

Commit 1e1691b

Browse files
authored
New challenge & improve stability
New challenge & improve stability
2 parents 0e4c42d + cda234e commit 1e1691b

14 files changed

Lines changed: 205 additions & 41 deletions

File tree

.github/workflows/1.bump-version.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,7 @@ jobs:
1717
contents: write
1818
steps:
1919
- name: Checkout
20-
uses: actions/checkout@v5
20+
uses: actions/checkout@v6
2121
- name: Bump version
2222
env:
2323
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

.github/workflows/2.build-publish.yml

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -17,9 +17,9 @@ jobs:
1717
# contents: read
1818
# steps:
1919
# - name: Checkout
20-
# uses: actions/checkout@v5
20+
# uses: actions/checkout@v6
2121
# - name: Set up Python
22-
# uses: actions/setup-python@v5
22+
# uses: actions/setup-python@v6
2323
# with:
2424
# python-version: "3.10"
2525
# - name: Install dependencies
@@ -37,13 +37,13 @@ jobs:
3737
contents: write
3838
steps:
3939
- name: Checkout
40-
uses: actions/checkout@v5
40+
uses: actions/checkout@v6
4141
- name: Get latest version
4242
run: |
4343
git pull origin main
4444
echo "Current version: v$(./scripts/get-version.sh)"
4545
- name: Set up Python
46-
uses: actions/setup-python@v5
46+
uses: actions/setup-python@v6
4747
with:
4848
python-version: "3.10"
4949
- name: Install dependencies

.github/workflows/3.update-changelog.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@ jobs:
1414
contents: write
1515
steps:
1616
- name: Checkout
17-
uses: actions/checkout@v5
17+
uses: actions/checkout@v6
1818
- name: Update changelog
1919
env:
2020
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

.github/workflows/publish-docs.yml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -17,11 +17,11 @@ jobs:
1717
contents: write
1818
steps:
1919
- name: Checkout
20-
uses: actions/checkout@v5
20+
uses: actions/checkout@v6
2121
with:
2222
fetch-depth: 0
2323
- name: Set up Python
24-
uses: actions/setup-python@v5
24+
uses: actions/setup-python@v6
2525
with:
2626
python-version: "3.10"
2727
- name: Install dependencies
Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
11
nav:
22
- README.md
3-
- Anti-Detect Automation Detection v1: v1.md
3+
- Anti-Detect Automation Detection v2: v2.md
44
- "*"

docs/challenges/ada_detection/README.md

Lines changed: 18 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22

33
## Overview
44

5-
The **Anti-Detect Automation Detection (AAD)** challenge evaluates a participant`s ability to reliably detect browser automation frameworks operating inside anti-detect browsers, while preserving human safety.
5+
The **Anti-Detect Automation Detection (AAD)** challenge evaluates a participant's ability to reliably detect browser automation frameworks operating inside anti-detect browsers, while preserving human safety.
66

77
Evaluation runs simulate real-world anti-detect usage where static signals are masked and fingerprints are fresh. Detection must rely on runtime behavior and orchestration patterns.
88

@@ -17,23 +17,23 @@ Each evaluation run involves:
1717
!!! Info "NST-Browser Dependency"
1818
Participants need an API key from the [NSTBrowser](https://www.nstbrowser.io/en/pricing) dashboard (Professional plan recommended) for local testing.
1919

20-
## Features of ADA Detection v1
20+
## Features of ADA Detection v2
2121

22-
- **Behavior-first detection**: Focus on runtime behavioral analysis inside NST-Browser; static driver fingerprints are spoofed.
23-
- **Payload-based flow**: Detection scripts run in-page, send payloads to local `/_payload` when automation is confirmed, and must stay silent during human sessions.
24-
- **Human safety gate**: Random human interactions are injected; flagging more than 2 humans as bots zeros the submission.
25-
- **Target coverage update**: Required detectors include `automation`, `nodriver`, `playwright`, `patchright`, and `puppeteer`.
26-
- **Fresh profiles, no state**: Each run uses a fresh profile with zero shared state between runs.
22+
- **Hardened NSTBrowser Environment**: Advanced obfuscation including dynamic fingerprinting and hardware simulation (8GB RAM, 16 Cores).
23+
- **Engine-Level Stealth**: `AutomationControlled` flags are disabled at the browser engine level to eliminate static detection signals.
24+
- **Fail-Fast Scoring**: High-stakes evaluation where missing critical targets (Selenium) or exceeding miss limits results in an immediate 0.0 score.
25+
- **Protocol-Level Verification**: Mandatory detection of low-level communication patterns (Webdriver and WebSocket).
26+
- **Selenium Safety Gate**: Mandatory detection of `seleniumbase` and `selenium_driverless` is required for incentive eligibility.
2727
- **Similarity/time decay**: Similarity penalties apply to lookalike submissions; scores decay over 15 days to encourage refreshed heuristics.
2828

2929
## Evaluation Flow
3030

3131
1. **Submission Received**: Detection scripts are submitted via the `/score` endpoint.
3232
2. **Task Generation**: A randomized sequence of multiple automation framework runs and human interactions is generated.
3333
3. **NST-Browser Launch**: A clean instance is started for each task.
34-
4. **Execution**: Automation frameworks connect via WebSocket, while humans interact manually.
34+
4. **Execution**: Automation frameworks connect via WebSocket or WebDriver, while humans interact manually.
3535
5. **Detection Phase**: Scripts may emit detection payloads to `/_payload`.
36-
6. **Scoring**: Results are aggregated and normalized.
36+
6. **Scoring**: Results are aggregated and normalized using the Fail-Fast pillars.
3737

3838
## Technical Constraints
3939

@@ -59,11 +59,16 @@ Submit your detection scripts as a JSON payload with the following structure:
5959
```json
6060
{
6161
"detection_files": [
62-
{ "file_name": "automation.js", "content": "..." },
6362
{ "file_name": "nodriver.js", "content": "..." },
6463
{ "file_name": "playwright.js", "content": "..." },
6564
{ "file_name": "patchright.js", "content": "..." },
66-
{ "file_name": "puppeteer.js", "content": "..." }
65+
{ "file_name": "puppeteer.js", "content": "..." },
66+
{ "file_name": "puppeteer_extra.js", "content": "..." },
67+
{ "file_name": "zendriver.js", "content": "..." },
68+
{ "file_name": "selenium_driverless.js", "content": "..." },
69+
{ "file_name": "seleniumbase.js", "content": "..." },
70+
{ "file_name": "webdriver.js", "content": "..." },
71+
{ "file_name": "websocket.js", "content": "..." }
6772
]
6873
}
6974
```
@@ -72,7 +77,8 @@ Each file must be named exactly as shown and contain self-contained JavaScript (
7277

7378
## Challenge Versions
7479

75-
- [**v1** (Active after Dec 15, 2025 10:00 UTC)](./v1.md) - Payload-based detection with human safety gate
80+
- [**v2** (Active after Feb 14, 2026 14:00 UTC)](./v2.md) - Hardened detection with Fail-Fast scoring
81+
- [**v1** (Deprecated)](./depricated/v1.md)
7682

7783
## Resources & Guides
7884

File renamed without changes.
Lines changed: 145 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,145 @@
1+
---
2+
title: Anti-Detect Automation Detection v2
3+
---
4+
5+
# Anti-Detect Automation Detection v2 (Active after 2026 February 14 14:00 UTC)
6+
7+
## Overview
8+
9+
**ADA Detection v2** focuses on advanced behavioral detection inside **NSTBrowser** environments. This version introduces hardened detection requirements where static signals are masked and engine-level obfuscation is active.
10+
11+
### Browser Hardening Features
12+
13+
- **Dynamic Fingerprinting**: Every session generates unique, randomized browser identifiers to prevent static pattern matching.
14+
- **Hardware Simulation**: Realistic signals are injected for `deviceMemory` (8GB), `hardwareConcurrency` (16), and screen resolution (1280x1024).
15+
- **Engine-Level Stealth**: Disables `AutomationControlled` flags at the browser engine level, eliminating traditional automation leaks.
16+
17+
Scoring has transitioned to a **Fail-Fast** model, emphasizing accuracy across three critical pillars: Human Detection, Framework Detection (with a Selenium Safety Gate), and Protocol Accuracy.
18+
19+
For general challenge information, environment details, and plagiarism policies, please refer to the [AAD README](./README.md).
20+
21+
---
22+
23+
## Target Frameworks & Protocols
24+
25+
Participants must submit detection scripts for the following frameworks and protocols:
26+
27+
- **Frameworks**: `nodriver`, `playwright`, `patchright`, `puppeteer`, `puppeteer_extra`, `zendriver`, `selenium_driverless`, `seleniumbase`
28+
- **Protocols**: `webdriver`, `websocket`
29+
30+
Missing any of these scripts results in an invalid submission.
31+
32+
### Selenium Safety Gate
33+
34+
Missing detection for either `seleniumbase` or `selenium_driverless` results in an immediate **final score of 0.0**. These frameworks are the primary automation vectors in v2; failure to detect them indicates a fundamental gap in the detection logic.
35+
36+
---
37+
38+
## Submission Format
39+
40+
Submissions must follow this structure:
41+
42+
```json
43+
{
44+
"detection_files": [
45+
{ "file_name": "nodriver.js", "content": "/* logic */" },
46+
{ "file_name": "playwright.js", "content": "/* logic */" },
47+
{ "file_name": "patchright.js", "content": "/* logic */" },
48+
{ "file_name": "puppeteer.js", "content": "/* logic */" },
49+
{ "file_name": "puppeteer_extra.js", "content": "/* logic */" },
50+
{ "file_name": "zendriver.js", "content": "/* logic */" },
51+
{ "file_name": "selenium_driverless.js", "content": "/* logic */" },
52+
{ "file_name": "seleniumbase.js", "content": "/* logic */" },
53+
{ "file_name": "webdriver.js", "content": "/* logic */" },
54+
{ "file_name": "websocket.js", "content": "/* logic */" }
55+
]
56+
}
57+
```
58+
59+
### Rules
60+
61+
- File names must match the framework/protocol names exactly.
62+
- Each file detects **only its own framework or protocol**.
63+
- No extra files or outputs are allowed.
64+
65+
---
66+
67+
## Scoring System: The Three Pillars
68+
69+
ADA v2 uses a **Fail-Fast** scoring model. If a submission fails any of the three critical pillars, the final score is immediately set to **0.0**.
70+
71+
### 1. Human Detection
72+
73+
Miners must distinguish between automated tasks and human-injected sessions.
74+
75+
- **Limit**: You are allowed a maximum of **1 mistake**. Exceeding this limit results in an immediate **score of 0.0**.
76+
- **Weight**: Perfect detection grants 1.0 point for this pillar. Partial penalties apply for a single miss based on the ratio of human injections.
77+
78+
### 2. Framework Detection
79+
80+
Points are earned for correctly identifying the specific automation framework.
81+
82+
- **Selenium Gate**: Missing `seleniumbase` or `selenium_driverless` zeros the entire score.
83+
- **Density**: You earn **1.0 point** for a framework only if you detect it perfectly in **all 3 of its runs**.
84+
- **Collision**: Reporting more than one framework or an incorrect protocol for a given session results in a collision penalty (earning only 0.1 points instead of 1.0).
85+
86+
### 3. Protocol Accuracy
87+
88+
Validates the low-level communication patterns of the browser.
89+
90+
- **Webdriver Protocol**: Expected to be `true` for Selenium-based frameworks and `false` for others (human, playwright, etc.).
91+
- **Websocket Protocol**: Expected to be `true` for non-Selenium frameworks (playwright, puppeteer, etc.) and `false` for human sessions and Selenium.
92+
- **Threshold**: You are allowed a maximum of **1 miss** per protocol type. Exceeding this limit results in an immediate **score of 0.0**.
93+
94+
### Final Formula
95+
96+
The final score is normalized between 0.0 and 1.0 using the formula:
97+
98+
```math
99+
Final Score = (Human Score + Framework Score + Protocol Score) / 10
100+
```
101+
102+
__(Where 10 = 8 frameworks + 1 human pillar + 1 protocol pillar)__
103+
104+
---
105+
106+
## Incentive Eligibility
107+
108+
To maintain high detection standards, a minimum performance threshold is required for incentives:
109+
110+
- **Minimum Score**: **0.6**
111+
- This threshold ensures that miners identify the main automation vectors (Selenium) and maintain high human safety while allowing for minor misses in other categories.
112+
113+
---
114+
115+
## Similarity & Time Decay
116+
117+
- **Similarity check:** Submissions are compared against other SDKs; high similarity incurs penalties.
118+
- **Score decay:** Scores decay over 15 days to incentivize refreshed heuristics.
119+
120+
### Example
121+
122+
Assume:
123+
124+
- 8 frameworks total
125+
- Perfect human accuracy → 1.0
126+
- Perfect protocol accuracy → 1.0
127+
- 6 frameworks detected perfectly → 6.0 points
128+
129+
```math
130+
Final Score = (1.0 + 1.0 + 6.0) / 10
131+
= 8.0 / 10
132+
= 0.8
133+
```
134+
135+
Any excessive human misclassification, protocol misses, or missing Selenium detection would reduce this to **0.0**.
136+
137+
---
138+
139+
## Submission Guide
140+
141+
To build and submit your solution, please follow the [Building a Submission Commit](../../miner/workflow/3.build-and-publish.md) guide.
142+
143+
## Submission Templates
144+
145+
Templates and building instructions can be found in the [ADA Detection repository](https://github.com/RedTeamSubnet/ada-detection-challenge/tree/main/templates/commit).

requirements.txt

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -10,4 +10,4 @@ pydantic[email,timezone]>=2.0.3,<3.0.0
1010
pydantic-settings>=2.8.1,<3.0.0
1111
GitPython~=3.1.44
1212
ab_sniffer @ git+https://github.com/RedTeamSubnet/ab-sniffer-challenge.git@v5.0.2
13-
ada_detection @ git+https://github.com/RedTeamSubnet/ada-detection-challenge.git@v1.0.2
13+
ada_detection @ git+https://github.com/RedTeamSubnet/ada-detection-challenge.git@v2.0.0

0 commit comments

Comments
 (0)