Merge pull request #105 from RedTeamSubnet/dfp-2

Device Fingerprinter V2

Author: aliyuldashev

.env.example

@@ -20,16 +20,3 @@ DEBUG=false
 # RT_EPOCH_LENGTH=1200
 RT_STORAGE_API_URL="https://storage-api.theredteam.io"
 
-
-## -- Challenge configs -- ##
-# CHALLENGE_API_KEY="changeme"
-# CHALLENGE_VERIFICATION_API_KEY="changeme"
-# RT_CHALLENGE_LOGS_DIR=volumes/storage/agent.miner/logs
-# CHALLENGE_VERIFICATION_ENDPOINT="http://localhost:8000/verify"
-# CHALLENGE_VERIFICATION_STARTUP_URL="http://localhost:8000/startup"
-
-## -- AAD Challenge configs -- ##
-# AAD_CHALLENGE_NSTBROWSER_API_KEY="changeme"
-# AAD_CHALLENGE_NSTBROWSER_HOST=localhost
-# AAD_CHALLENGE_NSTBROWSER_PORT=9222
-# AAD_CHALLENGE_NSTBROWSER_PROTOCOL=http

.gitmodules

@@ -1,6 +1,6 @@
-[submodule "src/challenges/ab_sniffer"]
-	path = src/challenges/ab_sniffer
-	url = ../ab-sniffer-challenge.git
 [submodule "src/challenges/ada_detection"]
 	path = src/challenges/ada_detection
 	url = ../ada-detection-challenge.git
+[submodule "src/challenges/dev_fingerprinter"]
+	path = src/challenges/dev_fingerprinter
+	url = https://github.com/RedTeamSubnet/dev-fingerprinter-challenge.git

docs/challenges/.nav.yml

@@ -1,8 +1,8 @@
 nav:
   - README.md
   - Active Challenges:
-    -   ab_sniffer : ./ab_sniffer
-    -   ada_detection : ./ada_detection
+      - ada_detection: ./ada_detection
+      - dev_fingerprinter: ./dev_fingerprinter
   - Inactive Challenges:
-    -   dev_fingerprinter : ./dev_fingerprinter
-    -   humanize_behaviour : ./humanize_behaviour
+      - humanize_behaviour: ./humanize_behaviour
+      - ab_sniffer: ./ab_sniffer

docs/challenges/dev_fingerprinter/README.md

@@ -2,7 +2,9 @@
 
 ## Overview
 
-The **Device Fingerprinter** challenge tests miners' ability to develop a browser SDK that can accurately detect the driver type used by bots interacting with a webpage. Miners must create a JavaScript fingerprinter that distinguishes between different devices using browser properties, behavior patterns, and technical signatures.
+The Device Fingerprinter challenge tests miners' ability to develop a browser SDK that can accurately and consistently identify physical
+devices across multiple browser environments. Miners must create a JavaScript fingerprinter that generates a unique, persistent ID for a
+specific device, regardless of whether it is accessed via Chrome, Safari, or Brave.
 
 ## General Technical Requirements
 
@@ -13,7 +15,7 @@ The **Device Fingerprinter** challenge tests miners' ability to develop a browse
 
 ## General Guidelines
 
-- **Fingerprint Collection**: Collect comprehensive browser and device properties while analyzing WebDriver signatures and automation artifacts.
+- **Fingerprint Collection**: Collect browser and device properties to identify the physical hardware across different browser engines.
 - **Data Processing**: Generate unique, consistent fingerprint hashes and follow the provided API endpoint structure for JSON payloads.
 - **Dependency Limitation**: Your dependencies must be older than January 1, 2025. Any package released on or after this date will not be accepted.
 - **Script Limitation**: Your script must not exceed 2,000 lines. Larger scripts will be considered invalid.
@@ -23,18 +25,9 @@ The **Device Fingerprinter** challenge tests miners' ability to develop a browse
 We maintain strict originality standards:
 
 - All submissions are compared against other miners' scripts.
-- 100% similarity = zero score.
-- Similarity above 60% results in proportional score penalties based on the **detected similarity percentage**.
-
-## Submission Path
-
-**Dedicated Path:** `examples/dev_fingerprinter_v1/src/fingerprinter/`
-
-Place your fingerprinting script in this directory before building your commit:
-
-- `fingerprinter.js` - Your main fingerprinting implementation
-
-Ensure your script keeps the `runFingerprinting()` function signature and export unchanged.
+- Under 40% Similarity: No penalty is applied.
+- 40% to 60% Similarity: Results in proportional score penalties based on the detected similarity percentage.
+- Above 60% Similarity: Results in an automatic score of zero (Disqualification).
 
 ## Submission
 

docs/challenges/dev_fingerprinter/testing_manuals.md

@@ -0,0 +1,58 @@
+---
+title: Testing Manual
+---
+
+# Device Fingerprinter (DFP) Testing Manual
+
+The **Miner Testing Sandbox** is the primary tool for developing and validating your `fingerprinter.js` script. It allows you to test your logic in isolation without the overhead of a full challenge session.
+
+## Why use the Sandbox?
+
+1.  **Isolated Testing**: Validate your `fingerprinter.js` immediately across different browser profiles.
+2.  **Performance Evaluation**: The sandbox uses the official "Two-Strike" rule to measure:
+    -   **Fragmentation (Internal Consistency)**: Ensures the same device consistently produces the same hash across different browsers.
+    -   **Collision (External Uniqueness)**: Ensures different devices produce unique hashes.
+3.  **Real-time Results**: View a breakdown of "Correct," "Collisions," and "Fragmentations" to see exactly where your script needs improvement.
+
+---
+
+## Quick Start Guide
+
+### Step 1: Setup the Proxy Environment
+You only need the DFP Proxy repository for testing:
+```bash
+git clone git@github.com:RedTeamSubnet/rest.dfp-proxy.git dfp_proxy
+cd dfp_proxy
+
+# Create the environment file (required for the app to start)
+cp .env.example .env
+```
+
+### Step 2: Provide Your Script
+Paste your implementation logic into:
+`src/api/static/js/fingerprinter.js`
+
+### Step 3: Start the Sandbox
+```bash
+./compose.sh start -l
+```
+
+### Step 4: Run Your Tests
+1.  **Open the Dashboard**: Visit **`http://localhost:8000/miner-test`** in your browser.
+2.  **Label Your Device**: Enter a name (e.g., `my-laptop`) and click **Submit Fingerprint**.
+3.  **Test Consistency (Fragmentation)**: 
+    - Open the same URL in a **different browser engine** (e.g., switch from Chrome to Brave or Safari).
+    - Use the **exact same Device Label**.
+    - Click Submit again. If your hash changed, you will see a **Fragmentation** error.
+4.  **Test Uniqueness (Collision)**: 
+    - Ask a friend to visit the URL or use a different physical machine.
+    - Use a **different Device Label** (e.g., `office-pc`).
+    - Click Submit. If they produce the same hash as you, you will see a **Collision** error.
+
+---
+
+## Development Workflow
+
+-   **View Results**: Click the **View Results** button at any time to see your current score and a detailed list of all collected hashes.
+-   **Clean Session**: Use the **Clean Session** button to wipe all test data and restart your testing cycle immediately.
+-   **Final Goal**: Aim for a **1.0 Score** in the sandbox with multiple browsers and devices before submitting your final commit.

docs/challenges/dev_fingerprinter/v2.md

@@ -0,0 +1,129 @@
+---
+title: Device Fingerprinter v2
+---
+
+# Device Fingerprinter (DFP) v2 (Active after 2026 February 25 10:00 UTC)
+
+## Overview
+
+**Device Fingerprinter (DFP) v2** is a high-precision challenge focused on device identity consistency and uniqueness. Participants must develop a browser SDK capable of generating a "fingerprint" string that remains identical for the same physical device across multiple browsers while remaining unique between different devices.
+
+The challenge evaluates your SDK's ability to resist browser-level variations (like user agents or window sizes) and extract hardware-level or deeper software signals that uniquely identify the hardware.
+
+For general information and technical setup, please refer to the [Main README](./README.md).
+
+---
+
+## Mandatory Requirements
+
+1. Use the template provided in the [`templates/commit/src/fingerprinter/`](./templates/commit/src/fingerprinter/fingerprinter.js) directory.
+2. Implement your logic within the `fingerprinter.js` file.
+3. Your SDK must:
+   - Generate a single string (the "fingerprint").
+   - Report this string to the evaluator when executed within a browser environment.
+   - Be self-contained and not rely on external API calls or non-standard browser features that require user permissions (e.g., Camera/Microphone).
+
+### Browsers
+
+Your SDK will be evaluated across multiple mobile and desktop browsers, including:
+
+- **Chrome**
+- **Brave**
+- **Firefox Focus**
+- **DuckDuckGo**
+- **Safari**
+
+All browsers are run in incognito mode. Each device will be tested in multiple "batches" across these browsers to ensure consistency.
+
+---
+
+## Key Guidelines
+
+- **Fragmentation (Internal Consistency)**: Your SDK must produce the same ID for a device even if the browser changes. If the ID changes between Chrome and Brave, you will be penalized for "Fragmentation."
+- **Collision (External Uniqueness)**: Your SDK must produce different IDs for different physical devices. If two devices share the same ID, you will be penalized for "Collision" via the **Two-Strike Rule**.
+- **Hardware Focus**: Avoid using volatile signals (like window size, battery level, or IP address) as they vary between browsers and sessions, leading to fragmentation penalties.
+
+---
+
+## Technical Architecture (Challenger & Proxy)
+
+DFP v2 utilizes a two-tier architecture to ensure a realistic evaluation environment:
+
+1. **Challenger API**: The "Brain" of the operation. It receives your submission, manages the test session, notifies physical devices via SMTP, and calculates the final score based on the data received.
+2. **DFP Proxy**: The "Gateway." It hosts your `fingerprinter.js` script and acts as the collection point for device reports. This prevents direct communication between the test devices and the main Challenger, ensuring security and isolation.
+
+### The Lifecycle of a Fingerprint
+
+- **Submission**: You send your `fingerprinter.js` to the Challenger.
+- **Distribution**: The Challenger automatically pushes your script to the Proxy.
+- **Collection**: Physical devices visit the Proxy, execute your script, and send the results back to the Proxy.
+- **Scoring**: The Proxy forwards the collected results to the Challenger for the final calculation.
+
+---
+
+## Mandatory Functions
+
+Participants must implement and export two specific async functions in their `fingerprinter.js` script. These are the entry points used by the evaluation environment.
+
+### 1. `runFingerprinting()`
+
+This function is responsible for gathering device signals and generating the fingerprint hash.
+
+- **Miner Requirement**: Miners must capture the exact 32-character string returned by this function and submit it unmodified.
+- **Goal**: Collect hardware/software data and return a unique, consistent hash.
+
+---
+
+## Evaluation Criteria
+
+- **High Consistency**: Maintain the same fingerprint for a single device across all 5+ target browsers.
+- **High Uniqueness**: Ensure no other physical device generates the same fingerprint.
+- **Minimum Participation**: At least 2 physical devices must successfully report fingerprints for a session to be valid.
+- **Stability**: The script must execute quickly and reliably within the evaluator's `fp_timeout` window.
+
+---
+
+## Scoring System
+
+The final score is an average of the points earned by each physical device in the session, normalized between **0.0 and 1.0**.
+
+### 1. Fragmentation Penalty
+
+Each device starts with **1.0 point**.
+
+- For every unique fingerprint string generated by the *same* device beyond the first, a penalty of **-0.3** is applied.
+- **Max Fragmentation**: If a device generates **3 or more** different IDs across browsers, its score for the session is automatically **0.0**.
+
+### 2. Two-Strike Collision Rule
+
+This rule ensures your ID is unique across the network.
+
+- **Strike 1**: If a fingerprint generated by a device is shared by *any* other physical device in **one** browser batch, a penalty of **-0.25** is applied.
+- **Strike 2**: If the same collision happens in **2 or more** batches, that device's score becomes **0.0**.
+
+### Scoring Example
+
+Imagine a session with 2 devices across 5 browsers (10 batches total with `n_repeat=2`):
+
+- **Device 1 (Perfect)**: Reports "ID-AAA" in all 10 batches.
+  - Unique IDs: 1 (Penalty: 0)
+  - Collisions: 0 (Penalty: 0)
+  - **Points: 1.0**
+
+- **Device 2 (Fragmented)**: Reports "ID-BBB" in 8 batches and "ID-CCC" in 2 batches.
+  - Unique IDs: 2 (Penalty: -0.3)
+  - Collisions: 0 (Penalty: 0)
+  - **Points: 0.7**
+
+- **Device 3 (Collided)**: Reports "ID-AAA" (Collides with Device 1) in 2 batches.
+  - Unique IDs: 1 (Penalty: 0)
+  - Collisions: 2 batches (Strike 2 triggered)
+  - **Points: 0.0**
+
+**Final Session Score**: `(1.0 + 0.7 + 0.0) / 3 = 0.567`
+
+---
+
+## Submission
+
+After local validation using the [Testing Manual](./testing_manuals.md), package your script into a submission container. Ensure your `fingerprinter.js` is correctly placed in the miner template.

requirements.txt

@@ -1,13 +1,10 @@
 bittensor~=9.9.0
 cryptography>=41.0.7,<50.0.0
 docker>=7.0.0,<8.0.0
-pandas>=2.1.4,<3.0.0
-datasets>=2.16.1,<3.0.0
 diskcache>=5.6.3,<6.0.0
 substrate-interface>=1.7.11,<1.8
 python-dotenv>=1.0.1,<2.0.0
 pydantic[email,timezone]>=2.0.3,<3.0.0
 pydantic-settings>=2.8.1,<3.0.0
-GitPython~=3.1.44
-ab_sniffer @ git+https://github.com/RedTeamSubnet/ab-sniffer-challenge.git@v5.0.2
-ada_detection @ git+https://github.com/RedTeamSubnet/ada-detection-challenge.git@v2.0.0
+dev_fingerprinter @ git+https://github.com/RedTeamSubnet/dev-fingerprinter-challenge.git@v2.0.0
+ada_detection @ git+https://github.com/RedTeamSubnet/ada-detection-challenge.git@v2.0.2