Merge pull request #83 from RedTeamSubnet/ab_sniffer_v4

Auto Browser Sniffer v4

Author: aliyuldashev

docs/challenge_templates/ab_sniffer_v3.md docs/challenge_templates/ab_sniffer_v4.mddocs/challenge_templates/ab_sniffer_v3.md renamed to docs/challenge_templates/ab_sniffer_v4.md

@@ -1,19 +1,19 @@
 ---
-title: Auto Browser Sniffer v3
+title: Auto Browser Sniffer v4
 ---
-# AB Sniffer v3 Submission Guide (Active after Aug 24th 2025 14:00 UTC)
+# AB Sniffer v4 Submission Guide (Active after Oct 16th 2025 10:00 UTC)
 
 ## Overview
 
-**AB Sniffer v3** is the next iteration of **AB Sniffer** challenge which tests participants' ability to develop a SDK that can detect and correctly identify automation frameworks by name. The challenge evaluates how well the SDK can analyze automation behavior and identify unique characteristics or "leaks" from different automation tools interacting with a web page. With the new iteration, we are introducing one more frameworks to be detected which is **Camoufox**
+**AB Sniffer v4** is the next iteration of **Auto Browser Sniffer** challenge which tests participants' ability to develop a SDK that can detect and correctly identify automation frameworks by name. The challenge evaluates how well the SDK can analyze automation behavior and identify unique characteristics or "leaks" from different automation tools interacting with a web page. With the new iteration, we are introducing one more frameworks to be detected which are **Botasaurus**, **Pydoll** and a **human-in-the-loop** interaction scenario.
 
-Participants must demonstrate precise detection capabilities across multiple automation frameworks while maintaining reliability across different execution headless mode.
+Participants must demonstrate precise detection capabilities across multiple automation frameworks while maintaining reliability across different execution headless mode and human-in-the-loop scenarios.
 
 ---
 
 ## Example Code and Submission Instructions
 
-Example codes for the AB Sniffer v3 can be found in the [`redteam_core/miner/commits/ab_sniffer_v3/`](https://github.com/RedTeamSubnet/RedTeam/blob/main/redteam_core/miner/commits/ab_sniffer_v3/) directory.
+Example codes for the AB Sniffer v4 can be found in the [`redteam_core/miner/commits/ab_sniffer_v4/`](https://github.com/RedTeamSubnet/RedTeam/blob/main/redteam_core/miner/commits/ab_sniffer_v4/) directory.
 
 ### Technical Requirements
 
@@ -23,7 +23,7 @@ Example codes for the AB Sniffer v3 can be found in the [`redteam_core/miner/com
 
 ### Core Requirements
 
-1. Use our template from [`redteam_core/miner/commits/ab_sniffer_v3/src/detection/detection.js`](https://github.com/RedTeamSubnet/RedTeam/blob/main/redteam_core/miner/commits/ab_sniffer_v3/src/detection/detection.js)
+1. Use our template from [`redteam_core/miner/commits/ab_sniffer_v4/src/detection/detection.js`](https://github.com/RedTeamSubnet/RedTeam/blob/main/redteam_core/miner/commits/ab_sniffer_v4/src/detection/detection.js)
 2. Keep the detection function signature unchanged
 3. Your SDK must:
    - Detect automation frameworks interacting with the page
@@ -41,6 +41,9 @@ Your SDK should be capable of detecting these automation frameworks:
 - **puppeteerextra**
 - **zendriver**
 - **camoufox**
+- **botasaurus**
+- **pydoll**
+- **human**
 
 ### Key Guidelines
 
@@ -62,7 +65,7 @@ Your SDK will be scored based on:
 - **Detection Accuracy**: Correctly identifying automation frameworks by name
 - **Consistency**: Maintaining accuracy across multiple test runs
 - **Coverage**: Number of frameworks successfully detected
-- **Minimum Requirement**: Must detect at least 2 of the 4 frameworks with 100% accuracy to qualify
+- **Minimum Requirement**: Must detect at least 7 frameworks with 100% accuracy to qualify
 
 ### Scoring System
 
@@ -81,21 +84,21 @@ We maintain strict originality standards:
 
 Follow 1~6 steps to submit your SDK.
 
-1. **Navigate to the AB Sniffer v2 Commit Directory**
+1. **Navigate to the AB Sniffer v4 Commit Directory**
 
     ```bash
-    cd redteam_core/miner/commits/ab_sniffer_v3
+    cd redteam_core/miner/commits/ab_sniffer_v4
     ```
 
 2. **Build the Docker Image**
 
-    To build the Docker image for the AB Sniffer v2 submission, run:
+    To build the Docker image for the AB Sniffer v4 submission, run:
 
     ```bash
-    docker build -t my_hub/ab_sniffer_v3-miner:0.0.1 .
+    docker build -t my_hub/ab_sniffer_v4-miner:0.0.1 .
 
     # For MacOS (Apple Silicon) to build AMD64:
-    DOCKER_BUILDKIT=1 docker build --platform linux/amd64 -t myhub/ab_sniffer_v3-miner:0.0.1 .
+    DOCKER_BUILDKIT=1 docker build --platform linux/amd64 -t myhub/ab_sniffer_v4-miner:0.0.1 .
     ```
 
 3. **Log in to Docker**
@@ -113,23 +116,23 @@ Follow 1~6 steps to submit your SDK.
     Push the tagged image to your Docker Hub repository:
 
     ```bash
-    docker push myhub/ab_sniffer_v3:0.0.1
+    docker push myhub/ab_sniffer_v4:0.0.1
     ```
 
 5. **Retrieve the SHA256 Digest**
 
     After pushing the image, retrieve the digest by running:
 
     ```bash
-    docker inspect --format='{{index .RepoDigests 0}}' myhub/ab_sniffer_v3:0.0.1
+    docker inspect --format='{{index .RepoDigests 0}}' myhub/ab_sniffer_v4:0.0.1
     ```
 
 6. **Update active_commit.yaml**
 
     Finally, go to the `neurons/miner/active_commit.yaml` file and update it with the new image tag:
 
     ```yaml
-    - ab_sniffer_v3---myhub/ab_sniffer_v3@<sha256:digest>
+    - ab_sniffer_v4---myhub/ab_sniffer_v4@<sha256:digest>
     ```
 
 ---

neurons/validator/base_validator.py

@@ -397,14 +397,8 @@ def forward_local_scoring(
         else:
             bt.logging.warning(
                 f"[FORWARD LOCAL SCORING] Skipping scoring for {today_key}"
-            )
-            bt.logging.info(
                 f"[FORWARD LOCAL SCORING] Current hour: {current_hour}, Scoring hour: {constants.SCORING_HOUR}"
-            )
-            bt.logging.info(
                 f"[FORWARD LOCAL SCORING] Scoring dates: {self.scoring_dates}"
-            )
-            bt.logging.info(
                 f"[FORWARD LOCAL SCORING] Revealed commits: {str(revealed_commits)[:100]}..."
             )
 
@@ -651,6 +645,9 @@ def get_revealed_commits(self) -> dict[str, list[MinerChallengeCommit]]:
         seen_docker_hub_ids: set[str] = set()
 
         revealed_commits: dict[str, list[MinerChallengeCommit]] = {}
+        _list_existing_commits = []
+        _list_revealed_commits = []
+        _list_skipped_commits = []
         for (uid, hotkey), commits in self.miner_commits.items():
             for challenge_name, commit in commits.items():
                 bt.logging.info(
@@ -669,22 +666,34 @@ def get_revealed_commits(self) -> dict[str, list[MinerChallengeCommit]]:
                             challenge_name
                         ].get_unique_scored_docker_hub_ids()
                     ):
-                        # Only reveal unique docker hub ids in one pass, also ignore if docker_hub_id has been scored
-                        bt.logging.info(
-                            f"[GET REVEALED COMMITS] Skipping commit, Already revealed: {uid} - {hotkey}"
+                        _list_existing_commits.append(
+                            f"{challenge_name}-{uid}-{hotkey}-{docker_hub_id}"
                         )
                         continue
                     else:
                         commit.docker_hub_id = docker_hub_id
                         this_challenge_revealed_commits.append(commit)
                         seen_docker_hub_ids.add(docker_hub_id)
-                        bt.logging.info(
-                            f"[GET REVEALED COMMITS] Revealed commit: {uid} - {hotkey} - {challenge_name} - {commit.encrypted_commit}"
+                        _list_revealed_commits.append(
+                            f"{challenge_name}-{uid}-{hotkey}-{docker_hub_id}"
                         )
                 else:
-                    bt.logging.info(
-                        f"[GET REVEALED COMMITS] Skipping commit: Not revealed yet: {uid} - {hotkey}"
+                    _list_skipped_commits.append(
+                        f"{challenge_name}-{uid}-{hotkey}-{commit.encrypted_commit}"
                     )
+        for list_name, list_data in [
+            ("Existing", sorted(_list_existing_commits)),
+            ("Revealed", sorted(_list_revealed_commits)),
+            ("Skipped", sorted(_list_skipped_commits)),
+        ]:
+            if list_data:
+                bt.logging.info(
+                    f"[GET REVEALED COMMITS] {list_name} commits: {'\n'.join(list_data)}"
+                )
+            else:
+                bt.logging.info(
+                    f"[GET REVEALED COMMITS] No {list_name.lower()} commits"
+                )
 
         return revealed_commits
 

redteam_core/challenge_pool/ab_sniffer_v3/src/api/endpoints/challenge/service.py

@@ -15,3 +15,10 @@ ABSC_API_PORT=10001
 # ABSC_API_DOCS_OPENAPI_URL="{api_prefix}/openapi.json"
 # ABSC_API_DOCS_DOCS_URL="{api_prefix}/docs"
 # ABSC_API_DOCS_REDOC_URL="{api_prefix}/redoc"
+
+ABSC_CHALLENGE_PUSHCUT_API_KEY="pushcut_api_key_here"
+ABSC_CHALLENGE_PUSHCUT_SHORTCUT="rt-abs"
+ABSC_CHALLENGE_PUSHCUT_SERVER_ID="pushcut_server_id_here"
+ABSC_CHALLENGE_PUSHCUT_WEB_URL="http://reward-app:10001/_web"
+
+RT_COMPARER_RANDOM_SEED="RANDOM_SEED"

…allenge_pool/ab_sniffer_v3/.dockerignore …allenge_pool/ab_sniffer_v4/.dockerignoreredteam_core/challenge_pool/ab_sniffer_v3/.dockerignore renamed to redteam_core/challenge_pool/ab_sniffer_v4/.dockerignore redteam_core/challenge_pool/ab_sniffer_v3/.dockerignore renamed to redteam_core/challenge_pool/ab_sniffer_v4/.dockerignore

@@ -49,14 +49,14 @@ cd ~/workspaces/projects
 
 ```sh
 git clone https://github.com/RedTeamSubnet/RedTeam.git && \
-    cd RedTeam/redteam_core/challenge_pool/ab_sniffer_v3
+    cd RedTeam/redteam_core/challenge_pool/ab_sniffer_v4
 ```
 
 **OPTION B.** Clone the repository (for **DEVELOPMENT**: git + ssh key):
 
 ```sh
 git clone --recursive git@github.com:RedTeamSubnet/RedTeam.git && \
-    cd RedTeam/redteam_core/challenge_pool/ab_sniffer_v3 && \
+    cd RedTeam/redteam_core/challenge_pool/ab_sniffer_v4 && \
     git submodule update --init --recursive && \
     git submodule foreach --recursive git checkout main
 ```

…hallenge_pool/ab_sniffer_v3/.env.example …hallenge_pool/ab_sniffer_v4/.env.exampleredteam_core/challenge_pool/ab_sniffer_v3/.env.example renamed to redteam_core/challenge_pool/ab_sniffer_v4/.env.example redteam_core/challenge_pool/ab_sniffer_v3/.env.example renamed to redteam_core/challenge_pool/ab_sniffer_v4/.env.example

@@ -32,7 +32,7 @@ def __init__(self, challenge_info: dict, metagraph: bt.metagraph):
 
         self.max_similarity = 0.4
         self.min_similarity = 0
-        self.min_score = 0.5
+        self.min_score = 0.7
         self.break_point = 0.6
         self.max_input = 1.0
         self.min_value = 0