Skip to content

feat: extend DeepSafe with scientific safety benchmarks#1

Open
coincidence2020 wants to merge 12 commits into
mainfrom
deepsafe-sci
Open

feat: extend DeepSafe with scientific safety benchmarks#1
coincidence2020 wants to merge 12 commits into
mainfrom
deepsafe-sci

Conversation

@coincidence2020

Copy link
Copy Markdown
Collaborator

Summary

This PR extends DeepSafe with three science-focused safety benchmarks while retaining the existing WMDP integration:

  • SciHazard: evaluates harmful scientific responses using checklist matching, evidence search, executability, and novelty analysis.
  • Safe-Scientist: evaluates whether models safely handle risky scientific research requests.
  • SOSBench: measures over-safety and unnecessary refusal in scientific scenarios.

It also adds the corresponding datasets, evaluators, metrics, YAML configurations, and local execution scripts.

Implementation details

  • Integrates the SciHazard DeHarm evaluation pipeline.
  • Includes the SciHazard inputs and reproducibility-related checklist/search caches.
  • Excludes prompt traces and completed experimental outputs.
  • Keeps API credentials outside configuration files and loads them through environment variables.
  • Updates both README.md and README_ch.md to document the four scientific safety benchmarks.
  • Retains the existing WMDP implementation without modifying its original workflow.

Dataset availability

SciHazard inputs are included in this PR. Safe-Scientist and SOSBench dataset payloads are not redistributed; their expected local paths
are documented in docs/benchmark-data.md.

Verification

  • 22 passed
  • Checked for exposed credentials, internal service addresses, unpublished experiment commands, prompt traces, and unintended dataset
    outputs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant