Make SciKnowEval pip-installable

[MattFisher]

Summary

Transform SciKnowEval into a pip-installable Python package.

Background

We're implementing SciKnowEval in Inspect Evals and were hoping to have the option to use it as a dependency rather than duplicating large amounts of code.

I think this is about the minimal changes that will allow the project to be pip-installable. Whether you choose to release it on Pypi in future is up to you.

Changes Made

• Package Structure: Reorganized code into Python package hierarchy

  • Created sciknoweval/ main package directory
  • Added __init__.py files for all modules
  • Moved evaluation code into package structure

• Modern Packaging: Added pyproject.toml with:

  • Dynamic dependency loading from requirements.txt
  • CLI entry point for sciknoweval command
  • Complete metadata and project URLs
  • Development dependencies configuration

• Import System: Fixed all import paths

  • Updated to relative imports throughout codebase
  • Resolved circular import issues
  • Maintained backward compatibility

• Command Line Interface:

  • Added sciknoweval CLI command
  • Entry point automatically created during installation
  • Supports all existing command-line arguments

• Documentation: Updated README.md with:

  • Multiple installation options (PyPI, source, development)
  • New usage examples with CLI command
  • Backward compatibility with legacy usage

• Development Tools:

  • Comprehensive .gitignore with Python, IDE, and project-specific patterns
  • Exclusion of model files, outputs, and sensitive data

Installation Options

# From source (recommended)
pip install .

# Development mode
pip install -e .

# From GitHub (when merged)
pip install git+https://github.com/HICAI-ZJU/SciKnowEval.git

Usage

  # New CLI command
  sciknoweval --data_path data.json --word2vec_model_path model.bin --gen_evaluator gpt-4o --output_path results.json

  # Python module (also supported)
  python -m sciknoweval.eval --data_path data.json --word2vec_model_path model.bin --gen_evaluator gpt-4o --output_path results.json

  # Legacy usage (still works)
  python sciknoweval/eval.py --data_path data.json --word2vec_model_path model.bin --gen_evaluator gpt-4o --output_path results.json

Testing Completed

• Package structure validation (all files in correct locations)
• Python syntax validation (all 10 Python files pass)
• Import system validation (relative imports work correctly)
• Installation testing (pip install -e . succeeds)
• CLI command creation (executable created and callable)
• Dependencies loading (dynamic loading from requirements.txt)
• Entry point validation (main function accessible)

Backward Compatibility

All existing functionality preserved
Original usage patterns still supported
No breaking changes to evaluation logic
Same command-line arguments and behavior

[MattFisher]

@Irving-Feng what do you think of making the project pip-installable as part of the updates for v2?