[Move DISCO queue to core]: Update docs to reflect MMLU, scorer, and queue changes

- Add missing documentation for new core components introduced
alongside the MMLU benchmark: ModelScorer reference page,
InformativeSubsetQueue/DISCOQueue in task reference, get_with_assert
in exceptions reference, and HuggingFacePipelineModelAdapter rename
in model/HuggingFace pages.
- Add mmlu extra to README install section.
- Fix grammar in MMLU docs and fill CHANGELOG PR placeholders.

Author: arubique

CHANGELOG.md

@@ -41,8 +41,9 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 **Core**
 
-- Added `DISCOQueue` to `maseval.core.task` for subset-based evaluation (e.g., anchor-point selection for DISCO). Available via `from maseval import DISCOQueue`. (PR: #34)
-- Added `ModelScorer` abstract base class in `maseval.core.scorer` for log-likelihood scoring, with `loglikelihood()`, `loglikelihood_batch()`, and `loglikelihood_choices()` methods. (PR: #PR_NUMBER_PLACEHOLDER)
+- Added `InformativeSubsetQueue` and `DISCOQueue` to `maseval.core.task` for subset-based evaluation (e.g., anchor-point selection for DISCO). `DISCOQueue` accepts `anchor_points_path` to load indices from a `.json`/`.pkl` file via `DISCOQueue.load_anchor_points()`. Available via `from maseval import DISCOQueue, InformativeSubsetQueue`. (PR: #34)
+- Added `get_with_assert()` utility in `maseval.core.exceptions` for strict dictionary access that raises `KeyError` instead of silently returning a default. Supports nested key lookups. (PR: #34)
+- Added `ModelScorer` abstract base class in `maseval.core.scorer` for log-likelihood scoring, with `loglikelihood()`, `loglikelihood_batch()`, and `loglikelihood_choices()` methods. (PR: #34)
 - Added `SeedGenerator` abstract base class and `DefaultSeedGenerator` implementation for reproducible benchmark runs via SHA-256-based seed derivation (PR: #24)
 - Added `seed` and `seed_generator` parameters to `Benchmark.__init__` for enabling reproducibility (PR: #24)
 - Added `seed_generator` parameter to all benchmark setup methods (`setup_environment`, `setup_user`, `setup_agents`, `setup_evaluators`) (PR: #24)
@@ -53,8 +54,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 **Interface**
 
-- Added `HuggingFaceModelScorer` in `maseval.interface.inference` — log-likelihood scorer backed by a HuggingFace `AutoModelForCausalLM`, with single-token optimisation for MCQ evaluation. Implements the `ModelScorer` interface. (PR: #PR_NUMBER_PLACEHOLDER)
-- Renamed `HuggingFaceModelAdapter` → `HuggingFacePipelineModelAdapter` to distinguish it from the new scorer. The old name remains as a backwards-compatible alias. (PR: #PR_NUMBER_PLACEHOLDER)
+- Added `HuggingFaceModelScorer` in `maseval.interface.inference` — log-likelihood scorer backed by a HuggingFace `AutoModelForCausalLM`, with single-token optimisation for MCQ evaluation. Implements the `ModelScorer` interface. (PR: #34)
+- Renamed `HuggingFaceModelAdapter` → `HuggingFacePipelineModelAdapter` to distinguish it from the new scorer. The old name remains as a backwards-compatible alias. (PR: #34)
 
 - CAMEL-AI integration: `CamelAgentAdapter` and `CamelLLMUser` for evaluating CAMEL-AI ChatAgent-based systems (PR: #22)
   - Added `CamelAgentUser` for using a CAMEL ChatAgent as the user in agent-to-agent evaluation (PR: #22)

README.md

@@ -109,6 +109,13 @@ pip install "maseval[langgraph]"
 pip install "maseval[llamaindex]"
 ```
 
+Or install benchmark-specific dependencies:
+
+```bash
+# MMLU (HuggingFace models)
+pip install "maseval[mmlu]"
+```
+
 ## Example
 
 Examples are available in the [Documentation](https://maseval.readthedocs.io/en/stable/).

docs/benchmark/mmlu.md

@@ -88,7 +88,7 @@ tasks = load_tasks(
     anchor_points_path="/path/to/anchor_points.json",
 )
 
-# tasks is an DISCOQueue — only anchor tasks are evaluated
+# tasks is a DISCOQueue — only anchor tasks are evaluated
 print(f"Evaluating {len(tasks)} anchor tasks")
 ```
 

docs/interface/inference/huggingface.md

@@ -1,7 +1,18 @@
-# HuggingFace Inference Adapter
+# HuggingFace Inference Adapters
 
-This page documents the HuggingFace model adapter for MASEval.
+This page documents the HuggingFace model adapters for MASEval.
+
+## Pipeline Model Adapter (Text Generation)
 
 [:material-github: View source](https://github.com/parameterlab/maseval/blob/main/maseval/interface/inference/huggingface.py){ .md-source-file }
 
-::: maseval.interface.inference.huggingface.HuggingFaceModelAdapter
+::: maseval.interface.inference.huggingface.HuggingFacePipelineModelAdapter
+
+!!! note
+    `HuggingFaceModelAdapter` is a backwards-compatible alias for `HuggingFacePipelineModelAdapter`.
+
+## Model Scorer (Log-Likelihood)
+
+[:material-github: View source](https://github.com/parameterlab/maseval/blob/main/maseval/interface/inference/huggingface_scorer.py){ .md-source-file }
+
+::: maseval.interface.inference.huggingface_scorer.HuggingFaceModelScorer

docs/reference/exceptions.md

@@ -38,6 +38,10 @@ SimulatorError (base for simulators)
 
 ::: maseval.core.simulator.UserSimulatorError
 
+## Data Access Helpers
+
+::: maseval.core.exceptions.get_with_assert
+
 ## Validation Helpers
 
 These functions simplify input validation and raise `AgentError` with helpful suggestions:

docs/reference/model.md

@@ -20,7 +20,7 @@ The following adapter classes implement the ModelAdapter interface for specific
 
 [:material-github: View source](https://github.com/parameterlab/maseval/blob/main/maseval/interface/inference/huggingface.py){ .md-source-file }
 
-::: maseval.interface.inference.huggingface.HuggingFaceModelAdapter
+::: maseval.interface.inference.huggingface.HuggingFacePipelineModelAdapter
 
 [:material-github: View source](https://github.com/parameterlab/maseval/blob/main/maseval/interface/inference/google_genai.py){ .md-source-file }
 

docs/reference/scorer.md

@@ -0,0 +1,19 @@
+# Model Scorers
+
+Model Scorers provide a uniform interface for log-likelihood computation across model providers. Unlike `ModelAdapter` (which handles text generation and chat), scorers evaluate how likely a model considers a given continuation given some context.
+
+!!! note
+
+    `ModelScorer` is the scoring counterpart to `ModelAdapter`. Use it when you need log-likelihood evaluation (e.g., multiple-choice benchmarks) rather than text generation.
+
+[:material-github: View source](https://github.com/parameterlab/maseval/blob/main/maseval/core/scorer.py){ .md-source-file }
+
+::: maseval.core.scorer.ModelScorer
+
+## Interfaces
+
+The following scorer classes implement the ModelScorer interface for specific providers.
+
+[:material-github: View source](https://github.com/parameterlab/maseval/blob/main/maseval/interface/inference/huggingface_scorer.py){ .md-source-file }
+
+::: maseval.interface.inference.huggingface_scorer.HuggingFaceModelScorer

docs/reference/task.md

@@ -2,34 +2,42 @@
 
 Tasks define individual benchmark scenarios including inputs, expected outputs, and metadata for evaluation. Task queues control execution order and scheduling strategy.
 
-[:material-github: View source](https://github.com/parameterlab/maseval/blob/main/maseval/core/task.py#L55){ .md-source-file }
+[:material-github: View source](https://github.com/parameterlab/maseval/blob/main/maseval/core/task.py#L56){ .md-source-file }
 
 ::: maseval.core.task.Task
 
-[:material-github: View source](https://github.com/parameterlab/maseval/blob/main/maseval/core/task.py#L27){ .md-source-file }
+[:material-github: View source](https://github.com/parameterlab/maseval/blob/main/maseval/core/task.py#L28){ .md-source-file }
 
 ::: maseval.core.task.TaskProtocol
 
-[:material-github: View source](https://github.com/parameterlab/maseval/blob/main/maseval/core/task.py#L18){ .md-source-file }
+[:material-github: View source](https://github.com/parameterlab/maseval/blob/main/maseval/core/task.py#L19){ .md-source-file }
 
 ::: maseval.core.task.TimeoutAction
 
 ## Task Queues
 
 Task queues determine the order in which tasks are executed. Pass a queue to `Benchmark.run(queue=...)` to customize scheduling.
 
-[:material-github: View source](https://github.com/parameterlab/maseval/blob/main/maseval/core/task.py#L86){ .md-source-file }
+[:material-github: View source](https://github.com/parameterlab/maseval/blob/main/maseval/core/task.py#L87){ .md-source-file }
 
 ::: maseval.core.task.BaseTaskQueue
 
-[:material-github: View source](https://github.com/parameterlab/maseval/blob/main/maseval/core/task.py#L256){ .md-source-file }
+[:material-github: View source](https://github.com/parameterlab/maseval/blob/main/maseval/core/task.py#L257){ .md-source-file }
 
 ::: maseval.core.task.SequentialTaskQueue
 
-[:material-github: View source](https://github.com/parameterlab/maseval/blob/main/maseval/core/task.py#L276){ .md-source-file }
+[:material-github: View source](https://github.com/parameterlab/maseval/blob/main/maseval/core/task.py#L277){ .md-source-file }
+
+::: maseval.core.task.InformativeSubsetQueue
+
+[:material-github: View source](https://github.com/parameterlab/maseval/blob/main/maseval/core/task.py#L325){ .md-source-file }
+
+::: maseval.core.task.DISCOQueue
+
+[:material-github: View source](https://github.com/parameterlab/maseval/blob/main/maseval/core/task.py#L413){ .md-source-file }
 
 ::: maseval.core.task.PriorityTaskQueue
 
-[:material-github: View source](https://github.com/parameterlab/maseval/blob/main/maseval/core/task.py#L322){ .md-source-file }
+[:material-github: View source](https://github.com/parameterlab/maseval/blob/main/maseval/core/task.py#L459){ .md-source-file }
 
 ::: maseval.core.task.AdaptiveTaskQueue

mkdocs.yml

@@ -110,6 +110,7 @@ nav:
           - Exceptions: reference/exceptions.md
           - History: reference/history.md
           - Model: reference/model.md
+          - Scorer: reference/scorer.md
           - Seeding: reference/seeding.md
           - Simulator: reference/simulator.md
           - Tasks: reference/task.md