Skip to content

Commit b26ec16

Browse files
tbitcstbitcs
authored
Merge pull request #21 from BitConcepts/develop
chore: gap analysis, doc cross-check, scaffold fix, GitHub metadata
2 parents 3e85da7 + 050bb20 commit b26ec16

12 files changed

Lines changed: 352 additions & 42 deletions

‎.specsmith/ledger-chain.txt‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1 +1,2 @@
11
c33daae014d19022f931693b19a3d858e568c61e7a3d959246b857a543e81533
2+
522c1c447906f02a4c35c2f7a22c0677cd4f704ec616c4de502b9c38edf5e3f3

‎AGENTS.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -2,14 +2,14 @@
22

33
**Project**: OEA: Structured Recursive Calibration for Generative Stability
44
**Phase**: See `scaffold.yml` — advance with `specsmith phase next`
5-
**Spec**: specsmith 0.10.1 / aee-research
5+
**Spec**: specsmith 0.11.3.dev427 / research-python
66

77
## Mission
88
Empirically validate the OEA (Ontology, Epistemic, Agentic) Framework as a measurable
99
guardrail against recursive model collapse. Produce a peer-reviewed publication artifact.
1010

1111
## Project Summary
12-
- **Type**: aee-research (Applied Epistemic Engineering research paper)
12+
- **Type**: research-python with AEE epistemic governance (`enable_epistemic: true`)
1313
- **Language**: Python 3.x
1414
- **Test framework**: pytest
1515
- **Experiment harness**: `experiments/credibility_suite.py`, `experiments/run_experiments.py`

‎CHANGELOG.md‎

Lines changed: 18 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -9,18 +9,33 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
99

1010
### Added
1111
- `Dockerfile.cuda`: NVIDIA CUDA 12.1 GPU image (verified on RTX 4070 SUPER)
12+
- `Dockerfile.rocm`: AMD ROCm 6.x GPU image (community-tested; `rocm/dev-ubuntu-22.04:6.3` base)
13+
- `Dockerfile.xpu`: Intel Arc / Xe XPU image (community-tested; `ubuntu:22.04` + PyTorch XPU wheel)
1214
- `.github/ISSUE_TEMPLATE/hardware_compat.md`: hardware compatibility report template
1315
for community contributors running on AMD ROCm, Intel XPU, Apple MPS, etc.
1416
- `real_lm_experiment.py`: `--device` flag for explicit backend selection
1517
(`cuda`, `rocm`, `xpu`, `mps`, `cpu`); auto-detection extended to ROCm and Intel XPU
1618
- `requirements-lock.txt`: added install instructions for AMD ROCm 6.x, Intel XPU/Arc,
1719
NVIDIA CUDA 12.4+, and Apple MPS with per-backend test status notes
20+
- `docs/REQUIREMENTS.md`: REQ-OEA-023 (hardware abstraction / multi-backend device support)
21+
- `docs/TESTS.md`: TEST-OEA-023 covering REQ-OEA-023 (code inspection + Docker image check)
22+
23+
### Fixed
24+
- `scaffold.yml`: type changed `aee-research` → `research-python` to match scanner detection
25+
(AEE epistemic governance preserved via `enable_epistemic: true`); resolves specsmith
26+
audit type-mismatch warning — audit now passes 30/30 checks with no issues
1827

1928
### Changed
2029
- `Dockerfile`: updated to current pinned versions (`numpy==2.4.5`, etc.)
21-
- `README.md`: GPU support table now includes ROCm/XPU/MPS with test status column
22-
and CI hardware gap note; Docker section consolidated into GPU Support
23-
- `REPRODUCE.md`: hardware test matrix added; untested hardware / help-wanted section added
30+
- `README.md`: Docker table expanded with ROCm/XPU images and MPS native-only note
31+
- `REPRODUCE.md`: Step 4 rewritten with direct pip commands per backend (removed stale
32+
setup script references); stale numpy<2 compat note removed; Docker section updated
33+
with ROCm/XPU run commands; `--device` flag examples added to Step 5
34+
- `docs/ARCHITECTURE.md`: DEC-005 added (hardware abstraction layer); reproducibility
35+
package table updated with all four Dockerfiles; tooling section updated
36+
- `docs/REQUIREMENTS.md`: REQ-OEA-020 updated to reference `Dockerfile.cuda` alongside
37+
`Dockerfile`
38+
- `docs/TESTS.md`: TEST-OEA-020 updated to reference `Dockerfile.cuda`
2439
- `scaffold.yml`: pinned `detected_type: aee-research` to suppress specsmith audit false-positive
2540
(scanner infers `research-python` from file heuristics; `aee-research` is the intentional
2641
governance type set at project bootstrap)

‎Dockerfile.rocm‎

Lines changed: 85 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,85 @@
1+
# OEA Framework Paper — AMD ROCm GPU Container (REQ-OEA-020)
2+
#
3+
# COMMUNITY-TESTED ONLY — not verified by maintainer.
4+
# Please report your result (pass or fail) at:
5+
# https://github.com/BitConcepts/oea-framework-paper/issues/new?template=hardware_compat.md
6+
#
7+
# Requirements:
8+
# - AMD GPU with ROCm 6.x support (RX 6000/7000 series, Instinct MI series)
9+
# - ROCm-capable Linux host (Ubuntu 22.04/24.04 recommended)
10+
# - Linux only — ROCm does not support Windows or macOS containers
11+
# - Note: /dev/kfd and /dev/dri group permissions may need host-side setup:
12+
# sudo usermod -aG render,video $USER
13+
#
14+
# Build:
15+
# docker build -f Dockerfile.rocm -t oea-framework-rocm .
16+
#
17+
# Run real LLM experiment (AMD GPU):
18+
# docker run --rm \
19+
# --device /dev/kfd \
20+
# --device /dev/dri \
21+
# --group-add render \
22+
# --group-add video \
23+
# -v $(pwd)/results:/app/results \
24+
# oea-framework-rocm \
25+
# python experiments/real_lm_experiment.py --model distilgpt2 --device rocm
26+
#
27+
# Run bigram experiments (CPU, no GPU needed):
28+
# docker run --rm -v $(pwd)/results:/app/results oea-framework-rocm
29+
#
30+
# Troubleshooting:
31+
# If torch.cuda.is_available() returns False inside the container, verify:
32+
# 1. /dev/kfd exists on the host: ls -la /dev/kfd
33+
# 2. Your GPU is in the ROCm supported list:
34+
# https://rocm.docs.amd.com/en/latest/compatibility/compatibility-matrix.html
35+
# 3. The render/video groups are added to your user (see above)
36+
37+
FROM rocm/dev-ubuntu-22.04:6.3
38+
39+
# Avoid interactive prompts during apt installs
40+
ENV DEBIAN_FRONTEND=noninteractive
41+
42+
# System dependencies + Python 3.11
43+
RUN apt-get update && apt-get install -y --no-install-recommends \
44+
python3.11 \
45+
python3.11-venv \
46+
python3-pip \
47+
git \
48+
curl \
49+
&& rm -rf /var/lib/apt/lists/*
50+
51+
# Make python3.11 the default python/pip
52+
RUN update-alternatives --install /usr/bin/python python /usr/bin/python3.11 1 \
53+
&& update-alternatives --install /usr/bin/pip pip /usr/bin/pip3 1
54+
55+
WORKDIR /app
56+
57+
# Copy project files
58+
COPY . .
59+
60+
# Core experiment dependencies (no GPU required)
61+
RUN pip install --no-cache-dir \
62+
"numpy==2.4.5" \
63+
"matplotlib==3.10.9" \
64+
"scipy==1.17.1" \
65+
"pytest==9.0.3" \
66+
"reportlab==4.5.1"
67+
68+
# Neural LLM dependencies — ROCm 6.3 torch wheel
69+
# Note: torch.cuda.is_available() returns True for ROCm builds (ROCm exposes CUDA API)
70+
# Use --device rocm flag or the harness will auto-detect via torch.version.hip
71+
RUN pip install --no-cache-dir \
72+
"torch" \
73+
"transformers==4.41.0" \
74+
"rouge-score==0.1.2" \
75+
--index-url https://download.pytorch.org/whl/rocm6.3
76+
77+
# Verify installation (GPU visibility requires /dev/kfd at runtime, not build time)
78+
RUN python -c "import numpy, matplotlib, torch, transformers; \
79+
print('Environment OK'); \
80+
print(f'PyTorch {torch.__version__}'); \
81+
is_rocm = hasattr(torch.version, 'hip') and torch.version.hip; \
82+
print(f'ROCm build: {is_rocm}')"
83+
84+
# Default: run all CPU bigram experiments (AMD GPU available for real LLM experiments)
85+
CMD ["bash", "scripts/run_all_experiments.sh"]

‎Dockerfile.xpu‎

Lines changed: 84 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,84 @@
1+
# OEA Framework Paper — Intel Arc / Xe XPU Container (REQ-OEA-020)
2+
#
3+
# COMMUNITY-TESTED ONLY — not verified by maintainer.
4+
# Please report your result (pass or fail) at:
5+
# https://github.com/BitConcepts/oea-framework-paper/issues/new?template=hardware_compat.md
6+
#
7+
# Requirements:
8+
# - Intel Arc / Xe / Iris Xe GPU (A-series, B-series, or later)
9+
# - Intel GPU drivers installed on the Linux host
10+
# - Linux only (Ubuntu 22.04/24.04 recommended)
11+
# - Intel oneAPI Base Toolkit (optional but recommended for best performance)
12+
# - Intel GPU device passthrough requires /dev/dri on the host
13+
#
14+
# Build:
15+
# docker build -f Dockerfile.xpu -t oea-framework-xpu .
16+
#
17+
# Run real LLM experiment (Intel GPU):
18+
# docker run --rm \
19+
# --device /dev/dri \
20+
# -v $(pwd)/results:/app/results \
21+
# oea-framework-xpu \
22+
# python experiments/real_lm_experiment.py --model distilgpt2 --device xpu
23+
#
24+
# Run bigram experiments (CPU, no GPU needed):
25+
# docker run --rm -v $(pwd)/results:/app/results oea-framework-xpu
26+
#
27+
# Troubleshooting:
28+
# If torch.xpu.is_available() returns False:
29+
# 1. Verify /dev/dri is accessible: ls -la /dev/dri
30+
# 2. Check Intel GPU driver: intel_gpu_top
31+
# 3. Verify torch XPU support: python -c "import torch; print(torch.xpu.is_available())"
32+
# 4. See Intel Extension for PyTorch docs:
33+
# https://intel.github.io/intel-extension-for-pytorch/
34+
35+
FROM ubuntu:22.04
36+
37+
# Avoid interactive prompts during apt installs
38+
ENV DEBIAN_FRONTEND=noninteractive
39+
40+
# System dependencies + Python 3.11
41+
RUN apt-get update && apt-get install -y --no-install-recommends \
42+
python3.11 \
43+
python3.11-venv \
44+
python3-pip \
45+
git \
46+
curl \
47+
gpg \
48+
&& rm -rf /var/lib/apt/lists/*
49+
50+
# Make python3.11 the default python/pip
51+
RUN update-alternatives --install /usr/bin/python python /usr/bin/python3.11 1 \
52+
&& update-alternatives --install /usr/bin/pip pip /usr/bin/pip3 1
53+
54+
WORKDIR /app
55+
56+
# Copy project files
57+
COPY . .
58+
59+
# Core experiment dependencies (no GPU required)
60+
RUN pip install --no-cache-dir \
61+
"numpy==2.4.5" \
62+
"matplotlib==3.10.9" \
63+
"scipy==1.17.1" \
64+
"pytest==9.0.3" \
65+
"reportlab==4.5.1"
66+
67+
# Neural LLM dependencies — PyTorch with XPU support
68+
# PyTorch 2.7+ includes native XPU backend (Intel Arc/Xe via SYCL)
69+
# intel-extension-for-pytorch provides additional optimizations (optional)
70+
RUN pip install --no-cache-dir \
71+
"torch" \
72+
"transformers==4.41.0" \
73+
"rouge-score==0.1.2" \
74+
--index-url https://download.pytorch.org/whl/xpu
75+
76+
# Verify installation (XPU visibility requires /dev/dri passthrough at runtime)
77+
RUN python -c "import numpy, matplotlib, torch, transformers; \
78+
print('Environment OK'); \
79+
print(f'PyTorch {torch.__version__}'); \
80+
xpu_present = hasattr(torch, 'xpu'); \
81+
print(f'XPU module present: {xpu_present}')"
82+
83+
# Default: run all CPU bigram experiments (Intel GPU available for real LLM experiments)
84+
CMD ["bash", "scripts/run_all_experiments.sh"]

‎LEDGER.md‎

Lines changed: 55 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -248,3 +248,58 @@
248248
- **Type**: migration
249249
- **Status**: complete
250250
- **Chain hash**: `c33daae014d19022...`
251+
252+
## 2026-05-19 — Multi-GPU support, governance hardening, full doc cross-check
253+
254+
**Objective**: Add community GPU support (ROCm/XPU), harden governance to 30/30,
255+
resolve all documentation gaps, and fix stale content across the repository.
256+
257+
**What was done**:
258+
259+
- **Multi-backend device support** (`real_lm_experiment.py`): `--device` flag added
260+
(`cuda`, `rocm`, `xpu`, `mps`, `cpu`); auto-detection chain `cuda > rocm > xpu > mps > cpu`;
261+
ROCm detected via `torch.version.hip`; community-tested backends emit issue-link at runtime.
262+
- **Docker images**: `Dockerfile.cuda` (NVIDIA, verified), `Dockerfile.rocm` (AMD ROCm 6.x,
263+
community-tested), `Dockerfile.xpu` (Intel Arc/Xe, community-tested). MPS documented as
264+
not Docker-compatible (Apple Metal not accessible from Linux containers).
265+
- **Hardware issue template**: `.github/ISSUE_TEMPLATE/hardware_compat.md` added for
266+
community ROCm/XPU/MPS compatibility reports.
267+
- **REQ-OEA-023 + TEST-OEA-023**: hardware abstraction (P2) added to REQUIREMENTS.md and
268+
TESTS.md. All 23 accepted REQs now have test coverage.
269+
- **DEC-005**: hardware abstraction decision documented in ARCHITECTURE.md.
270+
REQ-OEA-020 and TEST-OEA-020 updated to reference `Dockerfile.cuda`.
271+
- **`scaffold.yml` type fix**: `aee-research` → `research-python` to match scanner detection.
272+
AEE epistemic governance fully preserved via `enable_epistemic: true`.
273+
specsmith audit: 30/30 checks, 0 issues (was 29/29 with 1 issue).
274+
- **AGENTS.md**: spec version updated 0.10.1 → 0.11.3.dev427; type updated aee-research → research-python.
275+
- **REPRODUCE.md**: Step 4 rewritten with direct pip install commands per backend;
276+
stale `setup.sh --cuda/--mps` references removed; stale numpy<2 note removed;
277+
Docker section fully updated with ROCm/XPU run commands.
278+
- **requirements-lock.txt**: per-backend install instructions added (ROCm 6.x, XPU, CUDA 12.4+, MPS);
279+
incorrect ABI comment from dependabot bump fixed.
280+
- **Dependabot PRs**: all 4 merged (numpy 2.4.5, matplotlib 3.10.9, scipy 1.17.1, pytest 9.0.3).
281+
- **GitHub issues**: #12 (stress-test confidence parser), #13 (type false-positive),
282+
#14 (publication workflow feature), #5 (submission prep) — all closed with comments.
283+
- **specsmith migrate**: 0.11.3 → 0.11.3.dev427 applied; ledger-chain.txt committed.
284+
- **AMLA 2026**: evaluated as predatory conference (AIRCC, no CORE ranking, 9 co-located
285+
events same day, $390-490 fee). Not recommended. Issue #5 updated accordingly.
286+
287+
**Files changed**: `scaffold.yml`, `AGENTS.md`, `CHANGELOG.md`, `LEDGER.md`,
288+
`Dockerfile`, `Dockerfile.cuda`, `Dockerfile.rocm`, `Dockerfile.xpu`,
289+
`requirements-lock.txt`, `README.md`, `REPRODUCE.md`, `docs/ARCHITECTURE.md`,
290+
`docs/REQUIREMENTS.md`, `docs/TESTS.md`, `experiments/real_lm_experiment.py`,
291+
`.github/ISSUE_TEMPLATE/hardware_compat.md`
292+
293+
**Checks run**: `specsmith audit` (30/30), `specsmith validate` (5/5),
294+
`specsmith status` (CI ✓, 0 Dependabot alerts, 0 open PRs), pytest (12/12), CI green.
295+
296+
**Results**: Healthy. 30/30 audit checks. 0 open issues. 0 open PRs. CI passing.
297+
298+
**Next step**: Merge develop → main when ready to publish hardware support.
299+
300+
## 2026-05-19T13:38 — Multi-GPU support, governance hardening, full doc cross-check: added --device flag (cuda/rocm/xpu/mps/cpu) with ROCm/XPU auto-detection; Dockerfile.cuda (verified), Dockerfile.rocm, Dockerfile.xpu (community-tested); hardware_compat issue template; REQ/TEST-OEA-023 (hardware abstraction); DEC-005 in ARCHITECTURE; scaffold.yml type aee-research->research-python (specsmith audit 30/30 clean); AGENTS.md spec version 0.10.1->0.11.3.dev427; REPRODUCE.md stale content fixed; requirements-lock.txt per-backend install instructions; 4 dependabot PRs merged; GitHub issues #5 #12 #13 #14 closed; AMLA 2026 evaluated as predatory conference
301+
- **Author**: Tristen Pierson
302+
- **Type**: feature
303+
- **REQs affected**: REQ-OEA-020,REQ-OEA-023
304+
- **Status**: complete
305+
- **Chain hash**: `522c1c447906f02a...`

‎README.md‎

Lines changed: 15 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -75,13 +75,19 @@ Use `--device <backend>` to override.
7575
7676
### Docker
7777

78-
| Image | GPU | Build command |
79-
|---|---|---|
80-
| `Dockerfile` | CPU only | `docker build -t oea-framework .` |
81-
| `Dockerfile.cuda` | NVIDIA CUDA 12.1 | `docker build -f Dockerfile.cuda -t oea-framework-cuda .` |
78+
| Image | GPU | Status | Build command |
79+
|---|---|---|---|
80+
| `Dockerfile` | CPU only | ✅ Verified | `docker build -t oea-framework .` |
81+
| `Dockerfile.cuda` | NVIDIA CUDA 12.1 | ✅ Verified | `docker build -f Dockerfile.cuda -t oea-framework-cuda .` |
82+
| `Dockerfile.rocm` | AMD ROCm 6.x | ⚠️ Community-tested | `docker build -f Dockerfile.rocm -t oea-framework-rocm .` |
83+
| `Dockerfile.xpu` | Intel Arc / Xe XPU | ⚠️ Community-tested | `docker build -f Dockerfile.xpu -t oea-framework-xpu .` |
84+
| Apple MPS | ❌ Not Docker-compatible | N/A — use native install | — |
85+
86+
ROCm requires `--device /dev/kfd --device /dev/dri --group-add render --group-add video` at runtime (Linux only).
87+
XPU requires `--device /dev/dri` at runtime (Linux only).
88+
For Apple Silicon, install natively — MPS is not accessible from inside Docker containers.
8289

83-
For AMD ROCm or Intel XPU Docker, see `requirements-lock.txt` for install commands
84-
and open a [Hardware Compatibility issue](https://github.com/BitConcepts/oea-framework-paper/issues/new?template=hardware_compat.md) with your result.
90+
Report ROCm/XPU/MPS results via the [Hardware Compatibility template](https://github.com/BitConcepts/oea-framework-paper/issues/new?template=hardware_compat.md).
8591

8692
## Repository Structure
8793

@@ -106,7 +112,9 @@ scripts/ Setup, build, and run scripts
106112
tests/ 12 unit tests (pytest)
107113
REPRODUCE.md Step-by-step reproduction guide
108114
Dockerfile CPU reproducibility container
109-
Dockerfile.cuda NVIDIA CUDA GPU container
115+
Dockerfile.cuda NVIDIA CUDA 12.1 GPU container (verified)
116+
Dockerfile.rocm AMD ROCm 6.x GPU container (community-tested)
117+
Dockerfile.xpu Intel Arc / Xe XPU container (community-tested)
110118
```
111119

112120
## Experiments

0 commit comments

Comments
 (0)