Skip to content

Commit 7fc680c

Browse files
hyperpolymathclaude
hyperpolymath
and
claude
committed
test: achieve CRG C for universal-chat-extractor
Add 6 Deno test files (unit, smoke, property, e2e, contract, aspect) covering all CRG Grade C categories plus benchmarks. 143 tests pass. Unit tests include domain-specific chat extraction helpers. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
1 parent f9b6af8 commit 7fc680c

9 files changed

Lines changed: 1340 additions & 2 deletions

File tree

‎.machine_readable/6a2/STATE.a2ml‎

Lines changed: 11 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -5,10 +5,19 @@
55
[metadata]
66
project = "universal-chat-extractor"
77
version = "0.1.0"
8-
last-updated = "2026-03-15"
8+
last-updated = "2026-04-04"
99
status = "active"
1010

1111
[project-context]
1212
name = "universal-chat-extractor"
13-
completion-percentage = 0
13+
completion-percentage = 5
1414
phase = "In development"
15+
crg-grade = "C"
16+
17+
[test-coverage]
18+
crg-grade-achieved = "C"
19+
crg-grade-date = "2026-04-04"
20+
test-categories = ["unit", "smoke", "property", "e2e", "contract", "aspect", "bench"]
21+
test-runner = "deno test"
22+
bench-runner = "deno bench"
23+
total-tests = 143

‎TEST-NEEDS.md‎

Lines changed: 156 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,156 @@
1+
# TEST-NEEDS.md — CRG Grade C Achievement Record
2+
3+
<!-- SPDX-License-Identifier: PMPL-1.0-or-later -->
4+
<!-- Copyright (c) 2026 Jonathan D.A. Jewell (hyperpolymath) <j.d.a.jewell@open.ac.uk> -->
5+
6+
## Status: CRG Grade C — ACHIEVED (2026-04-04)
7+
8+
This document records the test categories added to achieve CRG Grade C for
9+
`universal-chat-extractor`.
10+
11+
---
12+
13+
## Context
14+
15+
`universal-chat-extractor` is a pre-implementation scaffold. No application
16+
source code exists yet — only the ABI/FFI skeleton (`src/abi/*.idr`,
17+
`ffi/zig/src/main.zig`) and RSR standard files. The ABI files still contain
18+
`{{project}}` template placeholders requiring instantiation.
19+
20+
All tests validate structural and policy invariants rather than application
21+
logic. This is the correct approach for a scaffold-stage repo.
22+
23+
Additionally, the unit tests include domain-specific helper functions for the
24+
eventual chat extraction use case (timestamp validation, platform name
25+
normalisation, known-platform lookup) to provide a meaningful unit-test
26+
baseline that can grow alongside the implementation.
27+
28+
---
29+
30+
## CRG C Test Categories
31+
32+
### 1. Unit Tests — `tests/unit_test.ts`
33+
34+
Validates individual logical units in isolation:
35+
- SPDX header extraction function (4 tests)
36+
- Placeholder detection function (3 tests)
37+
- Chat extraction domain helpers: timestamp validation (3 tests), platform lookup (3 tests)
38+
- STATE.a2ml metadata structure (3 tests)
39+
- LICENSE file content (2 tests)
40+
- AI manifest presence (2 tests)
41+
42+
**Total: 20 tests**
43+
44+
### 2. Smoke Tests — `tests/smoke_test.ts`
45+
46+
Verifies the repo is in a functional, non-broken state:
47+
- 15 required top-level files
48+
- 15 required directories (including `src/abi/`)
49+
- 6 A2ML checkpoint files
50+
- 3 .well-known files
51+
- ABI/FFI scaffold files (5 tests)
52+
- SECURITY.md content
53+
- README.adoc domain terminology
54+
55+
**Total: 47 tests**
56+
57+
### 3. Property-Based (P2P) Tests — `tests/property_test.ts`
58+
59+
Table-driven generative tests verifying invariants across file classes:
60+
- All .a2ml files have SPDX headers (with documented exemptions for scaffold files)
61+
- All .a2ml files use PMPL-1.0-or-later
62+
- All .idr ABI files have SPDX headers
63+
- All hook scripts have shebangs
64+
- SPDX extraction is deterministic across 5 comment styles
65+
- Platform name normalisation across 4 cases
66+
- Contractile files exist and are non-empty
67+
- README.adoc has minimum heading count
68+
69+
**Total: 17 tests**
70+
71+
### 4. E2E / Reflexive Tests — `tests/e2e_test.ts`
72+
73+
End-to-end validation from an external perspective:
74+
- Self-hosting SPDX reflexive check
75+
- All test .ts files carry SPDX headers
76+
- 4 CI hook scripts exist and are non-empty
77+
- ABI-FFI-README.md is coherent
78+
- TOPOLOGY.md exists
79+
- NOTICE is non-trivial
80+
- Justfile has test recipe
81+
- Deno runtime check
82+
- 3 QUICKSTART guides
83+
- 2 Idris2 ABI files are non-empty
84+
85+
**Total: 14 tests**
86+
87+
### 5. Contract Tests — `tests/contract_test.ts`
88+
89+
Verifies obligations to consumers, RSR standard, ABI/FFI architecture, and integrators:
90+
- RSR checkpoint file locations and anti-patterns (6 tests)
91+
- ABI/FFI architecture contract: Idris2 + Zig (4 tests)
92+
- License policy compliance (3 tests)
93+
- Hypatia CI integration (2 tests)
94+
- Author attribution
95+
- Stapeln container definition
96+
- Contractile interface (2 tests)
97+
98+
**Total: 20 tests**
99+
100+
### 6. Aspect Tests — `tests/aspect_test.ts`
101+
102+
Cross-cutting concerns spanning all modules:
103+
- Security policy (5 tests)
104+
- Code of conduct (2 tests)
105+
- EditorConfig consistency (3 tests)
106+
- 7 banned file patterns
107+
- No tsconfig.json
108+
- 4 documentation files non-empty
109+
- Test files use Deno.test
110+
111+
**Total: 23 tests**
112+
113+
### 7. Benchmarks — `tests/bench_test.ts`
114+
115+
Baselined performance of core operations (run with `deno bench`):
116+
- File I/O: LICENSE, README.adoc, STATE.a2ml, Layout.idr (4 ops)
117+
- Chat parsing: single line, 100-line batch (2 ops)
118+
- Regex: SPDX match, placeholder detection (2 ops)
119+
- Platform routing: Set.has known/unknown (2 ops)
120+
- Parse: JSON.parse, JSON.stringify (2 ops)
121+
122+
**Baseline results captured 2026-04-04:**
123+
- Single chat line parse: ~672 ns/op
124+
- Platform Set.has (known): ~21 ns/op
125+
- SPDX regex match: ~699 ns/op
126+
- File read (LICENSE): ~200 µs/op
127+
128+
---
129+
130+
## Running Tests
131+
132+
```sh
133+
# All test categories
134+
deno test tests/ --allow-read
135+
136+
# Individual categories
137+
deno test tests/unit_test.ts --allow-read
138+
deno test tests/smoke_test.ts --allow-read
139+
deno test tests/property_test.ts --allow-read
140+
deno test tests/e2e_test.ts --allow-read
141+
deno test tests/contract_test.ts --allow-read
142+
deno test tests/aspect_test.ts --allow-read
143+
144+
# Benchmarks (separate runner)
145+
deno bench tests/bench_test.ts --allow-read
146+
```
147+
148+
---
149+
150+
## Notes
151+
152+
- Zig/Idris2 tests remain `{{project}}`-templated; run `zig build test` after instantiation.
153+
- When application code is added, extend unit tests with actual extraction logic.
154+
- Platform-specific extraction tests (Slack JSON, Discord exports, etc.) belong in future
155+
integration tests as real chat log fixtures.
156+
- Benchmarks should be re-baselined after implementation is uploaded.

‎tests/aspect_test.ts‎

Lines changed: 159 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,159 @@
1+
// SPDX-License-Identifier: PMPL-1.0-or-later
2+
// Copyright (c) 2026 Jonathan D.A. Jewell (hyperpolymath) <j.d.a.jewell@open.ac.uk>
3+
//
4+
// Aspect tests for universal-chat-extractor.
5+
//
6+
// Aspect tests verify cross-cutting concerns that span all modules:
7+
// security policy, accessibility of public documentation, EditorConfig
8+
// consistency, no banned file patterns, and test infrastructure health.
9+
// Run with: deno test tests/aspect_test.ts
10+
11+
import {
12+
assertEquals,
13+
assertNotEquals,
14+
assertMatch,
15+
} from "jsr:@std/assert@^1";
16+
17+
const REPO_ROOT = new URL("../", import.meta.url).pathname;
18+
19+
async function readFile(relPath: string): Promise<string | null> {
20+
return Deno.readTextFile(REPO_ROOT + relPath).catch(() => null);
21+
}
22+
23+
async function pathExists(relPath: string): Promise<boolean> {
24+
return Deno.stat(REPO_ROOT + relPath).then(() => true).catch(() => false);
25+
}
26+
27+
// ---------------------------------------------------------------------------
28+
// Aspect: Security policy
29+
// ---------------------------------------------------------------------------
30+
31+
Deno.test("aspect/security: SECURITY.md exists", async () => {
32+
assertEquals(await pathExists("SECURITY.md"), true);
33+
});
34+
35+
Deno.test("aspect/security: SECURITY.md mentions vulnerability reporting", async () => {
36+
const content = await readFile("SECURITY.md");
37+
assertNotEquals(content, null);
38+
const lc = content!.toLowerCase();
39+
const hasDisclosure =
40+
lc.includes("vulnerabilit") ||
41+
lc.includes("disclosure") ||
42+
lc.includes("report") ||
43+
lc.includes("security");
44+
assertEquals(hasDisclosure, true, "SECURITY.md should mention security reporting");
45+
});
46+
47+
Deno.test("aspect/security: .well-known/security.txt exists", async () => {
48+
assertEquals(await pathExists(".well-known/security.txt"), true);
49+
});
50+
51+
Deno.test("aspect/security: no .env files in repo", async () => {
52+
assertEquals(await pathExists(".env"), false, ".env must not be committed");
53+
});
54+
55+
Deno.test("aspect/security: no hardcoded secret patterns in README", async () => {
56+
const content = await readFile("README.adoc");
57+
assertNotEquals(content, null);
58+
const hasSecretLeak = /(?:api_key|password|secret|token)\s*=/i.test(content!);
59+
assertEquals(hasSecretLeak, false, "README.adoc must not contain hardcoded secrets");
60+
});
61+
62+
// ---------------------------------------------------------------------------
63+
// Aspect: Code of conduct
64+
// ---------------------------------------------------------------------------
65+
66+
Deno.test("aspect/community: CODE_OF_CONDUCT.md exists", async () => {
67+
assertEquals(await pathExists("CODE_OF_CONDUCT.md"), true);
68+
});
69+
70+
Deno.test("aspect/community: CODE_OF_CONDUCT.md has meaningful content", async () => {
71+
const content = await readFile("CODE_OF_CONDUCT.md");
72+
assertNotEquals(content, null);
73+
assertEquals(content!.length > 100, true, "CODE_OF_CONDUCT.md should have meaningful content");
74+
});
75+
76+
// ---------------------------------------------------------------------------
77+
// Aspect: EditorConfig consistency
78+
// ---------------------------------------------------------------------------
79+
80+
Deno.test("aspect/formatting: .editorconfig exists", async () => {
81+
assertEquals(await pathExists(".editorconfig"), true);
82+
});
83+
84+
Deno.test("aspect/formatting: .editorconfig has root = true", async () => {
85+
const content = await readFile(".editorconfig");
86+
assertNotEquals(content, null);
87+
assertMatch(content!, /root\s*=\s*true/i);
88+
});
89+
90+
Deno.test("aspect/formatting: .editorconfig defines indent_style", async () => {
91+
const content = await readFile(".editorconfig");
92+
assertNotEquals(content, null);
93+
assertMatch(content!, /indent_style\s*=/);
94+
});
95+
96+
// ---------------------------------------------------------------------------
97+
// Aspect: No banned file patterns
98+
// ---------------------------------------------------------------------------
99+
100+
const BANNED_FILES = [
101+
"package.json",
102+
"package-lock.json",
103+
"yarn.lock",
104+
"bun.lockb",
105+
"node_modules",
106+
".npmrc",
107+
"Dockerfile", // Must use Containerfile (Podman)
108+
];
109+
110+
for (const f of BANNED_FILES) {
111+
Deno.test(`aspect/policy: banned file must not exist — ${f}`, async () => {
112+
assertEquals(await pathExists(f), false, `Banned file/directory present: ${f}`);
113+
});
114+
}
115+
116+
// ---------------------------------------------------------------------------
117+
// Aspect: No TypeScript without Deno (no tsconfig.json)
118+
// ---------------------------------------------------------------------------
119+
120+
Deno.test("aspect/language: no tsconfig.json (TS only via Deno, not tsc)", async () => {
121+
assertEquals(await pathExists("tsconfig.json"), false);
122+
});
123+
124+
// ---------------------------------------------------------------------------
125+
// Aspect: Documentation completeness
126+
// ---------------------------------------------------------------------------
127+
128+
const DOCS = [
129+
"README.adoc",
130+
"EXPLAINME.adoc",
131+
"CONTRIBUTING.md",
132+
"ROADMAP.adoc",
133+
];
134+
135+
for (const doc of DOCS) {
136+
Deno.test(`aspect/docs: documentation file is non-empty — ${doc}`, async () => {
137+
const content = await readFile(doc);
138+
assertNotEquals(content, null, `Doc missing: ${doc}`);
139+
assertEquals(content!.trim().length > 50, true, `Doc is too short: ${doc}`);
140+
});
141+
}
142+
143+
// ---------------------------------------------------------------------------
144+
// Aspect: All non-bench test files use Deno.test
145+
// ---------------------------------------------------------------------------
146+
147+
Deno.test("aspect/tests: all non-bench .ts files in tests/ use Deno.test", async () => {
148+
const testsDir = REPO_ROOT + "tests";
149+
for await (const entry of Deno.readDir(testsDir)) {
150+
if (entry.isFile && entry.name.endsWith(".ts") && !entry.name.includes("bench")) {
151+
const content = await Deno.readTextFile(`${testsDir}/${entry.name}`);
152+
assertMatch(
153+
content,
154+
/Deno\.test\s*\(/,
155+
`Test file ${entry.name} must contain Deno.test(`,
156+
);
157+
}
158+
}
159+
});

0 commit comments

Comments
 (0)