@@ -41,7 +41,7 @@ Map source files to their corresponding test files:
4141find packages/* /src/ -name ' *.py' -not -name ' __init__.py' -not -path ' */test*' | sort
4242
4343# Test files
44- find tests/ packages/* /tests/ -name ' test_*.py' 2> /dev/null | sort
44+ find packages/* /tests/ -name ' test_*.py' 2> /dev/null | sort
4545```
4646
4747For each source module, check if a corresponding test file exists. Flag:
@@ -61,10 +61,10 @@ Scan test files for tests that assert nothing meaningful:
6161``` bash
6262# Tests that only check "is not None" (often meaningless if the function
6363# can't return None)
64- grep -rn " assert .* is not None$" tests/ --include=' *.py'
64+ grep -rn " assert .* is not None$" packages/ * / tests/ --include=' *.py'
6565
6666# Test functions with no assert statements at all
67- grep -l " def test_" tests/ --include=' *.py' -r | while read f; do
67+ grep -l " def test_" packages/ * / tests/ --include=' *.py' -r | while read f; do
6868 # Count test functions vs assert statements
6969 TESTS=$( grep -c " def test_" " $f " )
7070 ASSERTS=$( grep -c " assert " " $f " )
@@ -92,12 +92,12 @@ Skip `tests_e2e/` - e2e tests have different assertion patterns.
9292
9393### 3. Import performance
9494
95- The repo has a 3-second import budget tested by ` tests/test_import_perf.py ` .
96- Check the current state:
95+ The repo has a 3-second import budget tested by
96+ ` packages/data-designer/tests/test_import_perf.py ` . Check the current state:
9797
9898``` bash
9999# Verify the test exists and read its thresholds
100- cat tests/test_import_perf.py 2> /dev/null || echo " not found"
100+ cat packages/data-designer/ tests/test_import_perf.py 2> /dev/null || echo " not found"
101101```
102102
103103Also check for heavy imports that bypass the lazy loading system:
@@ -200,15 +200,18 @@ same config every day.
200200
201201** What to always check:**
2022021 . Config build round-trip: column count and names survive ` .build() `
203- 2 . Validation: ` DataDesigner.validate(builder) ` succeeds for valid configs
204- 3 . Rejection: invalid inputs raise, not silently produce bad configs
203+ 2 . Rejection: invalid inputs raise, not silently produce bad configs
204+
205+ ** Limitation** : ` DataDesigner(model_providers=[]) ` raises
206+ ` NoModelProvidersError ` , so you cannot call ` DataDesigner.validate() `
207+ without at least one provider configured. Stick to config-layer checks
208+ (` DataDesignerConfigBuilder.build() ` , column type resolution) which do
209+ not require providers.
205210
206211** API reference** for writing checks:
207212
208213``` python
209214from data_designer.config.config_builder import DataDesignerConfigBuilder
210- from data_designer.interface.data_designer import DataDesigner
211- import tempfile
212215
213216# Build a config - use keyword args: name, column_type, sampler_type, params
214217builder = (
@@ -223,10 +226,6 @@ config = builder.build()
223226assert len (config.columns) >= 2
224227names = {c.name for c in config.columns}
225228assert ' id' in names and ' cat' in names
226-
227- # Validate through the full stack (no LLM needed for sampler-only)
228- dd = DataDesigner(artifact_path = tempfile.mkdtemp(), model_providers = [])
229- dd.validate(builder)
230229```
231230
232231Run at least 2 creative checks per audit. Document what you chose and why
@@ -290,15 +289,15 @@ Write the report to `/tmp/audit-{{suite}}.md`:
290289| Check | Status | Detail |
291290| -------| --------| --------|
292291| Package imports | OK/FAIL | All three packages import cleanly |
293- | Import timing | OK/FAIL | Xms (budget: 3s) |
292+ | Import timing | OK/FAIL | X.XXs (budget: 3s) |
294293| Registry completeness | OK/WARN | Column types resolve to config classes |
295294
296295** Creative checks** (describe what you tested and why):
297296
298- | Check | Sampler types used | Status | Detail |
299- | -------| -------------------- | --------| --------|
297+ | Check | What was tested | Status | Detail |
298+ | -------| -----------------| --------| --------|
300299| Config build #1 | e.g. uuid+poisson+datetime | OK/FAIL | ... |
301- | Validate #1 | ... | OK/FAIL | ... |
300+ | Rejection #1 | e.g. gaussian with negative std | OK/FAIL | ... |
302301| ... | ... | ... | ... |
303302
304303### Test isolation
0 commit comments