ALFMOB-272: Set up design token JSON parser & code generation

[khoinguyen-mindera]

Ticket

ALFMOB-272

Summary

• Add Tools/DesignTokenGen — a standalone, Foundation-only Swift generator that reads the W3C DTCG token export and emits committed, type-safe Swift. Kept outside the AlfieKit graph (app/CI never build it).
• Manifest-driven, iOS mode-selected load (System=ios, Screen Size=small); resolves {reference} chains; honours the export's cycle + broken-ref allow-lists exhaustively; deterministic (sorted, no timestamps) output.
• Preserves the token reference graph: primitives hold literals; Theme/Sizing/Typography reference Primitives.* symbols (e.g. Theme.buttonPrimaryBackgroundPrimaryDefault = Primitives.Colours.neutrals800, Typography.display.large).
• Add pull-design-tokens.sh (credential-safe, copies the iOS JSON subset) + generate-design-tokens.sh (gates the generator's own tests, then emits).
• Commit the source DTCG JSON (SharedUI/DesignTokens/, excluded from compile) and generated Swift (SharedUI/GeneratedTokens/); wire Package.swift exclude + SwiftLint exclude + never-edit marking; add Docs/DesignTokens.md.
• 33 generator unit tests (alias resolution, allow-list exhaustiveness, determinism, reference-graph, identifier safety); verify.sh passes (generated Swift compiles in SharedUI, app behaviour unchanged — nothing consumes the tokens yet).
• Planning docs: ALFMOB-264 epic plan + ALFMOB-272 red-team / validation / spike findings.

Screenshot

[Copilot]

Pull request overview

This PR introduces a standalone SwiftPM-based design-token generator (Tools/DesignTokenGen) that loads the W3C DTCG token export (manifest-driven iOS subset), validates/reshapes it (references, cycles, broken refs), and emits committed, deterministic, type-safe Swift into SharedUI/GeneratedTokens/. It also adds scripts and documentation to support a local “pull → generate → commit” workflow, while excluding the source JSON from compilation and excluding generated Swift from SwiftLint.

Changes:

• Add Tools/DesignTokenGen generator core (loader/resolver/emitter) plus a small CLI wrapper and unit tests/fixtures.
• Add local scripts (pull-design-tokens.sh, generate-design-tokens.sh) and docs describing the workflow and constraints.
• Commit the initial iOS-subset token JSON and the generated Swift output; wire SPM exclude and SwiftLint excluded.

Reviewed changes

Copilot reviewed 57 out of 57 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
Tools/DesignTokenGen/Tests/DesignTokenGenCoreTests/SwiftIdentifierTests.swift	Unit tests for deterministic Swift identifier mapping/collision detection
Tools/DesignTokenGen/Tests/DesignTokenGenCoreTests/GeneratorTests.swift	Unit tests for selection, parsing, resolution, emission contracts, determinism
Tools/DesignTokenGen/Tests/DesignTokenGenCoreTests/CoverageTests.swift	Edge-case tests for generator run/cleanup, resolution depth, parsing, color literal behavior
Tools/DesignTokenGen/Tests/DesignTokenGenCoreTests/Fixtures/mini/typography.styles.tokens.json	Minimal fixture for composite typography + doc token skipping
Tools/DesignTokenGen/Tests/DesignTokenGenCoreTests/Fixtures/mini/typography.alfie-theme.tokens.json	Minimal fixture for typography sub-token references/broken refs
Tools/DesignTokenGen/Tests/DesignTokenGenCoreTests/Fixtures/mini/theme.alfie-theme.tokens.json	Minimal fixture for semantic theme references to primitives
Tools/DesignTokenGen/Tests/DesignTokenGenCoreTests/Fixtures/mini/system.ios.tokens.json	Minimal fixture for iOS system-mode tokens
Tools/DesignTokenGen/Tests/DesignTokenGenCoreTests/Fixtures/mini/system.android.tokens.json	Minimal fixture for android system-mode tokens (ensures it’s skipped)
Tools/DesignTokenGen/Tests/DesignTokenGenCoreTests/Fixtures/mini/sizing.alfie-theme.tokens.json	Minimal fixture for sizing tokens (mix ref + literal)
Tools/DesignTokenGen/Tests/DesignTokenGenCoreTests/Fixtures/mini/screen-size.small-(s).tokens.json	Minimal fixture for selected screen-size mode
Tools/DesignTokenGen/Tests/DesignTokenGenCoreTests/Fixtures/mini/screen-size.large-(l).tokens.json	Minimal fixture for non-selected screen-size mode (ensures it’s skipped)
Tools/DesignTokenGen/Tests/DesignTokenGenCoreTests/Fixtures/mini/manifest.json	Minimal manifest fixture driving mode selection + styles inclusion
Tools/DesignTokenGen/Tests/DesignTokenGenCoreTests/Fixtures/mini/.primitives.alfie-theme.tokens.json	Minimal primitives fixture (colors/dimensions/font families)
Tools/DesignTokenGen/Tests/DesignTokenGenCoreTests/Fixtures/mini/.cycle-allowlist.json	Minimal cycle allowlist fixture
Tools/DesignTokenGen/Tests/DesignTokenGenCoreTests/Fixtures/mini/.broken-ref-allowlist.json	Minimal broken-ref allowlist fixture
Tools/DesignTokenGen/Sources/DesignTokenGenCore/TokenLoader.swift	Manifest-driven file selection + JSON parsing into a flat token graph
Tools/DesignTokenGen/Sources/DesignTokenGenCore/Token.swift	Token/value model + error types
Tools/DesignTokenGen/Sources/DesignTokenGenCore/SwiftIdentifier.swift	Token-name → Swift identifier conversion + collision assertions
Tools/DesignTokenGen/Sources/DesignTokenGenCore/Resolver.swift	Reference resolution with cycle/broken-ref allowlist enforcement and staleness checks
Tools/DesignTokenGen/Sources/DesignTokenGenCore/Generator.swift	Orchestration: load → validate → emit → clean/write
Tools/DesignTokenGen/Sources/DesignTokenGenCore/Emitter.swift	Deterministic Swift emission preserving the reference graph
Tools/DesignTokenGen/Sources/DesignTokenGenCore/Allowlists.swift	Load/represent cycle + broken-ref allowlists
Tools/DesignTokenGen/Sources/DesignTokenGen/main.swift	CLI wrapper for running the generator from scripts
Tools/DesignTokenGen/Package.swift	SwiftPM package definition for generator + tests/resources
Docs/DesignTokens.md	Developer documentation for pulling/generating/committing tokens
Alfie/scripts/pull-design-tokens.sh	Local script to pull iOS-subset token JSON + allowlists safely
Alfie/scripts/generate-design-tokens.sh	Local script that gates generator tests then regenerates committed Swift
Alfie/AlfieKit/Sources/SharedUI/DesignTokens/manifest.json	Committed DTCG manifest (iOS subset source)
Alfie/AlfieKit/Sources/SharedUI/DesignTokens/.primitives.alfie-theme.tokens.json	Committed DTCG primitives (iOS subset source)
Alfie/AlfieKit/Sources/SharedUI/DesignTokens/theme.alfie-theme.tokens.json	Committed DTCG semantic theme tokens (iOS subset source)
Alfie/AlfieKit/Sources/SharedUI/DesignTokens/sizing.alfie-theme.tokens.json	Committed DTCG sizing tokens (iOS subset source)
Alfie/AlfieKit/Sources/SharedUI/DesignTokens/typography.alfie-theme.tokens.json	Committed DTCG typography sub-tokens (iOS subset source)
Alfie/AlfieKit/Sources/SharedUI/DesignTokens/typography.styles.tokens.json	Committed DTCG composite typography styles (iOS subset source)
Alfie/AlfieKit/Sources/SharedUI/DesignTokens/system.ios.tokens.json	Committed DTCG system-mode iOS tokens (iOS subset source)
Alfie/AlfieKit/Sources/SharedUI/DesignTokens/screen-size.small-(s).tokens.json	Committed DTCG screen-size small mode tokens (iOS subset source)
Alfie/AlfieKit/Sources/SharedUI/DesignTokens/.cycle-allowlist.json	Committed allowlist for known exporter cycles
Alfie/AlfieKit/Sources/SharedUI/DesignTokens/.broken-ref-allowlist.json	Committed allowlist for pinned missing reference targets
Alfie/AlfieKit/Sources/SharedUI/GeneratedTokens/Primitives+Generated.swift	Generated Swift primitives namespace (committed output)
Alfie/AlfieKit/Sources/SharedUI/GeneratedTokens/Theme+Generated.swift	Generated semantic theme references to primitives (committed output)
Alfie/AlfieKit/Sources/SharedUI/GeneratedTokens/Sizing+Generated.swift	Generated sizing surface (committed output)
Alfie/AlfieKit/Sources/SharedUI/GeneratedTokens/Typography+Generated.swift	Generated composite typography surface (committed output)
Alfie/AlfieKit/Package.swift	Exclude DesignTokens/ from compilation/bundling
Alfie/.swiftlint.yml	Exclude SharedUI/GeneratedTokens from SwiftLint
AGENTS.md	Add SharedUI/GeneratedTokens/ to the “never edit generated files” list
Docs/Plans/260618-1552-ALFMOB-264-design-system-tokens/validation.md	Planning/validation documentation updates for epic execution decisions
Docs/Plans/260618-1552-ALFMOB-264-design-system-tokens/validation-272.md	Validation notes specific to ALFMOB-272 scope/process decisions
Docs/Plans/260618-1552-ALFMOB-264-design-system-tokens/spike-findings-272.md	Spike findings documenting the real token contract/export behavior
Docs/Plans/260618-1552-ALFMOB-264-design-system-tokens/scout.md	Scouting report/plan context for token integration work
Docs/Plans/260618-1552-ALFMOB-264-design-system-tokens/red-team.md	Red-team review notes for the epic plan
Docs/Plans/260618-1552-ALFMOB-264-design-system-tokens/red-team-272-pipeline.md	Red-team review notes specific to the token pipeline/generator
Docs/Plans/260618-1552-ALFMOB-264-design-system-tokens/plan.md	Updated plan document reflecting validated decisions and phased approach
Docs/Plans/260618-1552-ALFMOB-264-design-system-tokens/phase-1-token-pipeline.md	Phase 1 implementation plan (generator pipeline)
Docs/Plans/260618-1552-ALFMOB-264-design-system-tokens/phase-2-color.md	Phase 2 plan (color integration)
Docs/Plans/260618-1552-ALFMOB-264-design-system-tokens/phase-3-typography.md	Phase 3 plan (typography integration split)
Docs/Plans/260618-1552-ALFMOB-264-design-system-tokens/phase-4-spacing-shape.md	Phase 4 plan (spacing/shape integration)
Docs/Plans/260618-1552-ALFMOB-264-design-system-tokens/phase-5-component-refactors.md	Phase 5 plan (component refactors)
Docs/Plans/260618-1552-ALFMOB-264-design-system-tokens/phase-6-snapshot-baselines.md	Phase 6 plan (snapshot harness/baselines last)

[Copilot]

Pull request overview

Copilot reviewed 57 out of 57 changed files in this pull request and generated 2 comments.

[Copilot]

Pull request overview

Copilot reviewed 57 out of 57 changed files in this pull request and generated 3 comments.

[amccall-mindera]

Pipeline infrastructure is solid. All prior Copilot comments are addressed in the current code (alpha handling, string escaping, public TypographyStyle init, manifest fail-fast, stale allowlist detection, Swift concurrency keywords). Generated tokens compile, reference graph is preserved (no hardcoded hex in semantic layer), 33 unit tests pass including determinism and allowlist exhaustiveness. AC gap on protocol conformance is intentionally deferred per Docs/DesignTokens.md.