Merge pull request #562 from aziontech/docs/theme-tokens-readme

docs(theme): rewrite tokens README to match build-tokens pipeline

Author: isaquebock

packages/theme/src/tokens/README.md

@@ -1,13 +1,18 @@
 <h1 align="center">azion-theme · tokens</h1>
 
-Design tokens organized as JavaScript modules, ready to be compiled into CSS custom properties. Architecture targets **Tailwind CSS v4** (CSS-first `@theme` / single stylesheet) — tokens live in JS as the source-of-truth and are compiled to CSS variables at build time.
+Design tokens organized as JavaScript modules, compiled into CSS custom properties and a Tailwind config. The pipeline emits **two parallel targets** from the same source:
+
+- **`dist/v3/`** — Tailwind v3 preset (`tailwind-preset.js` / `tailwind.config.js`) + a self-contained `globals.css` with `@tailwind` directives, `:root` / `[data-theme=dark]` blocks, responsive `@media` overrides, and the component classes.
+- **`dist/v4/`** — Tailwind v4 CSS-first output: `globals.css` with `@import "tailwindcss"` + `@theme { … }` + plain `:root` for shape/typography semantics + `@layer components`.
+
+Tokens live in JS as the source-of-truth and are compiled at package build time, not in the browser.
 
 ## 📋 Table of contents
 
 - [Architecture overview](#-architecture-overview)
 - [File structure](#-file-structure)
 - [Token types](#-token-types)
-- [Compile & inject](#-compile--inject)
+- [Building & consuming](#-building--consuming)
 - [How to add a new token](#-how-to-add-a-new-token)
 - [Token references (`tokenRef`)](#-token-references-tokenref)
 - [Test pages](#-test-pages)
@@ -17,23 +22,31 @@ Design tokens organized as JavaScript modules, ready to be compiled into CSS cus
 
 ## 🏗 Architecture overview
 
-Two layers:
+Three layers:
 
-1. **Primitives** — raw, theme-invariant values (color hexes, px sizes, font sizes, etc.).
-2. **Theme (semantic)** — light/dark variants that **reference** primitives via `tokenRef(...)`.
+1. **Primitives** — raw, theme-invariant values (color hexes, px sizes, font sizes, durations).
+2. **Semantic / theme** — light/dark variants and responsive (per-breakpoint) tokens that **reference** primitives via `tokenRef(...)`.
+3. **Output bundles** — `dist/v3/` and `dist/v4/` artifacts emitted by `scripts/build-tokens.mjs`.
 
 Compile pipeline:
 
 ```
-tokens/primitives/*.js  ─┐
-                         ├─►  scripts/compile-primitives.js  ─►  :root { --color-… --size-… }
-tokens/theme/*.js       ─┘                                       (single block — values don't change)
-                         └─►  scripts/compile-theme.js       ─►  :root, [data-theme=light] { … }
-                                                                 [data-theme=dark]         { … }
+tokens/primitives/**          ─┐
+                               │
+tokens/semantic/colors.js     ─┤
+tokens/semantic/*.data.js     ─┼──►  scripts/build-tokens.mjs  ──►  dist/v3/{globals.css,tailwind-preset.js,tailwind.config.js}
+tokens/theme/**               ─┤                                    dist/v4/globals.css
+                               │
+scripts/{compile-primitives,  ─┘
+        compile-theme,
+        resolve,
+        css-vars,
+        refs}.js  (building blocks used by build-tokens.mjs)
 ```
 
-- Primitives are **absolute** and emit one block (`:root, [data-theme=light]`).
-- Theme tokens have **light** and **dark** variants and emit two blocks. Only theme vars flip between themes.
+- Primitives emit a single block (values don't change between themes).
+- Semantic colors flip between `:root, [data-theme=light]` and `[data-theme=dark], .dark, .azion.azion-dark`.
+- Responsive semantic tokens (containers, spacings, texts) emit `@media (min-width: …)` overrides.
 
 ---
 
@@ -44,52 +57,75 @@ src/
 ├── tokens/
 │   ├── primitives/
 │   │   ├── colors/
-│   │   │   ├── colors.js        # base, gray, violet, orange, slate, yellow,
-│   │   │   │                    # green, blue, neutral, red, surface (+ brand, alpha)
-│   │   │   ├── brand.js         # primary, accent, absolute (brand colors)
+│   │   │   ├── colors.js        # base, blue, gray, violet, orange, slate, yellow,
+│   │   │   │                    # green, red + surface palettes + brand re-export
+│   │   │   ├── brand.js         # primary, accent, surfaces, absolute
 │   │   │   └── alpha.js         # alpha variants for each palette
 │   │   ├── shape/
+│   │   │   ├── aspect-video.js
 │   │   │   ├── container.js     # container-3xs … container-7xl
 │   │   │   ├── height.js        # h-2 … h-96
-│   │   │   ├── radius.js        # none, sm, DEFAULT, md … 3xl, full
+│   │   │   ├── radius.js        # none, sm, DEFAULT, md, lg, xl, 2xl, 3xl, full
+│   │   │   ├── shape.js         # shape aliases (`max-w-*`, etc.)
 │   │   │   ├── size.js          # size-2 … size-96
 │   │   │   ├── spacing.js       # spacing-1 … spacing-96
 │   │   │   └── width.js         # w-3xs … w-7xl (alias → container.X)
 │   │   ├── typography/
-│   │   │   ├── font-family.js   # sans, code, display
+│   │   │   ├── font-family.js   # sans, mono, code
 │   │   │   ├── font-size.js     # text-xs … text-9xl
-│   │   │   └── line-height.js   # leading-none, leading-3 … leading-10
+│   │   │   ├── font-weight.js   # 100 … 900
+│   │   │   ├── leading.js       # leading-3 … leading-10
+│   │   │   ├── line-height.js   # line-height-none … line-height-loose
+│   │   │   └── tracking.js      # tracking-tighter … tracking-widest
 │   │   ├── effects/
 │   │   │   ├── blur.js          # blur-xs … blur-3xl
-│   │   │   └── opacity.js       # opacity-25/50/75/100
+│   │   │   ├── drop-shadow.js
+│   │   │   ├── inset-shadow.js
+│   │   │   ├── opacity.js       # opacity-0 … opacity-100
+│   │   │   ├── perspective.js
+│   │   │   └── shadow.js        # box-shadow scale
+│   │   ├── animations/
+│   │   │   ├── animate.js       # named keyframe animations
+│   │   │   └── ease.js          # easing curves
 │   │   ├── border-widths.js     # border-0 … border-4
 │   │   ├── breakpoints.js       # sm, md, lg, xl, 2xl
-│   │   └── ring-offset.js       # ring-offset
+│   │   └── ring-offset.js
 │   ├── theme/
-│   │   ├── primary.js           # primary, primary-mask/selected/hover/active/contrast
-│   │   ├── secondary.js         # secondary, secondary-*
-│   │   ├── accent.js            # accent, accent-*
-│   │   ├── surfaces.js          # surface-0 … surface-950 (aliases for gray)
+│   │   ├── primary.js           # primary, primary-mask/hover/contrast
+│   │   ├── secondary.js
+│   │   ├── accent.js
+│   │   ├── surfaces.js          # surface-0 … surface-950 (alias for gray)
 │   │   ├── background.js        # bg-canvas, bg-surface, bg-mask, …
-│   │   ├── border.js            # border-default, border-muted, border-strong, border-selected
+│   │   ├── border.js            # border-default/muted/strong/selected
 │   │   ├── text.js              # text-default, text-muted
 │   │   ├── ring.js              # ring-color
 │   │   └── feedback/
-│   │       ├── success.js       # success, success-border, success-contrast
+│   │       ├── success.js
 │   │       ├── warning.js
 │   │       ├── danger.js
 │   │       └── info.js
-│   └── semantic/                # legacy semantic colors (pre-v4 pipeline)
-│       └── colors.js
+│   ├── semantic/
+│   │   ├── colors.js            # text/background/border semantic refs (consumed by v3 preset)
+│   │   ├── containers.js        # static + responsive container tokens
+│   │   ├── containers.data.js   # `{ key: { sm, md, lg, … } }` data table
+│   │   ├── spacings.js
+│   │   ├── spacings.data.js
+│   │   ├── texts.js
+│   │   ├── texts.data.js        # font-size + line-height + letter-spacing bundles
+│   │   └── animations.js
+│   ├── theme.js                 # Tailwind theme.extend (colors + semantic mappings)
+│   └── index.js                 # public re-exports
 ├── scripts/
-│   ├── refs.js                  # tokenRef helper
-│   ├── resolve.js               # legacy resolver (semantic/colors.js)
-│   ├── css-vars.js              # legacy CSS-vars compiler
-│   ├── compile-primitives.js    # NEW: flattens primitives → :root
-│   └── compile-theme.js         # NEW: resolves theme refs → :root + [data-theme=dark]
+│   ├── refs.js                  # tokenRef helper + isTokenRef guard
+│   ├── resolve.js               # resolves `tokenRef` paths to literal values
+│   ├── css-vars.js              # builds light/dark CSS var maps from semantic refs
+│   ├── compile-primitives.js    # flattens primitive trees into CSS vars
+│   ├── compile-theme.js         # legacy theme compiler (used by the harness pages)
+│   └── build-tokens.mjs         # main entrypoint: emits dist/v3 and dist/v4 bundles
 └── tests/
     ├── primitives.html          # visual harness for all primitives
-    └── theme.html               # visual harness with light/dark toggle
+    ├── theme.html               # semantic tokens with light/dark toggle
+    └── tokens.html              # combined view
 ```
 
 ---
@@ -102,7 +138,7 @@ Plain JS objects with literal values. Theme-invariant.
 
 ```js
 // tokens/primitives/colors/brand.js
-export const brand = {
+export const brandPrimitives = {
   primary: { 50: '#FFF5EF', 100: '#FFE7D8', /* … */ 500: '#FE601F', /* … */ 950: '#401602' },
   accent:  { 50: '#F6F6FF', /* … */ 500: '#8A84EC', /* … */ 950: '#0B0A19' },
   absolute: { black: '#0A0A0A', white: '#FCFCFC' },
@@ -114,9 +150,9 @@ export const brand = {
 export const spacing = { 1: '4px', 2: '8px', /* … */ 96: '384px' };
 ```
 
-### Theme (semantic)
+### Semantic / theme
 
-Objects with `light` / `dark` variants. Values are `tokenRef(...)` calls that point to primitives (or to other semantic tokens like `theme.surfaces.surface-X`).
+Objects with `light` / `dark` variants. Values are `tokenRef(...)` calls that point to primitives.
 
 ```js
 // tokens/theme/primary.js
@@ -138,53 +174,89 @@ export const primary = {
 };
 ```
 
+### Responsive semantic data (`*.data.js`)
+
+Containers, spacings, and texts use a per-breakpoint table that `build-tokens.mjs` flattens into `@media` overrides plus matching component classes (`.gap-…`, `.p-…`, `.text-…`, `.px-container`, `.py-container`, `.max-container-width`).
+
+```js
+// tokens/semantic/spacings.data.js
+export const spacingsData = {
+  'gap-sm': { _: '8px', md: '12px', xl: '16px' },
+  'gap-md': { _: '12px', md: '16px', xl: '24px' },
+  // …
+};
+```
+
+`_` is the base value emitted in `:root`; the breakpoint keys (`sm`, `md`, `lg`, `xl`, `2xl`) become media-query overrides.
+
 ---
 
-## 🔧 Compile & inject
+## 🔧 Building & consuming
+
+### Building the package
+
+```bash
+# emit both v3 and v4 bundles
+pnpm --filter @aziontech/theme build:tokens
+
+# only one target
+pnpm --filter @aziontech/theme build:tokens:v3
+pnpm --filter @aziontech/theme build:tokens:v4
+
+# v3 + compile final CSS with tailwindcss
+pnpm --filter @aziontech/theme build:css:v3
+```
+
+Outputs land in `packages/theme/dist/v3/` and `packages/theme/dist/v4/`.
 
-### In the browser
+### Consuming in a Tailwind v3 project
 
 ```js
-import { injectPrimitivesCss } from '@aziontech/theme/scripts/compile-primitives.js';
-import { injectThemeCss }      from '@aziontech/theme/scripts/compile-theme.js';
+// tailwind.config.js
+import themePreset from '@aziontech/theme/tailwind-preset/v3'
 
-injectPrimitivesCss();   // <style data-azion-primitives> with all primitive vars
-injectThemeCss();        // <style data-azion-theme>      with light+dark theme vars
+export default {
+  presets: [themePreset],
+  content: ['./src/**/*.{vue,js,ts,jsx,tsx}'],
+  darkMode: ['class'],
+}
+```
+
+```css
+/* main.css */
+@import '@aziontech/theme/v3/globals.css';
 ```
 
-Then use the variables anywhere:
+Utilities like `text-default`, `bg-surface`, `border-default`, `text-primary`, `gap-md`, `p-md`, `.text-heading-lg`, etc. are then available everywhere.
+
+### Consuming in a Tailwind v4 project
 
 ```css
-.btn {
-  background: var(--primary);
-  color: var(--primary-contrast);
-  padding: var(--spacing-2) var(--spacing-4);
-  border-radius: var(--radius-md);
-  font-size: var(--font-size-sm);
-}
-.btn:hover { background: var(--primary-hover); }
+/* main.css */
+@import '@aziontech/theme/v4/globals.css';
 ```
 
-### As CSS strings (Node / SSR / build step)
+No `tailwind.config.js` needed — v4 reads `@theme { … }` directly from the imported CSS.
+
+### As JS objects (Node / build steps)
 
 ```js
-import { compilePrimitivesCss } from '@aziontech/theme/scripts/compile-primitives.js';
-import { compileThemeCss }      from '@aziontech/theme/scripts/compile-theme.js';
+import { createCssVars, cssVarsString } from '@aziontech/theme';
 
-const css = compilePrimitivesCss() + '\n' + compileThemeCss();
-fs.writeFileSync('dist/theme.css', css);
+createCssVars();   // → { light: { '--text-default': '#1A1A1A', … }, dark: { … } }
+cssVarsString();   // → ':root, [data-theme=light], .azion.azion-light { … } [data-theme=dark], … { … }'
 ```
 
-### As JS objects
+### Injecting at runtime (browser)
 
 ```js
-import { compilePrimitivesVars } from '@aziontech/theme/scripts/compile-primitives.js';
-import { compileThemeVars }      from '@aziontech/theme/scripts/compile-theme.js';
+import { injectCssVars } from '@aziontech/theme';
 
-compilePrimitivesVars();  // → { '--color-orange-500': '#FE601F', '--spacing-1': '4px', … }
-compileThemeVars();       // → { light: {…}, dark: {…} }
+injectCssVars(); // appends a <style data-azion-tokens> element to <head>
 ```
 
+Only useful if you can't precompile and import `globals.css` — production usage should prefer the static stylesheet.
+
 ---
 
 ## ➕ How to add a new token
@@ -201,7 +273,7 @@ compileThemeVars();       // → { light: {…}, dark: {…} }
    };
    ```
 
-3. Done. It is automatically picked up by `compile-primitives.js`, producing `--spacing-128: 512px`.
+3. Rebuild (`pnpm --filter @aziontech/theme build:tokens`). The compiler picks it up and emits `--spacing-128: 512px`.
 
 ### Add a new semantic (theme) token
 
@@ -222,34 +294,36 @@ compileThemeVars();       // → { light: {…}, dark: {…} }
    };
    ```
 
-3. The compile script picks it up automatically — emits `--border-emphasis` inside both blocks.
+3. Rebuild. The new variable lands in both `:root` and `[data-theme=dark]` blocks.
 
-### Add a new semantic group (new file)
+### Add a new responsive semantic token
 
-1. Create `tokens/theme/<group>.js` exporting `{ light, dark }`.
-2. Register it in `scripts/compile-theme.js`:
+1. Add an entry to the relevant `*.data.js` file in `tokens/semantic/`:
 
    ```js
-   import { yourGroup } from '../tokens/theme/your-group.js';
-   /* … inside compileVariant(variant): */
-   const groups = [ /* …, */ yourGroup[variant] ];
+   // tokens/semantic/spacings.data.js
+   export const spacingsData = {
+     /* … */
+     'gap-2xl': { _: '32px', md: '40px', xl: '56px' },
+   };
    ```
 
+2. Rebuild. `build-tokens.mjs` emits the base var in `:root`, per-breakpoint `@media` overrides, **and** a matching `.gap-2xl { gap: var(--gap-2xl) }` utility.
+
 ---
 
 ## 🔗 Token references (`tokenRef`)
 
-`tokenRef(path)` returns a marker object `{ __ref: path }` that the compiler resolves at build time.
-
-Supported path prefixes (handled by `scripts/compile-theme.js`):
+`tokenRef(path)` returns a marker object `{ __ref: path }` that the compiler resolves at build time. Supported path prefixes (see `scripts/resolve.js`):
 
 | Prefix | Looks up |
 |---|---|
 | `primitives.X.Y.Z` | `tokens/primitives/colors/colors.js` tree (e.g., `primitives.gray.900`, `primitives.alpha.orange.100`) |
-| `brand.primary.primary-N` | `brand.primary[N]` (e.g., `brand.primary.primary-500` → `#FE601F`) |
-| `brand.accent.accent-N` | `brand.accent[N]` |
-| `brand.absolute.X` | `brand.absolute.X` (`black` / `white`) |
-| `theme.surfaces.surface-N` | the semantic surface map (chain: surface → gray primitive) |
+| `surfacePrimitives.surface.N` | the surface palette (used internally; usually referenced via `brand.surfaces.…`) |
+| `brand.surfaces.surface-N` | `surfacePrimitives.surface[N]` (e.g., `brand.surfaces.surface-100` → `#F5F5F5`) |
+| `brand.primary.primary-N` | `brandPrimitives.primary[N]` (e.g., `brand.primary.primary-500` → `#FE601F`) |
+| `brand.accent.accent-N` | `brandPrimitives.accent[N]` |
+| `brand.absolute.X` | `brandPrimitives.absolute.X` (`black` / `white`) |
 
 Refs of unknown prefixes are left as the raw path string in the output — flag for "this needs resolver support".
 
@@ -263,8 +337,9 @@ Local visual harnesses (require a static server because of ESM imports):
 npx http-server packages/theme/src -p 8080
 ```
 
-- `http://localhost:8080/tests/primitives.html` — all primitives rendered as swatches/scales (396 vars).
-- `http://localhost:8080/tests/theme.html` — semantic tokens with a **light/dark toggle button** (59 + 59 vars). Body, buttons, alerts, surfaces, and swatches all react to the toggle.
+- `http://localhost:8080/tests/primitives.html` — every primitive rendered as a swatch / scale.
+- `http://localhost:8080/tests/theme.html` — semantic tokens with a **light/dark toggle**.
+- `http://localhost:8080/tests/tokens.html` — combined view.
 
 ---
 
@@ -285,16 +360,17 @@ The compiled CSS targets multiple hooks so the runtime can pick whichever conven
 Switching is a one-liner:
 
 ```js
+// data-attribute strategy
 document.documentElement.setAttribute('data-theme', 'dark'); // or 'light'
-```
-
-Or, if you prefer Tailwind's `dark` class strategy:
 
-```js
+// Tailwind class strategy
 document.documentElement.classList.toggle('dark');
+
+// Azion namespaced classes (use one or the other)
+document.documentElement.classList.add('azion', 'azion-dark');
 ```
 
-Both work because the selectors cover both conventions.
+All three strategies hit the same set of CSS variables.
 
 ---