Skip to content

feat(llms): Include source code in mistica package#1531

Merged
Marcosld merged 4 commits into
masterfrom
WEB-2431
Apr 22, 2026
Merged

feat(llms): Include source code in mistica package#1531
Marcosld merged 4 commits into
masterfrom
WEB-2431

Conversation

@Marcosld
Copy link
Copy Markdown
Contributor

@Marcosld Marcosld commented Apr 16, 2026

Summary

Adds source code to Mistica published package. To make up the instructions for the LLMs we conducted a series of experiments that are detailed below. The conclusions were:

  1. Docs first — LLMs tend to work best with documentation and examples. Less tokens consumption, faster responses, etc.
  2. .d.ts for prop questions — Reading .d.ts files is 10-17% cheaper than reading .tsx source, and LLMs understand these types. Also JSDoc is preserved in .d.ts files and taken in account by LLMs.
  3. **.tsx / .css.ts source can help LLMs when asked specifically about library internals or debugging errors caused by Mistica lib itself. When source files are not present LLMs read the compiled files reaching essentially the same conclusions, but reading the compiled code is more expensive in tokens/time.
  4. UI shows no improvements when having the source code available.
  5. Source reading sometimes regresses output quality. Seeing low-level APIs (render props, internal escape hatches) can tempt the LLM to use them instead of simpler declarative alternatives.

Following these, we concluded adding the source code can be helpful, but we tailored the instructions so LLMs just read it when needed and don't default to it.


Iteration 1 — Beta v16.59.0-beta.1 (with source code) vs Stable v16.58.0 (without source code)

Setup: Four greenfield UI evals (baseline vs "imperative read source")
Beta has the "Source code location" section; stable v16.58.0's llms.md doesn't ship it.

Eval Beta tokens/duration Stable tokens/duration
Settings 55.0k / 234s 56.6k / 302s
Checkout 54.8k / 291s 58.0k / 341s
Dashboard 60.1k / 316s 56.5k / 260s
Netflix 79.2k / 420s ~same / 524s*

*Stable Netflix hit API error after completing — screenshot/code succeeded.

Settings page


Beta v16.59.0-beta.1

Stable v16.58.0

Stable used HeaderLayout + Header and Callout — more polished structure. Both used Row switch and RowList idiomatically.

Checkout form


Beta v16.59.0-beta.1

Stable v16.58.0

Stable added a Stepper component (Cart/Shipping/Payment/Confirmation) — a composite the beta version didn't reach for. Ignore the misalignment, the stable version didn't include last changes from: #1526

Dashboard


Beta v16.59.0-beta.1

Stable v16.58.0

Stable used GridLayout template="8+4" for side-by-side composition of data usage and connected devices. Beta used simpler vertical stack and MainNavigationBar.

Netflix page


Beta v16.59.0-beta.1

Stable v16.58.0

Both visually striking with custom dark skins. Roughly tied.

Eval Winner Why
Settings Stable (slight) Richer page structure with HeaderLayout and Callout
Checkout Stable (clear) Used Stepper composite that beta missed
Dashboard Tied Both missed some components
Netflix Tied Both striking with custom skins

Conclusion: Stable v16.58.0 (without source location section) produced consistently equal-or-better outputs. The beta's "Source code location" section likely distracted agents from documented composites.


Iteration 1 — Baseline vs "imperative read source" (same beta version)

Added a "Step 3" to llms.md forcing agents to read component source before coding.

Settings page


Baseline

Modified (imperative read source)

Checkout form


Baseline

Modified (imperative read source)

Dashboard


Baseline

Modified (imperative read source)

Netflix page


Baseline

Modified (imperative read source)
Eval Baseline Modified Winner
Settings Row switch, NegativeBox, StartIcon Legacy skin, standalone Switch Baseline
Checkout Wrong validate type Correct types, CC field names from source Modified
Dashboard Manual Grid; missed DataCard/Carousel Carousel + DataCard Modified
Netflix 3 card types, best footer Single card type, weaker skin Baseline

While the results are inconsistent, token usage and time consumption was much higher for the modified version. Avg tokens: Baseline 67.6k, Modified 74.5k (+10%). Mixed results — source reading helped discover composites but regressed idiomatic patterns.


Iteration 1b — Three-way: baseline vs imperative vs "why-driven" rationale

Settings page


Baseline

V1 (imperative)

V2 (why-driven)

Checkout form


Baseline

V1 (imperative)

V2 (why-driven)

Dashboard


Baseline

V1 (imperative)

V2 (why-driven)

Netflix page


Baseline

V1 (imperative)

V2 (why-driven)

Produced code has essentially the same quality.

Conclusion: Explaining "why" to read source did NOT consistently improve over imperative. V2 took 24% more tokens but split wins evenly. The why-preamble may have displaced attention from actually executing the source reading thoroughly (V2's Netflix skin was visibly more incomplete).


Iterations 2–3 — Image upload: with/without Source location section

Here we don't include screenshots because they were essentially the same.

Prompt: "Generate a wireframe with mistica where the user can attach images to upload..."

  • Iteration 2: Neither variant used Mistica's FileUpload component — both built custom drop zones. Source location hint didn't help with component discovery.
  • Iteration 3 (explicitly asked to use FileUpload): WITHOUT-source used the cleaner declarative API (renderButton/renderFiles/withDropZone). WITH-source reached for the low-level render prop.

Conclusion: Source access tempts agents toward verbose low-level APIs when simpler declarative alternatives exist.


Iteration 4 — beta (source code) vs v16.58.0 (no source code) + FileUpload


WITH source (beta v16.59.0-beta.1)

WITHOUT source (stable v16.58.0)
WITH source (beta) WITHOUT source (v16.58.0)
Tokens 77.3k 59.7k (−23%)
Duration 295s 213s (−28%)
Mistica-native components HeaderLayout, Grid, Image, Align, Circle Raw CSS grid, raw <img>
FileUpload API Low-level render prop Declarative props

Conclusion: Source section drove deeper reading (+30% tokens) and broader component use. The WITHOUT-source version was more cost-efficient and picked the simpler API.


Iteration 5 — Fix DataCard accessibility (segregateTouchableContent)

Prompt

Fix the accessibility of the DataCard in example-fix-data-card/card.tsx so that all sections within the card (headline, title, subtitle, description, slot) are read one by one by screen readers.

Code

<DataCard
      asset={
        <Circle backgroundColor={skinVars.colors.brandLow} size={40}>
          <IconShopRegular color={skinVars.colors.brand} />
        </Circle>
      }
      headline={<Tag type="promo">Headline</Tag>}
      title="Title"
      subtitle="Subtitle"
      description="Description"
      slot={<Placeholder />}
      onPress={() => {
        console.log("hola");
      }}
    />

Both found the same correct answer: segregateTouchableContent prop.

WITH source (beta) WITHOUT source (v16.58.0)
Tokens 58.5k 48.6k (−17%)
Discovery path Read card-internal.tsx Read card-internal.d.ts

Conclusion: For prop discovery, .d.ts files are sufficient. JSDoc comments preserved there. Same answer, 17% cheaper.


Iteration 6 — Card with custom "hola" aria-label

Prompt

Generate a DataCard whose elements (headline, title, subtitle, description, slot) are read one by one by screen readers, AND whose clickable/touchable accessibility label is the word 'hola' (instead of the title text).

Both found the same compound solution: segregateTouchableContent + touchableAriaLabel="hola".

WITH source WITHOUT source
Tokens 63.1k 54.0k (−14%)
Duration 268s 174s (−35%)

Iteration 7 — Debug "aria-label ignored, title announced"

Prompt

I have a DataCard at example-fix-data-card/card.tsx. I added aria-label='hola' because I want screen readers to announce the clickable card as 'hola'. I also added segregateTouchableContent so each section is read individually. But when I test it with my screen reader, it announces the title 'Title' as the button label instead of 'hola'. The aria-label seems to be ignored. Why?

Both found the root cause: aria-label labels the outer container, not the touchable button. Fix: touchableAriaLabel.

WITH source WITHOUT source
Tokens 51.2k 46.2k (−10%)
Explanation Cited card-internal.tsx line 1359 + named hasTouchableInContent variable Paraphrased JSDoc from .d.ts

WITH-source gave deeper mechanistic explanation, but same correct fix. Useful for learning, not required for the fix.


Iteration 8 — Debug "search field not expanding in MainNavigationBar"

Here we asked the llm to reason about a ticket Vivo opened recently. It would need to inspect the source code in order to know why components weren't working as expected.

Prompt

I have a page with a MainNavigationBar at example-fix-data-card/page-example.tsx. The right prop contains an with a and a (Avatar + 'Entrar' text). The right portion of the navigation bar is NOT being stretched to occupy the full width of the available area. The SearchField does not expand horizontally to fill space — it just renders at its natural minimum width. I want the SearchField to grow and fill the available horizontal space inside the navbar's right slot, while the 'Entrar' button stays at its natural size on the right edge. Why is this happening, and how do I fix it?

Code

<ResponsiveLayout>
      <MainNavigationBar
        withBorder={false}
        burgerMenuExtra={true}
        large
        wide
        logo={<Logo type="imagotype" size={48} />}
        sections={[
          {
            title: "Baixar o App Vivo",
            onPress: () => setIndex(0),
          },
          {
            title: "Produtos e Serviços",
            onPress: () => setIndex(1),
          },
          {
            title: "Ajuda",
            onPress: () => setIndex(2),
          },
          {
            title: "Por que Vivo",
            onPress: () => setIndex(3),
          },
          {
            title: "Melhores Ofertas",
            onPress: () => setIndex(4),
          },
        ]}
        selectedIndex={index}
        right={
          <Inline space="between" fullWidth>
            <SearchField name="search" label="Search" fullWidth />
            <NavigationBarAction onPress={() => {}} aria-label="Entrar">
              <Avatar size={32} />
              <Text wordBreak={false} hyphens="manual">
                Entrar
              </Text>
              
            </NavigationBarAction>
          </Inline>
        }
      />

      <Box paddingY={80}>
        <Slideshow
          autoplay
          withBullets
          items={Array.from({ length: 3 }, (_, idx) => (
            <CoverCard
              aspectRatio="16:9"
              slot={
                <Box paddingY={16} paddingBottom={48}>
                  <ButtonGroup
                    primaryButton={
                      <ButtonPrimary onPress={() => {}}>
                        Pós com Vivo Fibra e Wi-Fi 6
                      </ButtonPrimary>
                    }
                  />
                </Box>
              }
              pretitle="Pós + Fibra"
              title="Pós com Vivo Fibra e Wi-Fi 6"
              description="Planos a partir de R$ 150/mês"
              imageSrc="https://cdn-assets-eu.frontify.com/s3/frontify-enterprise-files-eu/eyJwYXRoIjoidGVsZWZvbmljYVwvZmlsZVwvdUNqUU42c0Y3ZW9Db1NvV1h4eHguanBnIn0:telefonica:sFFXnOJrAlsAdyFQRwphLVSonDf7X4lYfZzy8cfWwM0?width=2400"
            />
          ))}
        />
      </Box>
    </ResponsiveLayout>

Both found the same 4-layer root cause and fixed with flex: 1 wrapper.

WITH source WITHOUT source
Tokens 48.0k 58.2k (+21%)
Duration 214s 390s (+82%)
Tool uses 38 68 (+79%)

First iteration where WITHOUT-source was MORE expensive. Layout/CSS behavior requires understanding DOM structure — something .d.ts can't describe. The WITHOUT-source agent had to reverse-engineer from compiled .js.


Key findings

  • JSDoc in .d.ts is an underrated discovery channel — for prop questions, same answer at a fraction of the cost
  • Source code location > forcing to read source code > explaning why has to read source code
  • Source reading sometimes regresses greenfield output — low-level APIs (render props, manual wrappers) tempt agents away from simpler documented composites (probably because their flexibility).
  • Source cost premium is real but task-dependent — +10% to +30% tokens on prop tasks; pays for itself on layout/CSS/skin tasks
  • Reading source can sometimes help when creating a skin from zero.

Ship src/** in the npm tarball (excluding tests, stories, generated
files) and document its location in doc/llms.md so LLMs can read the
real component source as a complement to the docs and type defs.

Refs WEB-2431
@github-actions
Copy link
Copy Markdown

Size stats

master this branch diff
Total JS 16.1 MB 16.1 MB 0 B
JS without icons 2.01 MB 2.01 MB 0 B
Lib overhead 92.5 kB 92.5 kB 0 B
Lib overhead (gzip) 19.9 kB 19.9 kB 0 B

@github-actions
Copy link
Copy Markdown

github-actions Bot commented Apr 16, 2026

Accessibility report
✔️ No issues found

ℹ️ You can run this locally by executing yarn audit-accessibility.

@github-actions
Copy link
Copy Markdown

github-actions Bot commented Apr 16, 2026

Screenshot tests report

✔️ All passing

Replace the broad "Source code location" section with a concise
"Source code" section that tells LLMs where source lives and when
to read it (debugging layout/CSS, event handling, skin authoring)
— and when not to (greenfield UI work).

Research across 9 A/B iterations showed that verbose source-reading
instructions pull agents toward low-level escape hatches when
documented composite components would produce simpler code.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@Marcosld Marcosld changed the title docs(llms): task-typed lookup guidance for LLM instructions docs(llms): Include source code in mistica package Apr 16, 2026
@github-actions
Copy link
Copy Markdown

Deploy preview for mistica-web ready!

Project:mistica-web
Status: ✅  Deploy successful!
Preview URL:https://mistica-ikgz8u47q-flows-projects-65bb050e.vercel.app
Latest Commit:70a47d0
Inspect:View deployment

Deployed with vercel-action

@Marcosld Marcosld marked this pull request as ready for review April 17, 2026 09:09
@Marcosld Marcosld changed the title docs(llms): Include source code in mistica package feat(llms): Include source code in mistica package Apr 17, 2026
@Marcosld Marcosld added AI AI Generated and removed AI AI Generated labels Apr 17, 2026
Comment thread doc/llms.md
If you cannot find a documentation file in `node_modules`, fetch the equivalent file from the GitHub
repository at `https://github.com/Telefonica/mistica-web/blob/master/doc/<filename>`.

## Source code
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we add instructions to avoid the agent to change the source code in node_modules?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It did not every try to modify source code in my tests. I think it knows this source code shouldn't be modified implicitly

Comment thread package.json
"doc/**",
"src/**",
"!src/generated/**",
"!src/**/__tests__/**",
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some tests may be relevant to understand how to use a component

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm...
Actually we found there are very few cases where reading the source code helps the llm answers prompts. I am not so sure having the tests too would add any additional context in those cases

@Marcosld Marcosld requested a review from atabel April 22, 2026 13:53
@Marcosld Marcosld added this pull request to the merge queue Apr 22, 2026
Merged via the queue into master with commit bf6280b Apr 22, 2026
11 of 12 checks passed
@tuentisre
Copy link
Copy Markdown
Collaborator

🎉 This PR is included in version 16.61.0 🎉

The release is available on:

Your semantic-release bot 📦🚀

yceballost pushed a commit to MurilloLeoni/mistica-web that referenced this pull request Apr 27, 2026
# [16.61.0](Telefonica/mistica-web@v16.60.0...v16.61.0) (2026-04-24)

### Bug Fixes

* **Image:** handle SSR image already complete before inline script runs ([Telefonica#1536](Telefonica#1536)) ([ec1e35c](Telefonica@ec1e35c))

### Features

* **chore:** Minimal github conventions ([Telefonica#1530](Telefonica#1530)) ([f7ca38a](Telefonica@f7ca38a))
* **fixedFooterLayout:** include footerScrollEffect prop ([Telefonica#1532](Telefonica#1532)) ([bd56fb2](Telefonica@bd56fb2))
* **llms:** Include source code in mistica package ([Telefonica#1531](Telefonica#1531)) ([bf6280b](Telefonica@bf6280b))
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants