Skip to content

Commit a602294

Browse files
Update annotation/tag docs for route formalization
The annotation/tag documentation was teaching patterns that don't match what Malloy does anymore. The compiler now parses annotation prefixes (the bracket form `#(myApp)`, malformed-route and reserved-route warnings, Python-style block dedent) and the read API moved from `tagParse({prefix: /regex/})` to a route-aware view — shipped in malloydata/malloy#2830, now on npm. - documentation/language/tags.malloynb restructured around prefix and route. Block dedent paragraph rewritten (common body prefix, not opener column; tabs no longer "not allowed"); `#"` documented as the live doc-string route; `#bar_chart` documented as the malformed-route warning it now is; small "bring your own payload" section for non-tag payloads. - documentation/experiments/givens.malloynb: Given API table now shows `annotations: Annotations`. - blog/2025-06-16-annotations-and-tags: canonical myTagsAndDocs example migrated to `annotations.parseAsTag('myTags')` / `annotations.texts('myDocs')`; Future-of-Annotation section gets a 2026-05 update note. - scripts/run_code.ts migrated to route strings; one synthetic-Note call site keeps annotationToTag (no inherits chain — notebook cell isolation). - scripts/render_document.ts: ModelDef literal needed sourceRegistry (pre-existing requirement from a separate recent malloy change). - package.json bumped via npm run malloy-update.
1 parent dcdbff2 commit a602294

7 files changed

Lines changed: 198 additions & 193 deletions

File tree

package-lock.json

Lines changed: 90 additions & 112 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

package.json

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -31,12 +31,12 @@
3131
},
3232
"homepage": "https://github.com/malloydata/malloydata.github.io#readme",
3333
"devDependencies": {
34-
"@malloydata/db-duckdb": "0.0.359",
35-
"@malloydata/malloy": "0.0.359",
36-
"@malloydata/malloy-sql": "0.0.359",
37-
"@malloydata/render": "0.0.359",
38-
"@malloydata/render-validator": "0.0.359",
39-
"@malloydata/syntax-highlight": "0.0.359",
34+
"@malloydata/db-duckdb": "0.0.397",
35+
"@malloydata/malloy": "0.0.397",
36+
"@malloydata/malloy-sql": "0.0.397",
37+
"@malloydata/render": "0.0.397",
38+
"@malloydata/render-validator": "0.0.397",
39+
"@malloydata/syntax-highlight": "0.0.397",
4040
"@types/uglify-js": "^3.17.5",
4141
"concurrently": "^6.2.1",
4242
"fs-extra": "^10.1.0",

scripts/render_document.ts

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -84,6 +84,7 @@ class Renderer {
8484
contents: {},
8585
queryList: [],
8686
dependencies: {},
87+
sourceRegistry: {},
8788
};
8889

8990
constructor(path: string) {

scripts/run_code.ts

Lines changed: 7 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -264,8 +264,6 @@ async function renderResult(
264264
</div>`;
265265
}
266266

267-
const DOCS_M_TAG_PREFIX = /##\(docs\)\s/;
268-
const DOCS_Q_TAG_PREFIX = /#\(docs\)\s/;
269267

270268
export async function runNotebookCode(
271269
code: string,
@@ -292,14 +290,13 @@ export async function runNotebookCode(
292290
._loadModelFromModelDef(modelDef)
293291
.extendModel(fakeURL);
294292
const model = await newModel.getModel();
295-
// TODO this is a quick hack to make each snippet only use its own
296-
// tags and not those from other cells, unsure if this is the right approach
297-
// long term, but it prevents `##(docs) hidden` from affecting subsequent cells
293+
// Synthetic Annotation with `notes` only (no `inherits`): each notebook
294+
// cell sees only its own model annotations, not `##(docs) hidden` from
295+
// prior cells. `annotationToTag` takes the synthetic value directly;
296+
// `.annotations.parseAsTag('docs')` on the model would walk inherits.
298297
const modelTagParse = annotationToTag(
299-
{
300-
notes: model._modelDef?.annotation?.notes,
301-
},
302-
{ prefix: DOCS_M_TAG_PREFIX }
298+
{ notes: model._modelDef?.annotation?.notes },
299+
'docs'
303300
);
304301
const modelTags = modelTagParse.tag;
305302
const newModelDef = model._modelDef;
@@ -314,7 +311,7 @@ export async function runNotebookCode(
314311
if (hasQuery) {
315312
const runnable = newModel.loadFinalQuery();
316313
const query = await runnable.getPreparedQuery();
317-
const tags = query.tagParse({ prefix: DOCS_Q_TAG_PREFIX }).tag;
314+
const tags = query.annotations.parseAsTag('docs').tag;
318315
options.pageSize = tags.numeric("limit");
319316
options.size = tags.text("size");
320317
options.showAs = tags.has("html")

src/blog/2025-06-16-annotations-and-tags/index.malloynb

Lines changed: 11 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -155,14 +155,14 @@ on an object from the Malloy API which has annotations.
155155

156156
```typescript
157157
function myTagsAndDocs(t: Taggable): { tags: Tag, docs: string[] } {
158-
const tagParse = t.tagParse({prefix: /^#\(myTags\)/});
159-
if (tagParse.log) {
158+
const tagParse = t.annotations.parseAsTag('myTags');
159+
if (tagParse.log.length > 0) {
160160
// deal with syntax errors, or ignore them
161161
throw new Error('syntax errors in tag parse');
162162
}
163163
return {
164164
tags: tagParse.tag,
165-
docs: t.getTaglines(/^#\(myDocs\)/),
165+
docs: t.annotations.texts('myDocs'),
166166
};
167167
}
168168

@@ -185,4 +185,11 @@ are ...
185185
or maybe some sort of "prefix registry" to make certain applications do not
186186
inadvertantly affect metadata from other applications.
187187
* More formality on which annotations are tags, and what the "schema" is for a tagged
188-
annotation, allowing the IDE to assist in tagging complex objects.
188+
annotation, allowing the IDE to assist in tagging complex objects.
189+
190+
**Update, 2026-05:** The first two of these shipped. Multi-line block annotations
191+
(`#|...|#`) and a formal prefix grammar (the bracketed `#(myApp)` form, plus
192+
warnings for malformed prefixes) are now part of Malloy. The reading API also
193+
moved from `tagParse({prefix: RegExp})` to a route-based view — see the
194+
current [Annotation documentation](/documentation/language/tags) for the model
195+
as it stands. The schema-validation / IDE-assistance idea is still open.

src/documentation/experiments/givens.malloynb

Lines changed: 2 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -364,12 +364,11 @@ class Given {
364364
readonly default: ConstantExpr | undefined; // undefined ⇒ caller must supply
365365
get location(): DocumentLocation | undefined;
366366

367-
tagParse(spec?: TagParseSpec): MalloyTagParse;
368-
getTaglines(prefix?: RegExp): string[];
367+
readonly annotations: Annotations;
369368
}
370369
```
371370

372-
Annotations on the declaration are accessible via `tagParse` / `getTaglines`, the same path used elsewhere in the foundation API.
371+
Annotations on the declaration are accessible via the `annotations` view — the same route-aware API used everywhere else in the foundation. See the [Annotation documentation](../language/tags.malloynb) for the prefix/route model and reader API.
373372

374373
## Givens vs. source parameters
375374

Lines changed: 81 additions & 58 deletions
Original file line numberDiff line numberDiff line change
@@ -1,25 +1,20 @@
11
>>>markdown
22
# Annotation
33

4-
Annotations are text strings collected with objects as a file is compiled. Annotations are intended to be a generally useful method for connecting meta-data about a model with the source code for a model.
4+
Annotations are text strings attached to objects when a Malloy file compiles. They carry metadata — display hints for the renderer, compiler flags, documentation strings, application-specific configuration. The compiler stores annotations verbatim and hands them back to whichever application asks; it does not interpret them.
55

6-
Annotations are strings of text in the model beginning with either `#` or `##` and immediately followed by some character(s) which define the "space" of the annotation.
7-
8-
Annotations which start `##` are collected with the file or "model" and annotations which only have one `#` are associated with the object being defined. An annotation starts at the `#` character and continues to the end of the line.
6+
Annotations start with `#` (attached to the object declared below) or `##` (attached to the entire model) and run to end-of-line:
97

108
```malloy
11-
// This is an annotation that applies to the entire model. The `!` prefix means it is a compiler annotation.
129
##! experimental.parameters
1310

14-
// This is an annotation on a single view.
15-
// The ` ` (space) prefix means it is a renderer annotation
1611
# bar_chart
1712
view: how_many is things -> { aggregate: total_count is count() }
1813
```
1914

20-
## Block Annotations
15+
## Block annotations
2116

22-
For longer annotations that span multiple lines, use block annotation syntax. Object block annotations start with `#|` and end with `|#`. Model block annotations start with `##|` and end with `|##`.
17+
For annotations that span multiple lines, use the block form. Object block annotations are bracketed by `#|` and `|#`; model block annotations by `##|` and `|##`:
2318

2419
```malloy
2520
source: flights is duckdb.table('flights.parquet') extend {
@@ -32,24 +27,24 @@ source: flights is duckdb.table('flights.parquet') extend {
3227
}
3328
```
3429

35-
The closing `|#` (or `|##`) must be at the same indentation level as the opening `#|` (or `##|`). Indentation is automatically stripped based on the column position of the opener — so the content above is equivalent to writing three separate annotations `# bar_chart`, `# size=xl`, and `# color=blue`. Tabs in the indentation area are not allowed — use spaces.
30+
The closing `|#` (or `|##`) must be at the same indentation level as the opener. Body lines are then dedented by the longest leading-whitespace prefix they share — so the block above is equivalent to writing three separate annotations `# bar_chart`, `# size=xl`, `# color=blue`. Pasting flush-left content works too: a body line with no leading whitespace forces the common prefix to zero and nothing gets stripped.
3631

37-
Block annotations can also include a routing prefix, just like single-line annotations:
32+
Block annotations can carry a route just like single-line ones:
3833

3934
```malloy
4035
#|(docs)
41-
This is a multi-line documentation annotation.
42-
It can contain as much text as needed.
36+
This is multi-line documentation.
37+
It can be as long as it needs to be.
4338
|#
4439
source: my_source is duckdb.table('my_table')
4540
```
4641

47-
## Annotation Distribution
42+
## Annotation distribution
4843

49-
An object annotation which happens before a definition list is distributed to each member of the list, but each member can also have their own unique annotations
44+
An annotation that appears before a definition list is distributed to each member of the list. Each member can also carry its own annotations:
5045

5146
```malloy
52-
// Every measure in the list will render as a currency, but only 'pct' will render as a percent
47+
// Every measure renders as currency, but only `pct` also renders as percent
5348
# currency
5449
measure:
5550
max_x is max(x)
@@ -58,58 +53,86 @@ measure:
5853
min_x is min(x)
5954
```
6055

61-
# Tags
56+
## Prefix and route
6257

63-
The primary use of annotations in Malloy is for tags, which are simply a subset of the annotations strings which will be interpreted in simple programming language for setting key/value properties.
58+
Everything from the marker (`#`/`##`/`#|`/`##|`) up to the first whitespace is the annotation's **prefix**. The prefix resolves to a **route** — a namespace key. The compiler validates the *shape* of the prefix and routes the annotation; it never parses what comes after.
6459

65-
Tags are a general-purpose feature of Malloy that allow arbitrary metadata to be attached to various Malloy objects (queries, sources, fields, etc.). These are used by the Malloy rendering library within VSCode to decide how fields and queries should be rendered, but they can be parsed and interpreted differently in other applications.
60+
`#(docs) Hello` is an annotation with prefix `#(docs)` and content `Hello`. It's routed to `docs`. An application that has claimed the `docs` route reads the content; everything else ignores it.
6661

67-
## Tag Prefixes
68-
The design of annotations is that the characters immediately following the `#` in an annotation will be available to applications to be able to add different types of annotations. Malloy itself uses the following annotation prefixes
62+
A non-empty prefix must match one of two shapes:
6963

70-
* `# ` and `## ` with a space as a prefix (e.g. `# percent` and `## bar_chart.size=xl`) The Malloy VSCode extension parses these as tags and uses these tags to render results
71-
* `##!` The malloy compiler uses these annotations as tags specifying compiler options
72-
* `#"` and `##"` Reserved for future documentation annotations
73-
* `#(docs)` and `##(docs)` Used by the malloy documentation site
64+
* **Punctuation** — `#` followed by punctuation characters: `#!`, `##!`, `#@`, `#"`, `#:`. These are reserved for Malloy's internal use; the compiler knows the full set. An unrecognized punctuation prefix (`#%`, `#~`) emits a `reserved-route` warning.
7465

75-
## Renderer Tags
76-
Annotations which do not start with `# ` or `## ` are not parsed as renderer tags, though an application may decide to parse annotations with some other prefix as as tags.
66+
* **Bracketed name** — `#` followed by a bracket pair (`()`, `<>`, `[]`, or `{}`) with any non-matching-close characters inside. The route is the literal text inside the brackets. Bracket pair doesn't matter: `#(docs)`, `#<docs>`, `#[docs]`, `#{docs}` all resolve to the same route `docs`. Content is opaque, so `#(bar-chart)`, `#(my.app)`, and `#(https://example.com/ns)` all work as their literal strings.
7767

78-
None of these are render tags, even though they use the tag property language.
68+
If the prefix is just `#` followed by whitespace (`# tag`, `## tag`), the route is the empty string — the default, used by the renderer.
7969

80-
* `#bar_chart` without a space after the `#` is not a render tag, it is a tag using an unrecognized `bar_chart` prefix
81-
* `##! disableWarnings` tags are parsed by the compiler and interpreted as compiler flags
82-
* `#(myApp) custom="application" values="here"` something like this could be used by an app to write custom tags
70+
Anything else (a bare word `#NO_UI`, an unclosed bracket `#(docs`, trailing junk after the close `#((X)))`) emits a `malformed-route` warning. The warning is non-fatal; the annotation is still stored.
8371

84-
For a thorough list of available Renderer Tags, refer to the [Render Tags Documentation](https://github.com/malloydata/malloy/blob/main/packages/malloy-render/docs/renderer_tags_overview.md) in the Malloy GitHub project.
72+
### Routes Malloy itself claims
8573

86-
## Tag Property Language
74+
| route | written as | who reads it |
75+
| --- | --- | --- |
76+
| `''` (empty) | `# tag` / `## tag` | the renderer |
77+
| `!` | `##! flag` | the compiler (compiler flags) |
78+
| `@` | `#@ directive` | the foundation persistence layer |
79+
| `"` | `#" markdown` | the explorer (description strings, rendered as markdown) |
8780

88-
A quick overview of the syntax of properties and values in the Malloy Tag Language used by the renderer.
81+
App routes use the bracketed form: `#(myApp) ...`. The route name is whatever the app claims. The Malloy [documentation site](https://malloydata.dev/) is the informal registry for claimed routes — claim yours there if you publish an app that reads annotations.
82+
83+
# Tags
84+
85+
Annotations whose route uses the empty form (`# tag`) or one of Malloy's punctuation-route forms speak the small property language described below — Malloy's **tag language**. This is what the renderer parses for display hints, what the compiler parses for `##!` flags, and what most apps will parse for their own routes (unless they choose otherwise — see "Bring your own payload" below).
86+
87+
A quick tour:
8988

9089
* `tName`
91-
* Sets the property `tName` to exist, but with no value
92-
* ( for example `# hidden` could mean to set a property called `hidden` on an object )
90+
* Sets the property `tName` to exist, with no value.
91+
* Example: `# hidden`.
9392
* `tName=tVal`
94-
* Sets the property tName to exist, and have the value `tVal`
95-
* ( for example `# color=red` sets the color property to the string `red`)
96-
* You can also use quotes to assign values as as `# name="John J. Johnson"`
97-
* `"` and `'` can be used to quote strings
98-
* To assign a property name which needs quoting, use `` `my long property name`=red ``
99-
* `tName=[val1, val2]` The value of a property can be a list of values
100-
* `-tName` unset the property tName
101-
* ( `-hidden` to remove the `hidden` property from an object)
102-
* `tName: { p1=v1 p2=v2 }` `tName` is a collection of sub properties
103-
* An example might look like `barchart: { bgColor=white fgColor=red }`
104-
105-
### Advanced Property Syntax
106-
107-
* `tName=value { p1=v1 p2=v2 }` It is possible for a property have both a top level value
108-
and a list of sub properties with values.
109-
* `tName=value` Assign new value to tName but delete any existing properties
110-
* `tName=value {...}` Assign a new value to `tName` but do not erase the sub properties like `tName=value` would
111-
* `tName: { p1=v1 p2=v2 p3 }` Replace properties and delete any existing value
112-
* `tName { p1=v1 p2=v2 p3 }` Merge properties into existing properties, preserve value
113-
* `tName=...{ p1=v1 p2=v2 p3 }` Assign new properties to tName, but keep the existing value
114-
* `tName.p1=value` Assign a value to one specific property of tName, value of tName and other properties are preserved.
115-
* `tName.p1=value { pp1=v1 pp2=v2 }` The value of a property can also have properties
93+
* Sets `tName` to a value.
94+
* Example: `# color=red`. Quote values that need spaces: `# name="John J. Johnson"`. Use backticks for property names that need quoting: `` `my long property name`=red ``.
95+
* Lists: `tName=[val1, val2]`.
96+
* `-tName` removes the property `tName`.
97+
* `tName: { p1=v1 p2=v2 }` — `tName` is a collection of sub-properties.
98+
* Example: `# barchart: { bgColor=white fgColor=red }`.
99+
100+
### Advanced property syntax
101+
102+
* `tName=value { p1=v1 p2=v2 }` — `tName` has both a value and sub-properties.
103+
* `tName=value` — assign a new value to `tName`, deleting any existing sub-properties.
104+
* `tName=value {...}` — assign a new value but keep existing sub-properties.
105+
* `tName: { p1=v1 p2=v2 }` — replace sub-properties, delete any existing value.
106+
* `tName { p1=v1 p2=v2 }` — merge sub-properties, preserve value.
107+
* `tName=...{ p1=v1 p2=v2 }` — assign new sub-properties, keep value.
108+
* `tName.p1=value` — set one nested property; other properties and the value of `tName` are preserved.
109+
* `tName.p1=value { pp1=v1 pp2=v2 }` — nested properties can themselves have properties.
110+
111+
For the renderer's catalog of meaningful tag names, see the [Render Tags documentation](https://github.com/malloydata/malloy/blob/main/packages/malloy-render/docs/renderer_tags_overview.md).
112+
113+
# Bring your own payload
114+
115+
If your app's annotations aren't shaped like tags — for example, you want to embed JSON or markdown — claim a bracketed route and parse the content yourself. Malloy hands you the raw content plus source-location offsets so your parser's error messages can point back to the model file:
116+
117+
```typescript
118+
// In the Malloy API
119+
for (const note of field.annotations.forRoute('myApp')) {
120+
const content = note.rawText.slice(note.contentIndex);
121+
try {
122+
handleConfig(JSON.parse(content));
123+
} catch (e) {
124+
reportError(e.message, note.at); // your error squigglies land in the model
125+
}
126+
}
127+
```
128+
129+
If your app *does* use the tag language on its route, the shorter call is:
130+
131+
```typescript
132+
const tag = field.annotations.parseAsTag('myApp').tag;
133+
if (tag.has('hidden')) hide(field);
134+
```
135+
136+
# Annotations are just text
137+
138+
The compiler stores annotation text verbatim and routes by prefix. It does not validate content. A route is a *claim* — by claiming `(myApp)`, you take responsibility for what `#(myApp) ...` annotations mean in your part of the world. Other apps using other routes don't collide with you, and the renderer's default route stays out of your way unless you write `# ...` deliberately.

0 commit comments

Comments
 (0)