You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: src/blog/2026-01-13-context-md-convention/index.malloynb
+3-3Lines changed: 3 additions & 3 deletions
Original file line number
Diff line number
Diff line change
@@ -10,7 +10,7 @@ This is based on our experience. We're sharing it to start a conversation. We ho
10
10
11
11
## The Malloy Experience
12
12
13
-
Starting from zero experience with LLMs we discovered that if you fill a conversation context with all information about the Malloy repository, the result is a context with very little room for exploration and problem solving. In our experience, the results of LLM assisted coding are much better when carefully feeding the LLM a context focused on the problem at hand. Over time we found there were particular areas of the code where it made sense to gather the information the LLM gleaned, and some guidance from a human who is very familiar with the code, into a context-unit which would be re-usable and shareable.
13
+
Starting from zero experience with LLMs, we discovered that filling a conversation context with all information about the Malloy repository leaves very little room for exploration and problem solving. Results are much better when you carefully feed the LLM context focused on the problem at hand. Over time, we found it made sense to gather the information the LLM gleaned — along with guidance from a human familiar with the code — into reusable, shareable context units.
14
14
15
15
As we review pull requests, we continue to update our contexts, and this is gradually making pull request reviews cleaner. If our scheme of `CONTEXT.md` maintenance were generally known and recognized by the LLMs used by contributors, it would make it easier for external contributors to make solid contributions to our repository.
16
16
@@ -22,11 +22,11 @@ We distribute `CONTEXT.md` files throughout the repository tree. Each file descr
22
22
23
23
- Our main repository is a monorepo, with multiple sub-components (Compiler, Renderer, API, etc.) so
24
24
a single context file doesn't make sense for these individual components.
25
-
- Keeps individual context files smaller, allows an LLMassisted activity to read the appropriate context
25
+
- Keeps individual context files smaller. Allows an LLM-assisted activity to read the appropriate context
26
26
to finish a task without reading all documentation for the project. Scales well to larger projects.
27
27
- Keeps the naming/location of these files LLM-agnostic
28
28
- Humans and LLMs are always pair programming, so both humans and LLMs need to be able to
29
-
both review code changes, and update and maintain context files for the code.
29
+
review code changes and update and maintain context files for the code.
Copy file name to clipboardExpand all lines: src/blog/2026-02-25-persistence/index.malloynb
+5-5Lines changed: 5 additions & 5 deletions
Original file line number
Diff line number
Diff line change
@@ -43,7 +43,7 @@ The `#@ persist` annotation is metadata, not a language keyword. It lives in Mal
43
43
44
44
### Inheritance
45
45
46
-
Persistence flows through `extend`. If the base data is worth persisting, derived versions usually are too:
46
+
Persistence flows through `extend`. When you extend a persistent source, the extension reads from the same pre-built table — it just adds computed fields on top:
47
47
48
48
```malloy
49
49
#@ persist name=by_carrier
@@ -52,12 +52,12 @@ source: by_carrier is flights -> {
52
52
aggregate: flight_count is count()
53
53
}
54
54
55
-
// Also persistent — inherits from by_carrier
55
+
// Reads from the pre-built by_carrier table, adds a computed field
56
56
source: enriched is by_carrier extend {
57
57
dimension: upper_carrier is upper(carrier)
58
58
}
59
59
60
-
// Break the chain when you need to
60
+
// Opt out: recomputes the query instead of using the pre-built table
61
61
#@ -persist
62
62
source: scratch is by_carrier extend { ... }
63
63
```
@@ -116,7 +116,7 @@ To verify it's working, compile a query without running it and check the SQL. Yo
116
116
117
117
Many connections work with no configuration at all — if a connection name matches a registered database type, Malloy creates one with default settings. For persistence, the main thing you need to decide is where the manifest lives:
118
118
119
-
- **Global config** (simplest): The CLI reads `~/.config/malloy/malloy-config.json` by default and writes the manifest next to it. Point VS Code's `malloy.globalConfigDirectory` at the same directory, and both tools share the same manifest. No flags needed.
119
+
- **Global config** (simplest): The CLI reads `~/.config/malloy/malloy-config.json` by default and writes the manifest next to it. Point VS Code's `malloy.globalConfigDirectory` at the same directory, and both tools share the same manifest. No flags needed. If you're just getting started, this is the easiest path.
120
120
121
121
- **Project config**: Put a `malloy-config.json` in your project root (it can be as minimal as `{}`) to anchor the manifest to your project. VS Code finds it automatically; the CLI needs `--config .`.
122
122
@@ -144,7 +144,7 @@ Because the manifest is optional, persistence supports very different experience
144
144
145
145
**Partial manifest (incremental builds):** Some sources are pre-built, others expand inline. The compiler doesn't care which sources are in the manifest and which aren't. This lets a builder do incremental work — build what's missing, skip what's fresh.
146
146
147
-
**Full manifest with `strict` mode (production):** When a manifest has `strict: true`, the compiler fails if any persistent source is *missing* from the manifest. This catches configuration errors — if a table should have been built but wasn't, you find out at compile time rather than by accidentally running an expensive query in production. The builder sets `strict` in the manifest file; an application can override it with `manifest.strict = false` if needed.
147
+
**Full manifest with `strict` mode (production):** When a manifest has `strict: true`, the compiler fails if any persistent source is *missing* from the manifest. This catches configuration errors — if a table should have been built but wasn't, you find out at compile time rather than by accidentally running an expensive query in production. The builder sets `strict` in the manifest file; an application using the API can override this programmatically if needed.
148
148
149
149
## Building a Sophisticated Persistence Application
Copy file name to clipboardExpand all lines: src/documentation/experiments/experiments.malloynb
+2-2Lines changed: 2 additions & 2 deletions
Original file line number
Diff line number
Diff line change
@@ -1,11 +1,11 @@
1
1
>>>markdown
2
2
# Experimental Features
3
3
4
-
Before releasing language features, we like to get a feel for how they work when used in programs. We often find bugs or language problems. We like when people use these features with the understanding that they might changed in their final form or never be released.
4
+
Before releasing language features, we like to get a feel for how they work when used in programs. We often find bugs or language problems. We like when people use these features with the understanding that they might change in their final form or never be released.
5
5
6
6
We value feedback during the experimental phase. Please make sure you let us know what you think.
7
7
8
-
Below is a list of currently running experiements and how to turn them on.
8
+
Below is a list of currently running experiments and how to turn them on.
Copy file name to clipboardExpand all lines: src/documentation/experiments/persistence.malloynb
+5-5Lines changed: 5 additions & 5 deletions
Original file line number
Diff line number
Diff line change
@@ -35,15 +35,15 @@ source: by_origin is flights -> {
35
35
}
36
36
```
37
37
38
-
Persistence is inherited when you `extend` a persistent source. The child keeps the same table name unless you override it. Use `#@ -persist` to opt out:
38
+
When you `extend` a persistent source, the extension still reads from the same pre-built table — the extension just adds computed fields on top. Use `#@ -persist` to opt out and force the underlying query to be recomputed instead:
39
39
40
40
```malloy
41
-
// Inherits persistence from by_carrier
41
+
// Reads from the pre-built by_carrier table, adds a computed field
42
42
source: enriched is by_carrier extend {
43
43
dimension: upper_carrier is upper(carrier)
44
44
}
45
45
46
-
// Opt outof persistence
46
+
// Opt out: recomputes the query instead of using the pre-built table
The builder compiles `.malloy` files, finds `#@ persist` sources, and creates the database tables:
94
+
The builder compiles `.malloy` files (or all files in a directory), finds `#@ persist` sources, and creates the database tables:
95
95
96
96
```bash
97
97
# Build all .malloy files in the current directory (recursive)
@@ -159,7 +159,7 @@ If VS Code and the builder are reading different config files, they'll have diff
159
159
160
160
By default, if a `#@ persist` source is missing from the manifest (e.g., it hasn't been built yet), queries fall through to computing the result inline — the same as if no manifest existed. This is convenient during development but risky in production, where an unbuilt table might mean an unexpectedly expensive query.
161
161
162
-
The builder creates manifests with `strict` mode enabled. When strict mode is on, a missing manifest entry for a `#@ persist` source causes an error instead of falling through. This catches configuration problems early — if a table should have been built but wasn't, you find out at compile time.
162
+
The builder creates manifests with `strict` mode enabled by default. When strict mode is on, a missing manifest entry for a `#@ persist` source causes an error instead of falling through. This catches configuration problems early — if a table should have been built but wasn't, you find out at compile time.
163
163
164
164
If you need to disable strict mode (e.g., during development when only some tables are built), you can edit `malloy-manifest.json` and set `"strict": false`. The builder will not overwrite this setting on subsequent builds.
Copy file name to clipboardExpand all lines: src/documentation/language/connections.malloynb
+2-2Lines changed: 2 additions & 2 deletions
Original file line number
Diff line number
Diff line change
@@ -17,7 +17,7 @@ There are currently two connection methods, `.table()` and `.sql()`.
17
17
18
18
## Table Connection Method
19
19
20
-
The `.table()` connection method is used to reference a table or view in a database. It accepts a single string representing a table. The exact semantics of how that string is resolved into a table schema depend on the database and application.
20
+
The `.table()` connection method is used to reference a table or view in a database. It accepts a single string representing a table. The exact semantics of how that string is resolved depend on the database and application.
21
21
22
22
In the official Malloy connection implementations, the behavior is as follows:
23
23
@@ -31,7 +31,7 @@ In Databricks, the string passed to the `.table()` connection method can be a on
31
31
32
32
### DuckDB
33
33
34
-
In DuckDB, the `.table()` method accepts the path (relative to the Malloy file) of CSV, JSON, or Parquet file containing the table data, e.g. `duckdb.table('data/users.csv')` or `duckdb.table('../../users.parquet')`. URLs to such files (or APIs) are also allowed: see [an example here](../patterns/apijson.malloynb).
34
+
In DuckDB, the `.table()` method accepts the path (relative to the Malloy file) of a CSV, JSON, or Parquet file containing the table data, e.g. `duckdb.table('data/users.csv')` or `duckdb.table('../../users.parquet')`. URLs to such files (or APIs) are also allowed: see [an example here](../patterns/apijson.malloynb).
Copy file name to clipboardExpand all lines: src/documentation/language/dialect/databricks.malloynb
+1-1Lines changed: 1 addition & 1 deletion
Original file line number
Diff line number
Diff line change
@@ -44,7 +44,7 @@ source: orders is databricks.table('my_catalog.analytics.orders')
44
44
45
45
## Limitations
46
46
47
-
- **`string_agg` ordering**: Databricks does not support `ORDER BY` inside `COLLECT_LIST`/`COLLECT_SET`, so `string_agg` and `string_agg_distinct` do not support the `order_by` modifier.
47
+
- **`string_agg` ordering**: Databricks does not support ordering within aggregate collection functions, so `string_agg` and `string_agg_distinct` do not support the `order_by` modifier.
48
48
- **`TIMESTAMP_NTZ`**: Databricks' `TIMESTAMP_NTZ` (timestamp without timezone) maps to `sql native` in Malloy. Use explicit casting to `timestamp` when needed.
Copy file name to clipboardExpand all lines: src/documentation/malloy_cli/index.malloynb
+2-2Lines changed: 2 additions & 2 deletions
Original file line number
Diff line number
Diff line change
@@ -1,7 +1,7 @@
1
1
>>>markdown
2
2
# About the Malloy CLI
3
3
4
-
The [Malloy CLI](https://github.com/malloydata/malloy-cli) is a command-line interface for running `.malloysql` and `.malloy` files. It can be used to automate transformations, build simple pipelines, or even integrate compiled Malloy SQL into other applications.
4
+
The [Malloy CLI](https://github.com/malloydata/malloy-cli) is a command-line interface for running `.malloysql` and `.malloy` files and building persistent tables. It can be used to automate transformations, build simple pipelines, or even integrate compiled Malloy SQL into other applications.
5
5
6
6
## Installation
7
7
@@ -25,7 +25,7 @@ If a connection name in your model matches a registered database type (DuckDB, B
25
25
26
26
## Usage
27
27
28
-
The main commands of the CLI are `run`and `compile` — `run` executes queries and returns results, whereas `compile` returns SQL for a query or many queries.
28
+
The main commands of the CLI are `run`, `compile`, and `build` — `run` executes queries and returns results, `compile` returns SQL for a query or many queries, and `build` creates persistent tables from `#@ persist` sources.
29
29
30
30
The CLI has detailed usage information for each command. You can get general help with `malloy-cli --help`, and command-specific help and options with `malloy-cli {command} --help`
Copy file name to clipboardExpand all lines: src/documentation/setup/cli.malloynb
+1-1Lines changed: 1 addition & 1 deletion
Original file line number
Diff line number
Diff line change
@@ -1,5 +1,5 @@
1
1
>>>markdown
2
-
# Installing the CLI
2
+
# CLI Setup
3
3
4
4
The [Malloy CLI](https://github.com/malloydata/malloy-cli) is a command-line interface for running `.malloysql` and `.malloy` files. Use it to automate transformations, build pipelines, or compile Malloy queries to SQL.
0 commit comments