Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions .github/skills/asim-parser-create-parameter-parser/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,18 @@ The file name should be prefixed with vim, followed by the name of the schema, e

Add the contents of the parameter-less version of the parser to the new file.

## Preserve parser development restrictions

The parameterized parser must preserve all development guidelines and prohibited patterns from the `asim-parser-create-parser` skill. Adding parameters or filters must not introduce:

- Same-table enrichment, such as a second source-table read or self-join.
- Cross-table or other table-shaped enrichment, including `join`, `lookup`, secondary `union` branches, `externaldata`, or inline `datatable` mappings.
- One-to-many fan-out; each source record must still produce at most one normalized record.
- Any `mv-*` operator, including `mv-expand` and `mv-apply`.
- Aggregation, reaggregation, correlation, or deduplication, including `summarize`, `distinct`, `arg_min`, or `arg_max`.

Use native source columns and scalar predicates for parameter filtering. If a parameter cannot be applied without a prohibited pattern, retain the standard empty-parameter check without adding enrichment, row expansion, or aggregation.

## Add parameters

From the parameters you have gathered, add it to function arguments for both the function and the function call.
Expand Down
24 changes: 23 additions & 1 deletion .github/skills/asim-parser-create-parser/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,11 +40,23 @@ Parsers are KQL functions that follow a clear flow: **Filter → Parse → Map**
- Use indexes so only relevant extents are scanned.
- Filter early on native columns before parsing to improve performance.
- Use high-performance parsing operators (`split`, `parse-kv`, `parse`) and avoid regular expressions for string parsing.
- Normalize values with `iff`, `case`, or lookup tables rather than copying source values directly.
- Normalize values with scalar expressions such as `iff` or `case` rather than copying source values directly.
- Use `project-rename` for mapping, `extend` for calculated or normalized fields.
- Try to map as many fields as possible, including optional ones. This increases usefulness and future-proofs the parser for schema changes.
- Do not use `project-away` to remove unmapped columns. Use `project` instead, as `project-away` does not protect the parser from schema changes in the source data.

### Prohibited patterns

ASIM parsers must remain a single pipeline over the declared source table. Do not use:

- **Same-table enrichment** — including a second read of the source table, a self-join, or a tabular subquery used to add fields.
- **Cross-table or other table-shaped enrichment** — including `join`, `lookup`, a secondary `union` branch, `externaldata`, or inline `datatable` mappings.
- **One-to-many fan-out** — each source record must produce at most one normalized record. Do not expand one record into multiple normalized records; this indicates that the connector or source event shape must be corrected.
- **Any `mv-*` operator** — including `mv-expand` and `mv-apply`.
- **Aggregation, reaggregation, correlation, or deduplication** — including `summarize`, `distinct`, `arg_min`, `arg_max`, or any other operation that combines source records or collapses expanded rows.

Use only values available on the current source row and preserve its record cardinality. Prefer scalar expressions, scalar dynamic-value access, and direct parsing. If a field cannot be mapped without a prohibited enrichment, fan-out, `mv-*`, or aggregation pattern, leave it unmapped.

### Parsing operators by performance ranking

| Rank | Operator | Description |
Expand All @@ -61,6 +73,16 @@ Parsers are KQL functions that follow a clear flow: **Filter → Parse → Map**
### Required columns

- Include the column `Type` in the output of the ASIM parser. This column indicates the source table name.
- Determine the placeholder entity fields applicable to the parser from the target schema and `Common` rows in `ASimTester.csv`. Include every defined column whose name ends in `EntityKey` or `AdditionalIds`, as well as `AdditionalEntities` when it is defined for the schema.
- Preserve the exact field names and casing from `ASimTester.csv`. Entity fields are role-specific (for example, `ActorUserEntityKey`, `SrcSystemEntityKey`, and `ActorUserAdditionalIds`); do not replace them with generic names such as `entityKey` or `AdditionalIds`.
- Until mappings for these fields are defined, set every `*EntityKey` field to an empty string and every `*AdditionalIds` field and `AdditionalEntities` to an empty dynamic array. For example:

```kusto
ActorUserEntityKey = "",
SrcSystemEntityKey = "",
ActorUserAdditionalIds = dynamic([]),
AdditionalEntities = dynamic([])
```

### Required parameters

Expand Down
18 changes: 17 additions & 1 deletion .github/skills/asim-parser-pr-reviewer/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,11 +44,23 @@ Review the parameter-less parser for the following:

- **No `project-away`**: The query must NOT use `project-away` to remove unmapped columns. It should use `project` instead, as `project-away` does not protect the parser from schema changes in the source data.

- **No same-table or cross-table enrichment**: Verify that the query is a single pipeline over the declared source table. Flag any second read of the source table, self-join, cross-table reference, `join`, `lookup`, secondary `union` branch, `externaldata`, inline `datatable` mapping, or equivalent table-shaped enrichment. If a field cannot be produced from the current source row, it should remain unmapped rather than be enriched.

- **One source record produces at most one normalized record**: Flag any operation that fans out one source record into multiple normalized records, regardless of operator. Fan-out indicates a connector or source event-shape defect that should be corrected rather than supported in the parser.

- **No `mv-*` operators**: Flag any KQL operator whose name begins with `mv-`, including `mv-expand` and `mv-apply`. The parser should use scalar dynamic-value access or other scalar expressions instead. If a field cannot be mapped without row expansion, it should remain unmapped.

- **No aggregation, reaggregation, correlation, or deduplication**: Flag `summarize`, `distinct`, `arg_min`, `arg_max`, or any equivalent operation that combines source records or collapses expanded rows. An ASIM parser should normalize each source record independently rather than correlate, deduplicate, or reaggregate records.

- **`pack` parameter**: If the query uses `AdditionalFields`, verify that a `pack: bool = false` parameter is included. This allows users to choose whether to populate `AdditionalFields` or return an empty dynamic, improving performance for users who do not need the extra information.

- **Placeholder entity fields**: Determine the complete set of applicable fields from the target schema and `Common` rows in `ASimTester.csv`. Verify that the parser includes every defined `*EntityKey` and `*AdditionalIds` field, plus `AdditionalEntities` when defined. Names and casing must exactly match `ASimTester.csv`; generic names such as `entityKey` or `AdditionalIds` are invalid. Every `*EntityKey` must be an empty string, and every `*AdditionalIds` field and `AdditionalEntities` must be an empty dynamic array until mappings are defined.

- **Parsing operator efficiency**: Check that high-performance parsing operators are used (`split`, `parse-kv`, `parse`) and that regular expressions are avoided where simpler operators would work.

- **General KQL performance**: Flag any other inefficient patterns such as unnecessary `let` statements, redundant filters, expensive joins, or operations that could be reordered for better performance.
- **General KQL performance**: Flag any other inefficient patterns such as unnecessary `let` statements, redundant filters, or operations that could be reordered for better performance.

Treat every same-table enrichment, cross-table/table-shaped enrichment, fan-out, `mv-*`, or aggregation/reaggregation violation as 🔴 High priority.

**Output format:**

Expand All @@ -57,6 +69,7 @@ Return your findings as a markdown table with the following columns:
| # | Priority | Issue | Suggestion |
| --- | -------- | ----- | ---------- |


Where:

- **Priority** is one of: 🔴 High, 🟡 Medium, 🟢 Low
Expand All @@ -81,6 +94,8 @@ Please review:
2. **Filter efficiency**: Are the parameter-based filters using native columns and indexed fields where possible?
3. **Redundant computation**: Are there any calculated fields or parsing operations that occur before the parameter filters, when they could be moved after?
4. **Parameter completeness**: Are the filtering parameters comprehensive enough to allow efficient querying for common use cases?
5. **Placeholder entity field consistency**: Compare the parameterized parser's placeholder fields and values with the parameter-less parser. Report fields missing from both parsers only in the parameter-less review above. In this section, report only differences introduced by the parameterized parser, such as a placeholder that is omitted, renamed, or populated when the parameter-less parser defines it correctly. Matching omissions are not acceptable; they are already reported in the parameter-less review.
6. **Prohibited pattern consistency**: Report any same-table enrichment, cross-table/table-shaped enrichment, fan-out, `mv-*`, or aggregation/reaggregation introduced only by the parameterized parser as 🔴 High priority. If the same violation exists in both parser versions, report it only in the parameter-less review above.

**Output format:**

Expand All @@ -89,5 +104,6 @@ Return findings as a markdown table:
| # | Priority | Issue | Suggestion |
| --- | -------- | ----- | ---------- |


Where Priority is one of: 🔴 High, 🟡 Medium, 🟢 Low.
Only include issues specific to the filtering/parameter logic.
Loading