Skip to content

docs(spec): document directive syntax in language grammar#11022

Open
Copilot wants to merge 4 commits into
mainfrom
copilot/update-grammar-spec-for-directives
Open

docs(spec): document directive syntax in language grammar#11022
Copilot wants to merge 4 commits into
mainfrom
copilot/update-grammar-spec-for-directives

Conversation

Copilot AI commented Jun 19, 2026

Copy link
Copy Markdown
Contributor

The grammar spec had no coverage of the #suppress/#deprecated directive syntax, and incorrectly claimed that newline sequences have no language-semantic impact — which is false since directives are newline-terminated.

Lexical grammar

  • Separate LineTerminatorSequence from WhiteSpace — the scanner already emits Token.NewLine (CR, LF, CRLF) separately from Token.Whitespace; the spec now reflects this
  • Add LineTerminator and LineTerminatorSequence productions defining the three recognised sequences
  • Update Trivia to include LineTerminatorSequence alongside Comment and WhiteSpace
  • Fix the newline comment — removes the incorrect claim; explains that line terminators are syntactically significant (directive termination)

Syntactic grammar

  • Add directive productions: DirectiveList, DirectiveExpression, DirectiveArguments, DirectiveArgument — with prose notes on newline-aware termination, the EOF edge case, the two recognised directive names, and their per-directive argument counts (validated semantically)
  • Thread DirectiveList? into all annotatable productionsBlocklessNamespaceStatement, ModelStatement, ScalarStatement, InterfaceStatement, InterfaceMember, UnionStatement, UnionVariant, EnumStatement, EnumMember, NamespaceStatement, OperationStatement, ModelProperty

Example of what is now documented:

#suppress "deprecated" "Using old API intentionally"
@decorator
model Foo { }

#deprecated "Use Bar instead"
model OldFoo { }

- Add LineTerminator and LineTerminatorSequence productions to lexical
  grammar, separate from WhiteSpace (matching how the scanner emits
  Token.NewLine vs Token.Whitespace)
- Update Trivia to include LineTerminatorSequence alongside Comment and
  WhiteSpace
- Fix the incorrect comment claiming newlines have no language-semantic
  impact; directives are terminated by newlines so they ARE significant
- Add DirectiveList, DirectiveExpression, DirectiveArguments, and
  DirectiveArgument productions to the syntactic grammar with notes on
  newline-aware termination and currently supported directive names
  (#suppress, #deprecated)
- Add DirectiveList? before DecoratorList? in all statement/declaration
  productions that accept annotations

Closes #949

Co-authored-by: timotheeguerin <1031227+timotheeguerin@users.noreply.github.com>
Copilot AI changed the title [WIP] Update grammar spec to document directive syntax docs(spec): document directive syntax in language grammar Jun 19, 2026
Copilot AI requested a review from timotheeguerin June 19, 2026 14:33
@timotheeguerin

Copy link
Copy Markdown
Member

@copilot ecmarkcup runs out memory now

> ecmarkup --strict src/spec.emu.html dist/spec.html 2>&1

Warning: inlining assets; this should not be done for production builds.

<--- Last few GCs --->

[3129:0x39fb4000]   103130 ms: Scavenge (interleaved) 4064.8 (4133.9) -> 4064.8 (4134.9) MB, pooled: 0 MB, 52.16 / 0.00 ms  (average mu = 0.231, current mu = 0.126) allocation failure; 
[3129:0x39fb4000]   103412 ms: Scavenge (interleaved) 4065.8 (4134.9) -> 4065.8 (4157.9) MB, pooled: 0 MB, 270.49 / 0.00 ms  (average mu = 0.231, current mu = 0.126) allocation failure; 


<--- JS stacktrace --->

FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memory
----- Native stack trace -----

 1: 0xe46bbe node::OOMErrorHandler(char const*, v8::OOMDetails const&) [node]
 2: 0x1243640 v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, v8::OOMDetails const&) [node]
 3: 0x1243917 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, v8::OOMDetails const&) [node]
 4: 0x1472825  [node]
 5: 0x148c0b9 v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [node]
 6: 0x14607b8 v8::internal::HeapAllocator::AllocateRawWithLightRetrySlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [node]
 7: 0x14616e5 v8::internal::HeapAllocator::AllocateRawWithRetryOrFailSlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [node]
 8: 0x143a2ee v8::internal::Factory::NewFillerObject(int, v8::internal::AllocationAlignment, v8::internal::AllocationType, v8::internal::AllocationOrigin) [node]
 9: 0x189cbfc v8::internal::Runtime_AllocateInYoungGeneration(int, unsigned long*, v8::internal::Isolate*) [node]
10: 0x7f69cae6caf6 
Aborted (core dumped)
 ELIFECYCLE  Command failed with exit code 134.

Defining `LineTerminator` caused ecmarkup to try expanding the
"SourceCharacter but not ... or LineTerminator" productions in
BacktickedIdentifierChar, StringCharacter, and SingleLineCommentChar.
This triggered a memory blowup that crashed the build at 4GB heap.

Since LineTerminator was already referenced-but-undefined in those
three productions before this PR, keep it undefined.  Define only
`LineTerminatorSequence` (using <LF>/<CR> character literals directly)
since it is the production that needs to appear in Trivia.

Also remove the two-alternative form of DirectiveExpression that used
`LineTerminatorSequence` as a syntactic-grammar terminal — directive
termination is explained in prose instead.

Co-authored-by: timotheeguerin <1031227+timotheeguerin@users.noreply.github.com>

Copilot AI commented Jun 19, 2026

Copy link
Copy Markdown
Contributor Author

@copilot ecmarkcup runs out memory now

...

Fixed in the latest commit. The root cause was the LineTerminator : one of <LF> <CR> production definition I added. It was already referenced (but undefined) in BacktickedIdentifierChar, StringCharacter, and SingleLineCommentChar — and ecmarkup handled those undefined references fine. Once I defined it, ecmarkup tried to expand all the "SourceCharacter but not … or LineTerminator" productions and blew up.

Fix: remove the LineTerminator definition entirely. LineTerminatorSequence is defined directly using <LF>/<CR> character literals (no reference to LineTerminator), so it's unaffected. I also removed the LineTerminatorSequence terminal from DirectiveExpression — Trivia elements are transparent in the syntactic grammar, so directive termination is now explained in prose only.

@timotheeguerin

Copy link
Copy Markdown
Member

@copilto fix spellcheck

@timotheeguerin

Copy link
Copy Markdown
Member

@copilot fix spellcheck

Co-authored-by: timotheeguerin <1031227+timotheeguerin@users.noreply.github.com>

Copilot AI commented Jun 19, 2026

Copy link
Copy Markdown
Contributor Author

@copilot fix spellcheck

Fixed — two instances of recognisedrecognized.

@azure-sdk-automation

Copy link
Copy Markdown

You can try these changes here

🛝 Playground 🌐 Website 🛝 VSCode Extension

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants