Add some docs on lex syntax

Author: ratmice

doc/src/SUMMARY.md

@@ -7,6 +7,7 @@
     - [Extensions](lexextensions.md)
   - [Hand-written lexers](manuallexer.md)
   - [Start States](start_states.md)
+  - [Syntax](lexsyntax.md)
 - [Parsing](parsing.md)
   - [Yacc compatibility](yacccompatibility.md)
     - [Extensions](yaccextensions.md)

doc/src/lexsyntax.md

@@ -0,0 +1,53 @@
+The syntax of the lrlex `.l` format, aims to be familiar to the format used by posix lex.
+It uses the same basic structure as lex.
+
+```
+Definitions
+%%
+Rules
+%%
+UserSubroutines
+```
+
+## Definitions
+
+### Optional %grmtools section
+
+Defined in [extensions](./lexextensions.md)
+
+## Rules 
+
+Each rule is given by
+
+### Optional Start State
+Between angle brackets <Start_State>
+
+### Regex
+
+Regular expressions are define by the rust [regex](https://docs.rs/regex) crate.
+
+
+### Separator space
+
+Any horizontal space character in the unicode `Pattern_White_Space` character set.
+
+### Optional State operator
+
+Between angle brackets these are documented in [Start States](./start_states.md)
+* `<+STATE>` Push the state given by `STATE` to the top of the stack.
+* `<-STATE> Pop the state off the top of the stack.
+* `<STATE>` Replace the state stack with the state `STATE`
+
+
+### Token Name
+
+Token name may contain any non-space character between double quotes, including double quotes
+
+### End of line
+
+The end of line finishes a rule.
+
+## UserSubroutines
+
+Since lrlex doesn't support user actions, it doesn't support User Subroutines either.
+