Literal expression \\ in jmespath expression

[BartelNieuwenhuyse]

the literal expression `\\` fails to produce the expected result:

expected:
search( ["1","2","3"], join(`\\`, @) ) -> "1\2\3"

actual:
search( ["1","2","3"], join(`\\`, @) ) -> jmespath expression fails to compile: syntax error

[BartelNieuwenhuyse]

search( ["1","2","3"], join(`\\`, @) ) is still supported by some implementations, but after JEP-12 it has been deprecated and should be written as search( ["1","2","3"], join(`"\\"`, @) ), which is successfully parsed by jsoncons and produces the expected result.

[danielaparker]

The grammar reads

raw-string        = "'" *raw-string-char "'"
raw-string-char   = (%x20-26 / %x28-5B / %x5D-10FFFF) / preserved-escape /
                      raw-string-escape
preserved-escape  = escape (%x20-26 / %28-5B / %x5D-10FFFF)
raw-string-escape = escape ("'" / escape)

which I think means that you're right.

But the change breaks the last test case in literal.json, which asserts that only the single quote character can be escaped in a raw string.

{
      "comment": "Backslash not followed by single quote is treated as any other character",
      "expression": "'\\\\'",
      "result": "\\\\"
}

The JMESPath Playground is consistent with the test case (and with the current implementation in jsoncons).

I'm a little hesitant to accept the pull request until these inconsistencies are resolved on the JMESPath side. In the meantime you could substitute a literal with backticks,

join(`"\\"`,@)

[danielaparker]

@BartelNieuwenhuyse, I appreciate your comment and referencing JEP-12. It's of note though that the raw-string-escape grammar rule in JEP-12,

raw-string-escape = escape ["'"] 

is different than the raw-string-escape grammar rule in the JMESPath spec,

raw-string-escape = escape ("'" / escape)

It seems this change was submitted in PR #26. From the PR,

[the existing grammar] does not allow backslash unless followed by single quote.

Allowing lone backslashes is problematic since it makes the semantics of '\' unclear. In the current go implementation, for example, this is treated as an unterminated raw string, but it could equally well be interpreted as a string with [a literal backslash] by another implementation, and neither would be wrong.

This clarifies that we want to treat backslashes literally, except when followed by single quote. Also, to make it possible to end a string with a backslash we also require backslashes to be escaped.

I have to confess that I don't fully understand the intent in the last paragraph. "treat backslashes literally, except when followed by single quote" and "require backslashes to be escaped" seem to me contradictory.

[BartelNieuwenhuyse]

@danielaparker thank you for your reaction. I think the current implementation of jsoncons jmespath is correct. That's also why I closed the pull request. But I appreciate you thinking along.

I had some trouble representing a single backslash literal, which I wanted to use in a custom function that would split a windows path C:\Directory\Sub\File into its directories + file -> ["C:", "Directory", "Sub", "File"].

There are two literals in the grammar: raw string literal (single quoted) and literal (backtick)

The raw string literal (single quote), I think, is intended to represent a json string value, where all characters that need to be escaped in a json string, should be escaped. And after evaluating, it should again output a json escaped string. Since backslash should be escaped in a json string value, '\\' will output '\\' again. And a single backslash '\' is invalid json, and hence also invalid in jmespath expression.

As to the literal (backtick): I was confused by the fact that other implementations (including jmespath playground) allow for unquoted strings in backtick literal: `\\`, while jsoncons implementation does not (it throws invalid syntax). But this is due to the fact that those other implementation still support deprecate syntax.

JEP-12a:

Deprecation of the current literal parsing behavior that allows for unquoted JSON strings to be parsed as JSON strings

Jsoncons is consistent with the new syntax, and does not allow invalid json inside backticks.

I think backtick literal are intended to contain valid json objects (serialized as string). An unquoted string is not a valid json object. So, if you want to represent just a string in backtick literal, you should double quote it.

The main difference between raw string literal '\\' and literal `\\` is that the first is outputted without any json parsing, and the second is outputted after parsing by a json parser.

Conclussion: the only valid way to represent a single backslash (e.g. to input as argument into a function) is a backtick literal with quoted double backslash: `\\`