Improved Identifiers¶

Unicode¶

JMESPath supports unicode through the char and unescaped-char rule:

unescaped-char    = %x30-10FFFF
char              = %x30-39 / ; 0-9
                    %x41-5A / ; A-Z
                    %x5F /    ; _
                    %x61-7A / ; a-z
                    %x7F-10FFFF

However, JSON supports a syntax for escaping unicode characters. Any character in the Basic Multilingual Plane (BMP) can be escaped with:

char = escape (%x75 4HEXDIG )  ; \uXXXX

Similar to the way that XPath supports numeric character references used in XML (&#nnnn), JMESPath should support the same escape sequences used in JSON. JSON also supports a 12 character escape sequence for characters outside of the BMP, by encoding the UTF-16 surrogate pair. For example, the code point U+1D11E can be represented as "\uD834\uDD1E".

Escape Sequences¶

Consider the following JSON object:

{"foo\nbar": "baz"}

A JMESPath expression should be able to retrieve the value of baz. With the current grammar, one must rely on the environment’s ability to input control characters such as the newline (%x0A). This can be problematic in certain environments. For example, in python, this is not a problem:

>>> jmespath_expression = "foo\nbar"

Python will interpret the sequence "\n" (%x5C %x6E) as the newline character %x0A. However, consider Bash:

$ foo --jmespath-expression "foo\nbar"

In this situation, bash will not interpret the "\n" (%x5C %x6E) sequence.

basic.json¶

The following needs to be changed because identifiers starting with a number must now be quoted:

-            "expression": "foo.1",
+            "expression": "foo.\"1\"",
             "result": ["one", "two", "three"]
          },
          {
-            "expression": "foo.1[0]",
+            "expression": "foo.\"1\"[0]",
             "result": "one"
          },

Similarly, the following needs to be changed because an unquoted identifier cannot start with -:

-            "expression": "foo.-1",
+            "expression": "foo.\"-1\"",
             "result": "bar"
          }

escape.json¶

The escape.json has several more interseting cases that need to be updated. This has to do with the updated escaping rules. Each one will be explained.

-            "expression": "\"foo\nbar\"",
+            "expression": "\"foo\\nbar\"",
             "result": "newline"
          },

This has to be updated because a JSON parser will interpret the \n sequence as the newline character. The newline character is not allowed in a JMESPath identifier (note that the newline character %0A is not in any rule). In order for a JSON parser to create a sequence of %x5C %x6E, the JSON string must be \\n (%x5C %x5C %x6E).

-            "expression": "\"c:\\\\windows\\path\"",
+            "expression": "\"c:\\\\\\\\windows\\\\path\"",
             "result": "windows"
          },

The above example is a more pathological case of escaping. In this example, we have a string that represents a windows path “c:\windowpath”. There are two levels of escaping happening here, one at the JSON parser, and one at the JMESPath parser. The JSON parser will take the sequence "\"c:\\\\\\\\windows\\\\path\"" and create the string "\"c:\\\\windows\\path\"". The JMESPath parser will take the string "\"c:\\\\windows\\path\"' and, applying its own escaping rules, will look for a key named c:\\windows\path.