Functions¶

Abstract¶

This document proposes modifying the JMESPath grammar to support function expressions.

Motivation¶

Functions allow users to easily transform and filter data in JMESPath expressions. As JMESPath is currently implemented, functions would be very useful in multi-select-list and multi-select-hash expressions to format the output of an expression to contain data that might not have been in the original JSON input. Combined with filtered expressions, functions would be a powerful mechanism to perform any kind of special comparisons for things like length(), contains(), etc.

Data Types¶

In order to support functions, a type system is needed. The JSON types are used:

• number (integers and double-precision floating-point format in JSON)
• string
• boolean (true or false)
• array (an ordered, sequence of values)
• object (an unordered collection of key value pairs)
• null

Syntax Changes¶

Functions are defined in the function-expression rule below. A function expression is an expression itself, and is valid any place an expression is allowed.

The grammar will require the following grammar additions:

function-expression = unquoted-string  (
                        no-args  /
                        one-or-more-args )
no-args             = "(" ")"
one-or-more-args    = "(" ( function-arg *( "," function-arg ) ) ")"
function-arg        = expression / number / current-node
current-node        = "@"

expression will need to be updated to add the function-expression production:

expression        = sub-expression / index-expression / or-expression / identifier / "*"
expression        =/ multi-select-list / multi-select-hash
expression        =/ literal / function-expression

A function can accept any number of arguments, and each argument can be an expression. Each function must define a signature that specifies the number and allowed types of its expected arguments. Functions can be variadic.

foo[].[count(@), bar]

Function Evaluation¶

Functions are evaluated in applicative order. Each argument must be an expression, each argument expression must be evaluated before evaluating the function. The function is then called with the evaluated function arguments. The result of the function-expression is the result returned by the function call. If a function-expression is evaluated for a function that does not exist, the JMESPath implementation must indicate to the caller that an unknown-function error occurred. How and when this error is raised is implementation specific, but implementations should indicate to the caller that this specific error occurred.

Functions can either have a specific arity or be variadic with a minimum number of arguments. If a function-expression is encountered where the arity does not match or the minimum number of arguments for a variadic function is not provided, then implementations must indicate to the caller than an invalid-arity error occurred. How and when this error is raised is implementation specific.

Each function signature declares the types of its input parameters. If any type constraints are not met, implementations must indicate that an invalid-type error occurred.

In order to accommodate type contraints, functions are provided to convert types to other types (to_string, to_number) which are defined below. No explicit type conversion happens unless a user specifically uses one of these type conversion functions.

Function expressions are also allowed as the child element of a sub expression. This allows functions to be used with projections, which can enable functions to be applied to every element in a projection. For example, given the input data of ["1", "2", "3", "notanumber", true], the following expression can be used to convert (and filter) all elements to numbers:

search([].to_number(@), ``["1", "2", "3", "notanumber", true]``) -> [1, 2, 3]

This provides a simple mechanism to explicitly convert types when needed.

Built-in Functions¶

JMESPath has various built-in functions that operate on different data types, documented below. Each function below has a signature that defines the expected types of the input and the type of the returned output:

return_type function_name(type $argname)
return_type function_name2(type1|type2 $argname)

If a function can accept multiple types for an input value, then the multiple types are separated with |. If the resolved arguments do not match the types specified in the signature, an invalid-type error occurs.

The array type can further specify requirements on the type of the elements if they want to enforce homogeneous types. The subtype is surrounded by [type], for example, the function signature below requires its input argument resolves to an array of numbers:

return_type foo(array[number] $argname)

As a shorthand, the type any is used to indicate that the argument can be of any type (array|object|number|string|boolean|null).

The first function below, abs is discussed in detail to demonstrate the above points. Subsequent function definitions will not include these details for brevity, but the same rules apply.

number abs(number $value)

{"foo": -1, "bar": "2"}

number avg(array[number] $elements)

boolean contains(array|string $subject, array|object|string|number|boolean $search)

number ceil(number $value)

number floor(number $value)

string join(string $glue, array[string] $stringsarray)

array keys(object $obj)

number length(string|array|object $subject)

number max(array[number] $collection)

number min(array[number] $collection)

array sort(array $list)

string to_string(string|number|array|object|boolean $arg)

number to_number(string|number $arg)

string type(array|object|string|number|boolean|null $subject)

array values(object $obj)

Compliance Tests¶

A functions.json will be added to the compliance test suite. The test suite will add the following new error types:

• unknown-function
• invalid-arity
• invalid-type

The compliance does not specify when the errors are raised, as this will depend on implementation details. For an implementation to be compliant they need to indicate that an error occurred while attempting to evaluate the JMESPath expression.

History¶

• This JEP originally proposed the literal syntax. The literal portion of this JEP was removed and added instead to JEP 7.
• This JEP originally specified that types matches should return null. This has been updated to specify that an invalid type error should occur instead.