Skip to content

Conversation

@joshrotenberg
Copy link
Contributor

This PR implements JEP-18 (Lexical Scoping) as an optional feature. Let expressions allow binding values to variables for use within an expression, enabling queries that reference elements from outer scopes.

Syntax

let $variable = expression in body
let $var1 = expr1, $var2 = expr2 in body

Example

// Bind a threshold and use it in a filter
let expr = jmespath::compile("let $threshold = `50` in numbers[? @ > $threshold]").unwrap();

// Reference parent data within a nested filter
let expr = jmespath::compile(
    "[*].[let $home_state = home_state in states[? name == $home_state].cities[]][]"
).unwrap();

Features

Per JEP-18 specification:

  • let and in are contextual keywords (not reserved)
  • Variable references use $name syntax
  • Multiple comma-separated bindings supported
  • Lexical scoping with inner scope shadowing
  • Bindings evaluated in outer scope before body executes
  • Projection stopping when bound to variable
  • Undefined variable triggers runtime error

Feature-gated behind let-expr to maintain backward compatibility:

[dependencies]
jmespath = { version = "0.5", features = ["let-expr"] }

Changes

  • Add let-expr feature flag to Cargo.toml
  • Add Variable and Assign tokens to lexer (feature-gated)
  • Add VariableRef and Let AST nodes
  • Implement let expression parsing in parser
  • Add scope stack to Context for variable lookup
  • Implement Let and VariableRef evaluation in interpreter
  • Add rustdoc documentation with examples

Testing

  • All 861 compliance tests pass
  • 34 unit tests covering:
    • Basic bindings, multiple bindings, nested scopes
    • Variable shadowing and scope restoration
    • Projection stopping behavior
    • Variables in filters, functions, pipes, multiselect
    • Boolean operators (&&, ||, !)
    • Flatten, slice operations
    • Error cases (undefined variable, out of scope, syntax errors)

Known Limitations

Using let or in as field names immediately before the in keyword (e.g., let $x = foo.let in ...) requires more complex parser lookahead. Workaround: use quoted identifiers (let $x = foo."let" in ...).

Implements official JEP-18 let expressions as an optional feature flag.
The let expression introduces lexical scoping with the syntax:

  let $var = expr, $var2 = expr2 in body

Features per JEP-18 specification:
- Contextual keywords: 'let' and 'in' are not reserved
- Variable references use $ prefix syntax
- Multiple comma-separated bindings supported
- Lexical scoping with inner scope shadowing
- Undefined variable triggers runtime error
- Projection stopping behavior preserved

- Add let-expr feature flag to Cargo.toml
- Add Variable and Assign tokens to lexer (feature-gated)
- Add VariableRef and Let AST nodes
- Implement let expression parsing in parser
- Add scope stack to Context for variable lookup
- Implement Let and VariableRef evaluation in interpreter
- Add comprehensive tests for scoping behavior
- All 861 compliance tests pass
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant