Rebuilding the Parser: Why We Changed and What It Enables

Over the past few weeks, we’ve been evolving the Luma parser to support richer language features like:

• Chained method calls and expressions

• Lambda expressions with inline or block bodies

• Interpolated strings

• Complex expression precedence handling

• Dot access + indexing + slicing

• Functional constructs like .filter(x -> x > 1), .walk(...) -> {}

But as we began introducing these features, we hit a wall:

our old parser was brittle — adding one new construct often broke several others.

What Was Wrong?

Our previous parser treated expressions in isolation, using a "greedy" or shallow parsing approach. It lacked:

• Robust precedence handling

• Context-aware parsing (e.g., dot chains after lambdas)

• Encapsulation of feature-specific logic (e.g., walk blocks)

• And importantly, no tolerance for future growth — small additions caused large breakages.

The Shift: Precedence-Driven Expression Parsing

To fix this, we rewrote core parts of the parser with Pratt-style precedence parsing, and introduced modular expression combinators like:

• parseMaybeLambdaOrPrimary()

• parseChainedExpr()

• parseWalkExpr()

• parseArgList()

This allowed us to separate concerns:

• Primary expressions handle literals, identifiers, and parenthesized expressions.

• Chained expressions handle post-fix operators like .method() or [index].

• Precedence-aware expressions now correctly parse a + b * c - d.

This structure now lets us write:

nums.filter(x -> x > 1)[0].to_str()

... and have it parse without panic

Resilience by Design

Thanks to skipType, isVarDecl, and our block-aware parseProgram, we’re now parsing:

• Variable declarations

• Assignments

• Expressions with arbitrary depth

• Functional flows like list.walk(...) -> {}

• Inline lambdas

...all while supporting comments, optional types, and built-in functions like .get() or .type().

Future-Proofing

This refactor sets us up to confidently add:

• User-defined functions (fn greet(name: str) { ... })

• Pattern matching

• Richer type inference

• Macros or meta constructs

• And even bytecode optimizations at compile time

With a clean, composable parsing structure, we can now build up instead of constantly patching over.