Support Interpolated String Literals

[nikomatsakis]

Support Interpolated String Literals

Status: In Progress

Current Understanding

Implement interpolated string literals in Dada, including both single and triple quoted variants with automatic dedenting for multiline strings.

Next Steps

• Implement triple-quoted strings with three-quote delimiters
• Ensure triple quotes are not parsed as empty string + quote
• Handle embedded quotes without escaping
• Implement string interpolation with brace syntax
• Add lexer changes to recognize braces within strings
• Track brace nesting for complex expressions
• Update AST representation for interpolated strings
• Add type checking for interpolation expressions

Open Questions

• None currently - design decisions finalized in RFC-0001

Context

RFC-0001 drafted with design tenets: Do what I mean, Rust-like syntax, Simple escape hatch.
Key design decisions:

• Interpolation is default behavior (no special syntax needed)
• Brace syntax for interpolation (Rust-like approach)
• Multiline strings auto-dedent when starting with newline
• Backslash prefix disables all "magic" (dedenting, eventually interpolation)

Parser documentation completed. Escape sequence processing bug fixed - TokenText interned struct now stores processed literal content.

[nikomatsakis]

Session summary (2025-01-06 - Initial RFC and Parser Documentation):

• Renamed RFC directory from 0001-interpolated-strings to 0001-string-literals
• Drafted RFC-0001 with comprehensive design including:
  • Design tenets (Do what I mean, Rust-like syntax, Simple escape hatch)
  • Single and triple-quoted string literals
  • Multiline string support with automatic dedenting
  • Backslash escape hatch for raw multiline strings
  • Executable examples using assert syntax
• Created parser overview documentation in dada-parser/docs/overview.md

Impact on approach:

• Established clear design principles to guide implementation
• Documented existing parser patterns for reference
• Commitment model and parse vs eat distinction clarified

Progress: RFC foundation established. Next: detailed parser component documentation.

[nikomatsakis]

Session summary (2025-06-15 - Parser Documentation and Bug Fix):

• Completed comprehensive parser documentation:
  • Tokenizer: documented string_literal() method and token types
  • Parse trait: documented commitment model and error handling
  • Expression parsing: documented literal handling in base_expr_precedence()

Critical bug discovered and fixed:

• Issue: String escape sequences were validated but never interpreted
• Solution: Added TokenText interned struct for processed literal content
• Changes: TokenKind::Literal now uses TokenText<'db> instead of raw string
• Tokenizer now processes escape sequences when creating TokenText
• All tests passing, escape sequences work correctly

Impact on approach:

• Found existing escape sequence bug that needed fixing first
• Technical foundation now solid for implementing new string features
• Used Salsa interned structs to store processed strings while keeping tokens Copy

Progress: Parser foundation complete and debugged. Ready for triple-quoted strings implementation.