Mastodon Politics, Power, and Science: Implementation Plan: Inline Expression Evaluation

Monday, August 4, 2025

Implementation Plan: Inline Expression Evaluation

 The problem

right now to build a conditional agent like if then else, the inputs would have to spit the expression up into strings and then interally evaluate the conditional from the parts inside python.

The solution is to allow the input to just evaluate "$variable >  5" at runtime for an input.  This greatly simplifies the design of agents and adds a lot of power to  the entire core with one simple change.


Phase 1: Defining the Safe Operations and Precedence

The foundation of a secure parser is a whitelist of allowed operations and a clear order of execution.

  1. Modify 

    • At the top of the script (or within a new "Parser" section), define two constants. These will be the heart of your new parser.

    • SAFE_OPERATORS: A dictionary mapping operator symbols (strings) to their corresponding functions from Python's operator module.

    • PRECEDENCE: A dictionary mapping operator symbols to an integer precedence level. Higher numbers mean higher precedence.

    • Code:

      Generated python
      import operator
      
      # --- Expression Parser Constants ---
      SAFE_OPERATORS = {
          # Arithmetic
          '+': operator.add, '-': operator.sub,
          '*': operator.mul, '/': operator.truediv,
          '%': operator.mod, '**': operator.pow,
          # Comparison
          '==': operator.eq, '!=': operator.ne,
          '<': operator.lt, '<=': operator.le,
          '>': operator.gt, '>=': operator.ge,
          # Logical (optional but very useful)
          'and': operator.and_, 'or': operator.or_,
      }
      
      PRECEDENCE = {
          '(': 1, ')': 1,
          'or': 2, 'and': 3,
          '==': 4, '!=': 4, '<': 4, '<=': 4, '>': 4, '>=': 4,
          '+': 5, '-': 5,
          '*': 6, '/': 6, '%': 6,
          '**': 7,
      }

Phase 2: Implementing the Shunting-Yard Algorithm

This is the classic algorithm for converting an infix expression (like 3 + 4 * 2) into a postfix expression (Reverse Polish Notation or RPN: 3 4 2 * +), which is trivial to evaluate. This algorithm naturally handles operator precedence and parentheses.

  1. Create a New Helper Function: 

    • This function will be the "parser" that builds the evaluatable structure.

    • Algorithm (Shunting-Yard):

      1. Initialize an empty list for output (output_queue) and an empty list for operators (operator_stack).

      2. Tokenize the input string (e.g., split "10 + 2 * 3" into ['10', '+', '2', '*', '3']). You will need a robust tokenizer that can handle numbers, operators, and parentheses.

      3. For each token:

        • If it's a number, add it to output_queue.

        • If it's an operator, while there's an operator on the stack with higher or equal precedence, pop it from the stack to the output_queue. Then push the current operator onto the operator_stack.

        • If it's a left parenthesis (, push it onto the operator_stack.

        • If it's a right parenthesis ), pop operators from the stack to the output_queue until you hit a left parenthesis. Pop the left parenthesis but don't add it to the queue.

      4. When all tokens are read, pop any remaining operators from the operator_stack to the output_queue.

    • Return: The output_queue now holds the expression in RPN.

Phase 3: Implementing the RPN Evaluator

Evaluating an RPN expression is extremely simple and efficient, requiring only a single stack.

  1. Create a New Helper Function: 

    • This function takes the RPN queue from the previous step.

    • Algorithm (RPN Evaluation):

      1. Initialize an empty list for values (value_stack).

      2. For each token in the RPN queue:

        • If the token is a number (operand), push it onto the value_stack.

        • If the token is an operator, pop the required number of operands from the value_stack (e.g., two for binary operators like +), apply the operator function from SAFE_OPERATORS, and push the result back onto the value_stack.

      3. After all tokens are processed, the value_stack will contain exactly one item: the final result.

    • Return: The final result from the value_stack.

Phase 4: Integrating into 

This is the final step where you replace the old, simplistic logic with the new, powerful parser.

  1. Refactor 

    • The function signature remains the same.

    • New Logic:

      Generated python
      import re
      
      def resolve_value(value: any, scoped_params: dict) -> any:
          if not isinstance(value, str):
              return value
      
          # Step 1: Substitute all $variables in the string.
          # Use a regex to find and replace all instances of $var.
          # Example: "$var1 + 5" becomes "10 + 5" if var1 is 10.
          substituted_string = _substitute_variables(value, scoped_params) # You'll need this helper
      
          # Step 2: Try to parse and evaluate the result.
          try:
              # Tokenize the substituted string into numbers, operators, etc.
              tokens = _tokenize(substituted_string) # You'll need a tokenizer helper
              
              # Convert to Reverse Polish Notation
              rpn_queue = _parse_to_rpn(tokens)
              
              # Evaluate the RPN expression
              result = _evaluate_rpn(rpn_queue)
              
              return result
      
          except (ValueError, TypeError, IndexError, KeyError):
              # If ANY step of parsing/evaluation fails, it means the string
              # is not a valid expression (e.g., "Hello World").
              # In this case, we fall back to returning the string with
              # variables substituted, which is the old behavior.
              return substituted_string

Plan Summary & Benefits

This plan provides a robust, secure, and maintainable path to full expression evaluation.

  • Secure by Design: It uses a whitelisted set of operators, completely avoiding the risks of eval().

  • Correct by Algorithm: Using the standard Shunting-Yard and RPN evaluation algorithms ensures mathematical correctness for precedence and parentheses.

  • Localized Change: The entire implementation is contained within a few new helper functions and a single modification to resolve_value. No other part of the core engine needs to be aware of this change.

  • Graceful Fallback: The try...except block ensures that strings not meant to be expressions (like plain text or XML tags) are handled correctly and returned as-is, preserving all existing functionality.

No comments:

Post a Comment

Progress on the campaign manager

You can see that you can build tactical maps automatically from the world map data.  You can place roads, streams, buildings. The framework ...