Dynamic parametrization

Sequor supports two methods for dynamically generating values in your flows: Jinja templates and Python functions. These methods allow you to create flexible, data-driven flows.

Parametrization methods

Jinja templates

You can inject Jinja expressions into any string value within your YAML configuration. Sequor uses standard Jinja 2 syntax. For comprehensive documentation on Jinja templates, please refer to the official Jinja documentation .

Sequor support the following built-in functions:

An example of using Jinja:

url: "https://api.bigcommerce.com/stores/{{ var('store_hash') }}/{{ var('api_version') }}/customers"

Using Jinja templates with in-operation iterators

If you use Jinja templates in operations that have built-in iterators (e.g. http_request iterates over source table records and executes while loop in response parsing), Jinja templates are rendered on each iteration. This allows you to use Jinja templates with variables that are modified during iterations.

In the following example, the http_request operation iterates over the input table and a Jinja template is used to dynamically set the id:in URL parameter to the id of column of the current input record.

- op: http_request
    id: delete_inserted_customers
    for_each:
      source: "stage"
      table: "{{ var('input_table') }}"
      as: customer
    request:
      source: "bigcommerce_rest"
      url: "https://api.bigcommerce.com/stores/{{ var('store_hash') }}/{{ var('api_version') }}/customers"
      method: DELETE
      parameters:
        "id:in": "{{ var('customer').get('id') }}"
      headers:
        "Content-Type": "application/json"
    response:
      parser_expression: |
        def evaluate(context, response):
          status_code = response.status_code
          if status_code != 204:
            raise Exception(f"Error status code: {status_code} deleting record '{var('customer').get('id')}' body: {response.text}")
          return {
          }        

Python functions

For more complex logic, any YAML parameter can be computed dynamically by adding an _expression suffix to the parameter name. Such expression is defined as a Python function that is executed at runtime and returns the value for the YAML parameter.

The Python function must follow this pattern:

def evaluate(context, ...additional_parameters):
    # Your code here
    return result # Returns the value of the YAML parameter

The context parameter provides access to variables and other runtime information. Additional parameters may be available depending on the operation and YAML parameter.

Available context methods

The context object provides the following methods:

Using Jinja templates in _expression YAML parameters

You can use Jinja templates in the value of _expression parameters, but because the evaluate function is compiled once at the beginning of the operation execution, Jinja templates are also evaluated only once at the start. This means that if you configure an operation with builtin iterators, you cannot assume that Jinja expressions will be evaluated on each iterations: variables referenced in Jinja expression will have the values from the beginning of the operation, not values modified during iterations.

Example: Using both methods

The following example demonstrates both parametrization methods in an http_request operation:

- op: http_request
  request:
    source: "bigcommerce_rest"
    url: "https://api.bigcommerce.com/stores/{{ var('store_hash') }}/{{ var('api_version') }}/customers"
    method: POST
    headers:
      "Content-Type": "application/json"
    body_format: json
    body:
      "first_name": "{{ var('customer').get('first_name') }}"
      "last_name": "{{ var('customer').get('last_name') }}"
      "email": "{{ var('customer').get('email') }}"
    # Equivalent version using body_expression
    # body_expression: |
    #   def evaluate(context):
    #     return {
    #       "first_name": context.var("customer").get("first_name"),
    #       "last_name": context.var("customer").get("last_name"),
    #       "email": context.var("customer").get("email")
    #     }
  response:
    parser_expression: |
      def evaluate(context, response):
        if response.status_code != 200:
          raise Exception(f"Error status code: {response.status_code}, body: {response.text}")
        customers_created = response.json()['data']
        customers_created['source_id'] = context.var("customer").get("id")
        return {
          "tables": [{
            "source": "postgres",
            "table": "bc_customers_inserted",
            "data": customers_created,
            "model": {
              "columns": [
                {"name": "id", "type": "text"}, # newly created customer id
                {"name": "source_id", "type": "text"} # id of the input record
              ]
            }
          }]
        }

In the example above, the body parameter demonstrates the Jinja template approach, while the commented-out body_expression shows the equivalent Python function approach. Both achieve the same result but use different syntax.

The parser_expression demonstrates a more complex Python function that processes an API response and transforms it into structured data for database insertion.