transform operation

The transform operation executes a SQL query in a source and stores the result in a table within the same source. For efficiency, transform internally generates and execute a “CREATE TABLE … AS …” statement, preventing data from passing through Sequor.

Parameters are:

• id (type: str, optional) - unique identifier of the operation in the flow
• source (type: str, required) - source name where the query will be executed
• query (type: str, required) - SQL query to execute
• target_database (type: str, optional) - database name where the table will be created
• target_namespace (type: str, optional) - namespace name where the table will be created
• target_table (type: str, required) - name of the table to be created

If a table with the specified name already exists, it will be dropped and recreated.

Examples

- op: transform
  source: "postgres"
  query: |
    select *, concat(first_name, ' ', last_name) as full_name
    from customers
  target_table: customers_transformed