lpts

Optimized-plan inspection and cross-system SQL transpilation

Maintainer(s): ila

Installing and Loading

INSTALL lpts FROM community;
LOAD lpts;

Example

-- Set both input and output dialects to DuckDB for a first round trip.
D SET lpts_input_dialect = 'duckdb';
D SET lpts_dialect = 'duckdb';
D CREATE TABLE users (id INTEGER, name VARCHAR, age INTEGER);
D INSERT INTO users VALUES (1, 'Alice', 30), (2, 'Bob', 22), (3, 'Carol', 28);
-- Return the optimized plan as readable CTE SQL.
D PRAGMA lpts('SELECT name FROM users WHERE age > 25');
WITH
t0_scan (t0_name) AS (
SELECT "name"
FROM memory.main.users
WHERE (age>25)
)
SELECT t0_name AS "name"
FROM t0_scan;
-- The same output, exposed as a first-class EXPLAIN statement.
D EXPLAIN (FORMAT SQL) SELECT name FROM users WHERE age > 25;
┌─────────────────────────────┐
│┌───────────────────────────┐│
││ Optimized Logical Plan ││
│└───────────────────────────┘│
└─────────────────────────────┘
WITH
t0_scan (t0_name) AS (
SELECT "name"
FROM memory.main.users
WHERE (age>25)
)
SELECT t0_name AS "name"
FROM t0_scan;
-- Turn on transparent round-trip checking. Every top-level SELECT is now
-- rewritten by LPTS and its result compared against the original; a wrong
-- rewrite raises an error. The query otherwise returns its normal rows.
D SET lpts_check = true;
D SELECT name FROM users WHERE age > 25;
┌─────────┐
name
varchar
├─────────┤
│ Alice │
│ Carol │
└─────────┘

About lpts

LPTS is a DuckDB extension for optimized-plan inspection and cross-system SQL transpilation. LPTS takes DuckDB’s post-optimizer logical plan and reconstructs equivalent SQL as a sequence of named CTEs.

PRAGMA Syntax

PRAGMA lpts('<query>');

EXPLAIN (FORMAT SQL)

LPTS also extends DuckDB’s EXPLAIN with a SQL format. EXPLAIN (FORMAT SQL) <query> returns the optimized logical plan rendered as equivalent CTE SQL — the same output as PRAGMA lpts, but exposed as a first-class EXPLAIN statement. The CLI prints the SQL as plain multi-line text, while JDBC, Python, and other clients receive the standard two-column (explain_key, explain_value) EXPLAIN result. It honors lpts_dialect just like PRAGMA lpts.

EXPLAIN (FORMAT SQL) SELECT name FROM users WHERE age > 25;
┌─────────────────────────────┐
│┌───────────────────────────┐│
││ Optimized Logical Plan ││
│└───────────────────────────┘│
└─────────────────────────────┘
WITH
t0_scan (t0_name) AS (
SELECT "name"
FROM memory.main.users
WHERE (age>25)
)
SELECT t0_name AS "name"
FROM t0_scan;

LPTS plans the query through DuckDB, optimizes it, then serializes the optimized logical plan.

Supported Dialects

The dialect settings accept these values:

Dialect Accepted values
DuckDB duckdb
PostgreSQL postgres, postgresql
Spark SQL spark
Hive hive
Trino / Presto trino, presto
Snowflake snowflake
BigQuery bigquery, bq
Redshift redshift
MySQL / MariaDB mysql, mariadb

Use Cases

  • Inspect optimized DuckDB plans as SQL.
  • Debug optimizer rewrites such as filter pushdown, join reordering, top-N, materialized CTEs, and subquery decorrelation.
  • Generate a CTE program that communicates the optimized execution shape.
  • Emit SQL for another engine with lpts_dialect.
  • Convert other SQL dialect syntax to DuckDB SQL with lpts_input_dialect, then execute or inspect it.

Supported Operators

LPTS is intended to cover all logical operators produced by optimized DuckDB SELECT plans. The current regression suite round-trips all 22 TPC-H queries and exercises joins, aggregates, windows, set operations, CTEs, recursive CTEs, table functions, DuckLake scans, and inserts.

Unsupported optimizer edge cases fail explicitly with NotImplementedException.

Examples

D CREATE TABLE events (id INTEGER, ts TIMESTAMP, name VARCHAR, "order" INTEGER);
D INSERT INTO events VALUES
(1, TIMESTAMP '2024-01-15 08:09:10', 'alpha', 10),
(11, TIMESTAMP '2024-01-16 11:12:13', 'beta', 20);
-- Render generated SQL for PostgreSQL.
D SET lpts_dialect = 'postgres';
-- Return generated CTE SQL directly in the shell.
D PRAGMA lpts(
'SELECT strftime(ts, ''%Y-%m-%d'') AS day
FROM events
WHERE id > 10
ORDER BY day'
);
-- Return generated CTE SQL as a table row, useful in scripts and tests.
D SELECT sql
FROM lpts_query(
'SELECT strftime(ts, ''%Y-%m-%d'') AS day
FROM events
WHERE id > 10
ORDER BY day'
);
WITH
t0_scan (ts) AS (
SELECT ts
FROM events
WHERE (id>10)
)
SELECT to_char(ts, 'YYYY-MM-DD') AS "day"
FROM t0_scan
ORDER BY to_char(ts, 'YYYY-MM-DD') ASC NULLS LAST;
-- Switch back to DuckDB rendering.
D SET lpts_dialect = 'duckdb';
-- Turn on transparent round-trip checking, then run the query directly.
-- LPTS rewrites every top-level SELECT and compares its result bag against
-- the original, raising on a mismatch; the query returns its normal rows.
D SET lpts_check = true;
D SELECT name FROM events WHERE id > 10 ORDER BY name;
┌─────────┐
│ name │
│ varchar │
├─────────┤
│ beta │
└─────────┘
-- Print the AST tree to stdout for interactive debugging.
D PRAGMA print_ast('SELECT name FROM events WHERE id > 10 ORDER BY name');
-- Return the AST tree as a table row, useful for tools and regression tests.
D SELECT ast
FROM print_ast_query('SELECT name FROM events WHERE id > 10 ORDER BY name');
-- Normalize MySQL syntax before DuckDB parses and plans the query.
D SET lpts_input_dialect = 'mysql';
-- Return source-dialect SQL normalized to DuckDB SQL.
D SELECT sql
FROM lpts_normalize_query(
'SELECT `order`, DATE_FORMAT(ts, ''%Y-%m-%d %H:%i:%s'') AS formatted FROM events LIMIT 5, 10'
);
SELECT "order", strftime(ts, '%Y-%m-%d %H:%M:%S') AS formatted FROM events LIMIT 10 OFFSET 5

Added Functions

function_name function_type description comment examples
lpts pragma Return the optimized logical plan for a query as equivalent CTE SQL. NULL [PRAGMA lpts(‘SELECT name FROM users WHERE age > 25’)]
lpts_normalize_query table Normalize SQL from lpts_input_dialect into DuckDB SQL without planning it. NULL [SET lpts_input_dialect = ‘mysql’; SELECT sql FROM lpts_normalize_query(‘SELECT order FROM users LIMIT 1, 2’)]
lpts_query table Table-function form of PRAGMA lpts. Returns the generated CTE SQL as a single sql column. NULL [SELECT sql FROM lpts_query(‘SELECT name FROM users WHERE age > 25’)]
print_ast pragma Print the LPTS AST tree for a query to stdout. NULL [PRAGMA print_ast(‘SELECT name FROM users WHERE age > 25’)]
print_ast_query table Table-function form of PRAGMA print_ast. Returns the rendered AST tree as a single ast column. NULL [SELECT ast FROM print_ast_query(‘SELECT name FROM users WHERE age > 25’)]

Overloaded Functions

This extension does not add any function overloads.

Added Types

This extension does not add any types.

Added Settings

name description input_type scope aliases
lpts_check When true, transparently verify every top-level SELECT: the query returns its normal result, but LPTS rewrites it and, on a different result, raises ‘LPTS check failed’ (an unsupported query raises a specific ‘LPTS check: unsupported’ error). If the LPTS_CHECK_LOG environment variable points to a file, it instead logs one ‘<query#> OK|WRONG|UNSUPPORTED|FAIL’ line per SELECT and never raises (suite-coverage mode). BOOLEAN GLOBAL []
lpts_dialect SQL dialect for lpts output. Valid values: ‘duckdb’ (default), ‘postgres’, ‘spark’, ‘hive’, ‘trino’, ‘presto’, ‘snowflake’, ‘bigquery’, ‘redshift’, ‘mysql’, ‘mariadb’ VARCHAR GLOBAL []
lpts_enable_data_dependent_optimizers Enable LPTS planning optimizers that depend on current data, statistics, cardinality estimates, row groups, or runtime dynamic filters. BOOLEAN GLOBAL []
lpts_input_dialect SQL dialect for lpts input normalization. Valid values: ‘duckdb’ (default), ‘postgres’, ‘spark’, ‘hive’, ‘trino’, ‘presto’, ‘snowflake’, ‘bigquery’, ‘redshift’, ‘mysql’, ‘mariadb’ VARCHAR GLOBAL []
lpts_merge_pipeline Fuse chains of single-child pipeline operators (Limit/OrderBy/Project/Aggregate/Filter, and pushdown-free base-table scans) into one flat SELECT per query block instead of one CTE per operator. BOOLEAN GLOBAL []