raster

DuckDB-Erweiterung zum Lesen und Schreiben georäumlicher Rasterdaten mit SQL.

Maintainer: ahuarte47

Installation und Laden

INSTALL raster FROM community;
LOAD raster;

Beispiel

-- List available GDAL raster drivers and formats:
SELECT short_name, long_name, help_url FROM RT_Drivers();
┌────────────────┬──────────────────────────────────────────────────────────┬─────────────────────────────────────────────────────┐
│ short_name │ long_name │ help_url │
varcharvarcharvarchar
├────────────────┼──────────────────────────────────────────────────────────┼─────────────────────────────────────────────────────┤
│ VRT │ Virtual Raster │ https://gdal.org/drivers/raster/vrt.html
│ GTiff │ GeoTIFF │ https://gdal.org/drivers/raster/gtiff.html
│ COG │ Cloud optimized GeoTIFF generator │ https://gdal.org/drivers/raster/cog.html
│ · │ · │ · │
│ · │ · │ · │
│ · │ · │ · │
│ ENVI │ ENVI .hdr Labelled │ https://gdal.org/drivers/raster/envi.html
│ Zarr │ Zarr │ NULL
HTTPHTTP Fetching Wrapper │ NULL
└────────────────┴──────────────────────────────────────────────────────────┴─────────────────────────────────────────────────────┘
-- Read a raster file — one row per tile, one datacube column per band:
SELECT * FROM RT_Read('path/to/raster/file.tif');
┌───────┬───────────┬────────────┬────────────────────────────────┬─────────────────────────┬───────┬────────┬────────┬───────┬───────┬────────────┬────────────┐
│ id │ x │ y │ bbox │ geometrylevel │ tile_x │ tile_y │ cols │ rows │ metadata │ databand_1 │
│ int64 │ double │ double │ struct(xmin, ymin, xmax, ymax) │ geometry('epsg:25830') │ int32 │ int32 │ int32 │ int32 │ int32 │ JSON │ BLOB │
├───────┼───────────┼────────────┼────────────────────────────────┼─────────────────────────┼───────┼────────┼────────┼───────┼───────┼────────────┼────────────┤
0545619.754724508.25 │ {xmin: 545539.75, ...} │ POLYGON ((...)) │ 0003208 │ {...} │ ... │
1545619.754724504.25 │ {xmin: 545539.75, ...} │ POLYGON ((...)) │ 0013208 │ {...} │ ... │
└───────┴───────────┴────────────┴────────────────────────────────┴─────────────────────────┴───────┴────────┴────────┴───────┴───────┴────────────┴────────────┘
-- Compute NDVI directly in SQL using band algebra:
WITH __input AS (
SELECT
databand_1 AS red,
databand_2 AS nir
FROM
RT_Read('path/to/raster/file.tif', blocksize_x := 512, blocksize_y := 512)
)
SELECT
RT_Cube2TypeFloat((nir - red) / (nir + red)) AS ndvi
FROM
__input
;
-- Extract pixel values into SQL arrays for aggregate analysis:
WITH __input AS (
SELECT RT_Cube2ArrayFloat(databand_1, true) AS band
FROM RT_Read('path/to/raster/file.tif', blocksize_x := 512, blocksize_y := 512)
)
SELECT
list_min(band.values) AS band_min,
list_stddev_pop(band.values) AS band_stddev,
list_max(band.values) AS band_max
FROM __input
;
-- Write a raster file using COPY ... FORMAT RASTER:
COPY (
SELECT geometry, databand_1, databand_2, databand_3
FROM RT_Read('path/to/raster/file.tif')
)
TO 'path/to/output/file.tif'
WITH (
FORMAT RASTER,
DRIVER 'COG',
CREATION_OPTIONS ('COMPRESS=LZW'),
RESAMPLING 'nearest',
COMPUTE_VALID_ENVELOPE true,
SRS 'EPSG:25830',
GEOMETRY_COLUMN 'geometry',
DATABAND_COLUMNS ['databand_3', 'databand_2', 'databand_1']
);
-- Clip raster tiles to a polygon geometry and write to a new file:
LOAD spatial;
LOAD json;
COPY (
WITH __clip_layer AS (
SELECT geom FROM ST_Read('./test/data/CATAST_Pol_Township-PNA.geojson')
),
__dataset AS (
SELECT id, x, y, tile_x, tile_y, cols, rows, geometry, databand_1, metadata
FROM
__clip_layer,
RT_Read([
'./test/data/mosaic/SCL.tif-land-clip00.tiff',
'./test/data/mosaic/SCL.tif-land-clip01.tiff',
'./test/data/mosaic/SCL.tif-land-clip10.tiff',
'./test/data/mosaic/SCL.tif-land-clip11.tiff'
])
WHERE ST_Intersects(__clip_layer.geom, geometry)
),
__clipped AS (
SELECT
RT_CubeClip(databand_1,
tile_x,
tile_y,
metadata,
__clip_layer.geom,
(metadata->'bands'->0->'nodata')::DOUBLE
) AS databand,
* EXCLUDE(databand_1)
FROM __clip_layer, __dataset
)
SELECT geometry, databand FROM __clipped
)
TO 'path/to/output/clipped.tiff'
WITH (
FORMAT RASTER,
DRIVER 'GTiff',
CREATION_OPTIONS ('COMPRESS=LZW'),
RESAMPLING 'nearest',
SRS 'EPSG:32630',
GEOMETRY_COLUMN 'geometry',
DATABAND_COLUMNS ['databand']
);

Über raster

Erweiterung für DuckDB zum Lesen und Schreiben von Rasterdateien mit SQL.

Der Hauptzweck dieser Erweiterung ist, Rasterdateien als Tabellen in DuckDB zu lesen und anschließend Bandalgebra-Operationen auf den Rasterdaten mit SQL auszuführen. Ein Raster ist ein Datacube, und die Idee ist, ihn als solchen zu manipulieren.

Die Erweiterung verwendet die GDAL-Bibliothek zum Lesen von Rasterdateien und unterstützt eine große Bandbreite an Rasterformaten.

Die Funktion RT_Read ist die Hauptfunktion zum Lesen von Rasterdateien, zum Extrahieren von Metadaten und zum Ausführen grundlegender Operationen auf den Rasterdaten, bevor eine geoparquet-ähnliche DuckDB-Tabelle zurückgegeben wird. Die Funktion nimmt den Pfad zur Rasterdatei sowie weitere optionale Parameter entgegen und gibt eine Tabelle mit folgenden Spalten zurück:

  • id: Eine eindeutige Kennung für jede Kachel.
  • x: Die x-Mittelpunktkoordinate der Kachel.
  • y: Die y-Mittelpunktkoordinate der Kachel.
  • bbox: Die Bounding Box der Kachel als Struct mit den Feldern xmin, ymin, xmax, ymax.
  • geometry: Die Geometrie der Kachel, dargestellt als Polygon.
  • level: Die Zoomstufe der Kachel.
  • tile_x: Die x-Koordinate der Kachel in der Zoomstufe.
  • tile_y: Die y-Koordinate der Kachel in der Zoomstufe.
  • cols: Die Anzahl der Spalten in der Kachel.
  • rows: Die Anzahl der Zeilen in der Kachel.
  • metadata: Ein JSON-Objekt mit den Metadaten der Kachel.
  • databand_x: Ein BLOB mit den Daten des x-ten Bands der Kachel. Anzahl und Namen der Bänder stehen in der Spalte metadata.

Die Funktion RT_Read bietet eine Option datacube, mit der eine einzelne datacube-BLOB-Spalte mit dem n-dimensionalen Array der Daten aller Bänder der Kachel zurückgegeben wird, statt eines BLOBs pro Band.

RT_Read unterstützt Filter-Pushdown auf den Nicht-BLOB-Spalten, sodass Sie die geladenen Kacheln anhand ihrer Metadaten oder räumlichen Lage vorfiltern können. Sie können etwa Kacheln filtern, die eine bestimmte Geometrie schneiden oder einen bestimmten Wert in den Metadaten haben.

Sie können Rasterdaten auch auf Pixelebene mit der Funktion RT_ReadCells lesen, die eine Zeile pro Wertzelle im Raster zusammen mit ihren Pixeln und räumlichen Koordinaten zurückgibt.

Dies ist die Liste der von RT_ReadCells zurückgegebenen Spalten:

  • id ist eine eindeutige Kennung für jede Zelle des Rasters.
  • x die Mittelpunktkoordinaten jeder Zelle im Raster. Das Koordinatenreferenzsystem ist dasselbe wie das der Rasterdatei.
  • y die Mittelpunktkoordinaten jeder Zelle im Raster. Das Koordinatenreferenzsystem ist dasselbe wie das der Rasterdatei.
  • geometry ist die Punktgeometrie jeder Zelle.
  • col ist der Spaltenindex jeder Zelle im Raster, wobei 0 die linkeste Spalte ist.
  • row ist der Zeilenindex jeder Zelle im Raster, wobei 0 die oberste Zeile ist.
  • band_1, band_2, … sind numerische Spalten mit den Pixelwerten einer Zelle. Die Anzahl der Bänder hängt von der Eingabedatei ab, der Datentyp vom Rasterdatentyp.

COPY TO mit FORMAT RASTER erlaubt den Export einer Tabelle mit derselben Struktur wie der von RT_Read zurückgegebenen in eine neue Rasterdatei. Sie können den Treiber und mehrere Erstellungoptionen angeben:

  • DRIVER: The GDAL driver to use for writing the raster file. For example, GTiff or COG.
  • CREATION_OPTIONS: A list of creation options to pass to the GDAL driver, specified as strings in the format KEY=VALUE.
  • RESAMPLING: The resampling method to use when writing the raster file. For example, nearest, bilinear, cubic, etc.
  • ENVELOPE: The spatial extent of the raster file, specified as a list of four values: [xmin, ymin, xmax, ymax].
  • SRS: The spatial reference system of the raster file, specified as an EPSG code (e.g. EPSG:4326) or a WKT string.
  • GEOMETRY_COLUMN: The name of the column that contains the geometry of the tiles. This column will be used to determine the spatial location and the resolution of the tiles in the output raster file.
  • DATABAND_COLUMNS: A list with the names of the columns that contain the data of the bands. The order of the columns in the list will determine the order of the bands in the output raster file.

Hinzugefügte Funktionen

function_name function_type description comment examples
RT_Array2Cube scalar Packages a plain SQL array of numeric values back into a datacube BLOB, the inverse of RT_Cube2Array. NULL [SELECT RT_Array2Cube(r.values, ‘RAW’, r.bands, r.cols, r.rows, r.no_data) FROM …;]
RT_CoordValue scalar Returns the value in a specified band of a datacube at the specified world coordinates (x, y). NULL [SELECT RT_CoordValue(databand_1, 0, 545600.0, 4724500.0, metadata, -9999.0) AS value FROM RT_Read(‘path/to/raster/file.tif’);]
RT_CoordValue_Agg aggregate Returns the value in a set of datacubes at the specified world coordinates (x, y). NULL [SELECT RT_CoordValue_Agg(databand_1, 0, 545600.0, 4724500.0, tile_x, tile_y, metadata, -9999.0) FROM RT_Read(‘path/to/raster/file.tif’);]
RT_CoordValues scalar Returns the values in a band of a datacube at the specified array of world coordinates (x, y). NULL [SELECT RT_CoordValues(databand_1, 0, [545600.0, 545601.0], [4724500.0, 4724501.0], metadata, -9999.0) FROM RT_Read(‘path/to/raster/file.tif’);]
RT_CubeAbs scalar Returns a datacube with the absolute value of each cell. Nodata cells are preserved. NULL [SELECT RT_CubeAbs(databand_1) FROM …;]
RT_CubeAdd scalar Returns a datacube with each cell equal to the sum of the two inputs. Inputs can be two datacubes or a datacube and a scalar. Nodata cells are preserved. NULL [SELECT RT_CubeAdd(databand_1, 10) FROM …;]
RT_CubeBurn scalar Returns a datacube where cells inside the given geometry are replaced by the specified value. No-data cells are preserved. NULL [SELECT RT_CubeBurn(databand_1, tile_x, tile_y, metadata, ST_GeomFromText(‘POLYGON((…))’), 1.0) FROM …;]
RT_CubeClip scalar Returns a datacube where cells outside the given geometry are replaced by the specified value. No-data cells are preserved. NULL [SELECT RT_CubeClip(databand_1, tile_x, tile_y, metadata, ST_GeomFromText(‘POLYGON((…))’), 0.0) FROM …;]
RT_CubeDivide scalar Returns a datacube with each cell equal to the left-hand cell divided by the right-hand cell. Inputs can be two datacubes or a datacube and a scalar. Nodata cells are preserved. NULL [SELECT RT_CubeDivide(databand_1, 10) FROM …;]
RT_CubeEqual scalar Returns a datacube where each cell is 1 if left == right, 0 otherwise. Inputs can be two datacubes or a datacube and a scalar. Nodata cells are preserved. NULL [SELECT RT_CubeEqual(databand_1, 10) FROM …;]
RT_CubeExp scalar Returns a datacube with the exponential (e^x) of each cell. Nodata cells are preserved. NULL [SELECT RT_CubeExp(databand_1) FROM …;]
RT_CubeFill scalar Returns a datacube where all cells (including nodata) are unconditionally replaced by the right-hand value. NULL [SELECT RT_CubeFill(databand_1, 0) FROM …;]
RT_CubeGreater scalar Returns a datacube where each cell is 1 if left > right, 0 otherwise. Inputs can be two datacubes or a datacube and a scalar. Nodata cells are preserved. NULL [SELECT RT_CubeGreater(databand_1, 10) FROM …;]
RT_CubeGreaterEqual scalar Returns a datacube where each cell is 1 if left >= right, 0 otherwise. Inputs can be two datacubes or a datacube and a scalar. Nodata cells are preserved. NULL [SELECT RT_CubeGreaterEqual(databand_1, 10) FROM …;]
RT_CubeLess scalar Returns a datacube where each cell is 1 if left < right, 0 otherwise. Inputs can be two datacubes or a datacube and a scalar. Nodata cells are preserved. NULL [SELECT RT_CubeLess(databand_1, 10) FROM …;]
RT_CubeLessEqual scalar Returns a datacube where each cell is 1 if left <= right, 0 otherwise. Inputs can be two datacubes or a datacube and a scalar. Nodata cells are preserved. NULL [SELECT RT_CubeLessEqual(databand_1, 10) FROM …;]
RT_CubeLog scalar Returns a datacube with the natural logarithm of each cell. Nodata cells are preserved. NULL [SELECT RT_CubeLog(databand_1) FROM …;]
RT_CubeMax scalar Returns a datacube with each cell equal to the maximum of the two inputs. Inputs can be two datacubes or a datacube and a scalar. Nodata cells are preserved. NULL [SELECT RT_CubeMax(databand_1, databand_2) FROM …;]
RT_CubeMin scalar Returns a datacube with each cell equal to the minimum of the two inputs. Inputs can be two datacubes or a datacube and a scalar. Nodata cells are preserved. NULL [SELECT RT_CubeMin(databand_1, databand_2) FROM …;]
RT_CubeMod scalar Returns a datacube with each cell equal to the remainder of dividing the left-hand cell by the right-hand value. Inputs can be two datacubes or a datacube and a scalar. Nodata cells are preserved. NULL [SELECT RT_CubeMod(databand_1, 3) FROM …;]
RT_CubeMultiply scalar Returns a datacube with each cell equal to the product of the two inputs. Inputs can be two datacubes or a datacube and a scalar. Nodata cells are preserved. NULL [SELECT RT_CubeMultiply(databand_1, 10) FROM …;]
RT_CubeNeg scalar Returns a datacube with each cell negated (multiplied by -1). Nodata cells are preserved. NULL [SELECT RT_CubeNeg(databand_1) FROM …;]
RT_CubeNotEqual scalar Returns a datacube where each cell is 1 if left != right, 0 otherwise. Inputs can be two datacubes or a datacube and a scalar. Nodata cells are preserved. NULL [SELECT RT_CubeNotEqual(databand_1, 10) FROM …;]
RT_CubeNullOrEmpty scalar Returns true if the datacube is NULL or contains only nodata cells, false otherwise. NULL [SELECT RT_CubeNullOrEmpty(databand_1) FROM …;]
RT_CubePow scalar Returns a datacube with each cell raised to the power of the right-hand value. Inputs can be two datacubes or a datacube and a scalar. Nodata cells are preserved. NULL [SELECT RT_CubePow(databand_1, 2) FROM …;]
RT_CubeSet scalar Returns a datacube where valid cells are replaced by the right-hand value. Nodata cells in the source are preserved. NULL [SELECT RT_CubeSet(databand_1, 0) FROM …;]
RT_CubeSetNoData scalar Returns a datacube where nodata cells are replaced by the specified value, and sets this value as the new nodata sentinel. NULL [SELECT RT_CubeSetNoData(databand_1, -9999) FROM …]
RT_CubeSqrt scalar Returns a datacube with the square root of each cell. Nodata cells are preserved. NULL [SELECT RT_CubeSqrt(databand_1) FROM …;]
RT_CubeStats scalar Calculates statistics for a specific band (0-based index) of a datacube. Returns a STRUCT with minimum, maximum, sum, mean, stddev, valid_count and nodata_count fields. NULL [SELECT RT_CubeStats(databand_1, 0) AS stats FROM RT_Read(‘path/to/raster/file.tif’);]
RT_CubeStats_Agg aggregate Calculates statistics for a specific band (0-based index) in a set of datacubes. Returns a STRUCT with minimum, maximum, sum, mean, stddev, valid_count and nodata_count fields. NULL [SELECT RT_CubeStats_Agg(databand_1, 0) AS stats FROM RT_Read(‘path/to/raster/file.tif’);]
RT_CubeSubtract scalar Returns a datacube with each cell equal to the left-hand cell minus the right-hand cell. Inputs can be two datacubes or a datacube and a scalar. Nodata cells are preserved. NULL [SELECT RT_CubeSubtract(databand_1, 10) FROM …;]
RT_Drivers table Returns the list of supported GDAL raster drivers and file formats. NULL [SELECT * FROM RT_Drivers();]
RT_Envelope scalar Computes the bounding box of the valid (non-no-data) cells in the input datacube for a specific band and returns it as a geometry. NULL [SELECT RT_Envelope(databand_1, 0, tile_x, tile_y, metadata) AS geometry FROM RT_Read(‘path/to/raster/file.tif’);]
RT_GdalConfig scalar Sets a GDAL configuration option (equivalent to CPLSetConfigOption). Pass NULL as the value to unset the option. NULL [SELECT RT_GdalConfig(‘AWS_NO_SIGN_REQUEST’, ‘YES’);]
RT_Polygon scalar Creates a polygon geometry for each contiguous region of non-no-data values for a specific band in the datacube. NULL [SELECT RT_Polygon(databand_1, 0, tile_x, tile_y, metadata) AS geometry FROM RT_Read(‘path/to/raster/file.tif’);]
RT_RasterValue scalar Returns the value in a specified band of a datacube at the specified pixel coordinates (column, row). NULL [SELECT RT_RasterValue(databand_1, 0, 10, 20, -9999.0) FROM RT_Read(‘path/to/raster/file.tif’);]
RT_RasterValue_Agg aggregate Returns the value in a set of datacubes at the specified pixel coordinates (column, row). NULL [SELECT RT_RasterValue_Agg(databand_1, 0, 0, 20, tile_x, tile_y, metadata, -9999.0) FROM RT_Read(‘path/to/raster/file.tif’);]
RT_RasterValues scalar Returns the values in a band of a datacube at the specified array of pixel coordinates (column, row). NULL [SELECT RT_RasterValues(databand_1, 0, [10, 11], [20, 21], -9999.0) FROM RT_Read(‘path/to/raster/file.tif’);]
RT_Read table Opens a raster file (or a mosaic of raster files) and returns a table with the raster data. Supports filter pushdown on all non-BLOB columns. NULL [SELECT * FROM RT_Read(‘path/to/raster/file.tif’);]
RT_ReadCells table Reads a raster file (or a mosaic of raster files) and returns a table with one row per value cell in the raster. NULL [SELECT id, x, y, geometry, col, row, band_1, band_2, band_3 FROM RT_ReadCells(‘path/to/raster/file.tif’);]
RT_Cube2ArrayInt32 scalar NULL NULL NULL
RT_Cube2TypeUInt64 scalar NULL NULL NULL
RT_Cube2TypeInt8 scalar NULL NULL NULL
RT_Cube2ArrayFloat scalar NULL NULL NULL
RT_Cube2ArrayUInt32 scalar NULL NULL NULL
RT_Cube2ArrayDouble scalar NULL NULL NULL
RT_Cube2TypeUInt8 scalar NULL NULL NULL
RT_Cube2ArrayInt64 scalar NULL NULL NULL
RT_Cube2TypeUInt16 scalar NULL NULL NULL
RT_Cube2TypeDouble scalar NULL NULL NULL
RT_Cube2TypeInt32 scalar NULL NULL NULL
RT_Cube2TypeInt64 scalar NULL NULL NULL
RT_Cube2ArrayInt16 scalar NULL NULL NULL
RT_Cube2ArrayInt8 scalar NULL NULL NULL
RT_Cube2ArrayUInt8 scalar NULL NULL NULL
RT_Cube2TypeFloat scalar NULL NULL NULL
RT_Cube2TypeInt16 scalar NULL NULL NULL
RT_Cube2ArrayUInt16 scalar NULL NULL NULL
RT_Cube2TypeUInt32 scalar NULL NULL NULL
RT_Cube2ArrayUInt64 scalar NULL NULL NULL

Überladene Funktionen

function_name function_type description comment examples
- scalar NULL NULL NULL
/ scalar NULL NULL NULL
+ scalar NULL NULL NULL
* scalar NULL NULL NULL
% scalar NULL NULL NULL
^ scalar NULL NULL NULL

Hinzugefügte Typen

type_name type_size logical_type type_category internal
RT_BBOX 0 STRUCT COMPOSITE true
RT_DATACUBE 16 BLOB NULL true
RT_STATS 0 STRUCT COMPOSITE true

Hinzugefügte Einstellungen

Diese Erweiterung fügt keine Einstellungen hinzu.