hostfs

Navigate and explore the filesystem using SQL

Maintainer(s): Gropaul

Installing and Loading

INSTALL hostfs FROM community;
LOAD hostfs;

Example

-- Navigate to the workspace and list the files
D PRAGMA cd('/Users/paul/workspace');
D PRAGMA ls;
┌───────────────────────────────┐
path
varchar
├───────────────────────────────┤
│ ./duckdb │
│ ./playground │
│ ./hostfs │
-- Find the files you were working on last
D SELECT path, file_last_modified(path) AS date FROM ls() WHERE 'csv' IN file_extension(path) ORDER BY date LIMIT 1 ;
┌───────────────────────────┬─────────────────────┐
pathdate
varchartimestamp
├───────────────────────────┼─────────────────────┤
│ ./sketch_results_join.csv2024-07-13 23:25:48
└───────────────────────────┴─────────────────────┘
-- List the top 3 file types by total size, with file count, ordered by size.
D SELECT size, count, file_extension AS "type"
FROM (
SELECT SUM(file_size(path)) AS size_raw, format_bytes(size_raw) AS size, COUNT(*) AS count, file_extension(path) AS file_extension
FROM lsr('/Users/paul/workspace', 10)
GROUP BY file_extension(path)
) AS subquery
ORDER BY size_raw DESC LIMIT 3;
┌───────────┬───────┬─────────┐
size │ count │ type
varchar │ int64 │ varchar
├───────────┼───────┼─────────┤
246.95 GB │ 29 │ .duckdb │
90.33 GB │ 3776 │ .tmp │
26.17 GB │ 28175 │ .csv │
└───────────┴───────┴─────────┘

About hostfs

The HostFS extension allows you to navigate and explore the filesystem using SQL. It provides a set of functions to list files, get file metadata, and more. For more information, please see the HostFS documentation.

Added Functions

function_name function_type description comment examples
absolute_path scalar NULL NULL
cd pragma NULL NULL
cd table NULL NULL
file_extension scalar NULL NULL
file_last_modified scalar NULL NULL
file_name scalar NULL NULL
file_size scalar NULL NULL
hostfs scalar NULL NULL
hsize scalar NULL NULL
is_dir scalar NULL NULL
is_file scalar NULL NULL
ls pragma NULL NULL
ls table NULL NULL
lsr pragma NULL NULL
lsr table NULL NULL
path_exists scalar NULL NULL
path_separator scalar NULL NULL
path_split macro NULL NULL
path_type scalar NULL NULL
pwd pragma NULL NULL
pwd scalar NULL NULL

Overloaded Functions

This extension does not add any function overloads.

Added Types

This extension does not add any types.

Added Settings

This extension does not add any settings.