Supported Formats

Beacon finds the datasets in its storage root automatically. You register nothing. Copy your files into the datasets folder or the S3 prefix. You can then query them at once. Use a reader function or an external table.

The default local path in the Docker container is /beacon/data/datasets/.

Format support matrix

Format	Recognized files	STORED AS	read_* function	Output format
Parquet	.parquet	PARQUET	read_parquet	yes
GeoParquet	.geoparquet	GEOPARQUET	read_geoparquet	yes
CSV / TSV	.csv, .tsv	CSV	read_csv	yes
Arrow IPC	.arrow, .feather	ARROW	read_arrow	yes (ipc)
NetCDF	.nc	NC	read_netcdf	yes (+ ND-NetCDF)
Zarr	zarr.json marker	ZARR	read_zarr	no
Atlas	atlas.json marker	ATLAS	read_atlas	no
GeoTIFF / COG	.tif, .tiff	TIFF	read_tiff	no
BBF	.bbf	BBF	read_bbf	no
Delta Lake	_delta_log/ directory	DELTA	read_delta	no
Apache Iceberg	metadata/ directory	ICEBERG	read_iceberg	no
ODV ASCII	.txt	not supported	read_odv_ascii	yes

Beacon finds every format above in the datasets store. Delta Lake, Apache Iceberg and ODV ASCII are the exception. Point a read_* function at them. For Delta and Iceberg, CREATE EXTERNAL TABLE … STORED AS DELTA|ICEBERG LOCATION … also works. The "Output format" column marks the formats that output.format can export a query result to.

Per-format reference

Each format has its own chapter. The chapter covers the read behaviour, the attribute columns, the limitations and the tuning. See External Files.

Where files live

The datasets store is a local directory or an S3-compatible bucket:

• Local disk: the files under the datasets folder. Mount that folder into the container.
• Object storage: an S3, GCS or Azure prefix. See Object Storage for the credentials and the setup.

Beacon reads the files in place. Beacon never copies or converts them. A new file is queryable at once. Use a crawler to register many datasets under a prefix as named tables in one step. A crawler also handles a partitioned layout.

Next

• External Files: the read reference for each format.
• Create External Tables: give a set of files a stable table name.
• Performance Tuning: the layout and format choices that make a scan faster.