Delta Lake

Experimental

Creates:

Assets

This plugin can be configured directly in the Marmot UI with a step-by-step wizard.

Example Configuration

table_paths:
  - "/data/delta/events"
  - "/data/delta/users"
tags:
  - "delta-lake"

The following configuration options are available:

Property	Type	Required	Description
external_links	[]ExternalLink	false	External links to show on all assets
filter	Filter	false	Filter discovered assets by name (regex)
table_paths	[]string	false	Paths to Delta Lake table directories
tags	TagsConfig	false	Tags to apply to discovered assets

The following metadata fields are available:

Field	Type	Description
created_time	int64	Table creation timestamp in milliseconds
current_version	int64	Current Delta log version
format	string	Data format (e.g. parquet)
location	string	Table directory path
min_reader_version	int	Minimum reader protocol version
min_writer_version	int	Minimum writer protocol version
num_files	int	Number of active data files
partition_columns	string	Comma-separated partition column names
schema_field_count	int	Number of schema fields
table_id	string	Delta table unique identifier
total_size	int64	Total size of active data files in bytes