Redpanda

Experimental

Creates:

Assets

Configure in the UI

This plugin can be configured directly in the Marmot UI with a step-by-step wizard.

View Guide

The Redpanda plugin discovers topics from Redpanda clusters. It uses the same discovery engine as the Kafka plugin since Redpanda is Kafka API-compatible.

Because it is the same engine, topics are catalogued under the Kafka provider and addressed as mrn://topic/kafka/<topic>, not under a Redpanda provider. Running Kafka, Redpanda and Confluent Cloud against clusters that share a topic name will therefore land them on one asset.

Connection

Redpanda Cloud

bootstrap_servers: "seed-xxxxx.cloud.redpanda.com:9092"
client_id: "marmot-discovery"
authentication:
  type: "sasl_ssl"
  username: "your-username"
  password: "your-password"
  mechanism: "SCRAM-SHA-256"
tls:
  enabled: true

Self-Hosted Redpanda

bootstrap_servers: "redpanda-0.example.com:9092,redpanda-1.example.com:9092"
client_id: "marmot-discovery"

Example Configuration

bootstrap_servers: "kafka-1.prod.com:9092,kafka-2.prod.com:9092"
client_id: "marmot-discovery"
authentication:
  type: "sasl_ssl"
  username: "your-api-key"
  password: "your-api-secret"
  mechanism: "PLAIN"
tls:
  enabled: true
tags:
  - "kafka"
  - "streaming"

Configuration

The following configuration options are available:

Property	Type	Required	Description
authentication	AuthConfig	false	Authentication configuration
bootstrap_servers	string	false	Comma-separated list of bootstrap servers
client_id	string	false	Client ID for the consumer
client_timeout_seconds	int	false	Request timeout in seconds
consumer_config	map[string]string	false	Additional consumer configuration
external_links	[]ExternalLink	false	External links to show on all assets
filter	Filter	false	Filter discovered assets by name (regex)
include_partition_info	bool	false	Whether to include partition information in metadata
include_topic_config	bool	false	Whether to include topic configuration in metadata
schema_registry	SchemaRegistryConfig	false	Schema Registry configuration
tags	TagsConfig	false	Tags to apply to discovered assets
tls	TLSConfig	false	TLS configuration

Available Metadata

The following metadata fields are available:

Field	Type	Description
cleanup_policy	string	Topic cleanup policy
delete_retention_ms	string	Time to retain deleted segments in milliseconds
key_schema	string	Key schema definition
key_schema_id	int	ID of the key schema in Schema Registry
key_schema_type	string	Type of the key schema (AVRO, JSON, etc.)
key_schema_version	int	Version of the key schema
max_message_bytes	string	Maximum message size in bytes
min_insync_replicas	string	Minimum number of in-sync replicas
partition_count	int32	Number of partitions
replication_factor	int16	Replication factor
retention_bytes	string	Maximum size of the topic in bytes
retention_ms	string	Message retention period in milliseconds
segment_bytes	string	Segment file size in bytes
segment_ms	string	Segment file roll time in milliseconds
topic_name	string	Name of the Kafka topic
value_schema	string	Value schema definition
value_schema_id	int	ID of the value schema in Schema Registry
value_schema_type	string	Type of the value schema (AVRO, JSON, etc.)
value_schema_version	int	Version of the value schema