Skip to content

feat(sink): add Redis Strings sink#22

Merged
psteinroe merged 10 commits into
mainfrom
feat/sink-redis-strings
Jan 13, 2026
Merged

feat(sink): add Redis Strings sink#22
psteinroe merged 10 commits into
mainfrom
feat/sink-redis-strings

Conversation

@psteinroe
Copy link
Copy Markdown
Owner

@psteinroe psteinroe commented Jan 7, 2026

Summary

  • Implements a Redis sink that stores events as string key-value pairs
  • Sends event.payload as JSON string to Redis
  • Supports dynamic key routing via event metadata

Configuration

sink:
  type: redis-strings
  url: redis://localhost:6379
  key_prefix: pgstream  # optional fallback prefix

Routing

The Redis key is resolved from:

  1. key field in event metadata (from subscription's metadata or metadata_extensions)
  2. Fallback to event ID with optional key_prefix
-- Dynamic key from row data
metadata_extensions = '[{"json_path": "key", "expression": "''user:'' || new.user_id::text"}]'

@psteinroe psteinroe force-pushed the feat/sink-anysink-infrastructure branch from 80a1ebf to 9b72e54 Compare January 8, 2026 11:26
Implements a Redis sink that stores events as string key-value pairs.
Each event is stored with its ID as the key and JSON payload as value.

- Add `sink-redis-strings` feature flag
- Add `RedisStringsSink` with `url` and optional `key_prefix` config
- Add `ensure_redis()` test helper for testcontainers
- Add integration tests verifying publish and prefix functionality
- Add RedisStringsSinkConfigWithoutSecrets for safe serialization
- Add From implementations for config conversion
- Document payload_extensions usage in module docs
- Update config doc comments to clarify sensitive data handling
- Add resolve_key() for dynamic key from metadata key "key"
- Key fallback: metadata key -> event ID (with optional prefix)
- Remove info! logging
- Use DestinationError for publish failures
- Update tests to verify metadata-based key routing
@psteinroe psteinroe force-pushed the feat/sink-redis-strings branch from 06dc115 to cfc0445 Compare January 8, 2026 11:48
@psteinroe psteinroe changed the base branch from feat/sink-anysink-infrastructure to main January 8, 2026 13:20
psteinroe and others added 3 commits January 8, 2026 14:21
🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Iterate over events by value to consume the Vec and free memory
incrementally as each event is processed.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@psteinroe psteinroe merged commit 8215dec into main Jan 13, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant