Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Kafka Connect: route to table using topic name #11163

Open
1 of 3 tasks
bryanck opened this issue Sep 18, 2024 · 0 comments
Open
1 of 3 tasks

Kafka Connect: route to table using topic name #11163

bryanck opened this issue Sep 18, 2024 · 0 comments
Labels
improvement PR that improves existing functionality KAFKACONNECT

Comments

@bryanck
Copy link
Contributor

bryanck commented Sep 18, 2024

Feature Request / Improvement

Currently the Kafka Connect sink supports writing to multiple tables, and you can configure the sink to route a record to a table based on a field value. A common use case of the sink is ingesting from multiple source topics and writing to multiple tables, routing records to tables based on the source topic rather than a field value. This can be achieved by adding the topic name to the record via an SMT, but a better approach would be to use the record's topic without having to add a field.

Query engine

None

Willingness to contribute

  • I can contribute this improvement/feature independently
  • I would be willing to contribute this improvement/feature with guidance from the Iceberg community
  • I cannot contribute this improvement/feature at this time
@bryanck bryanck added improvement PR that improves existing functionality KAFKACONNECT labels Sep 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
improvement PR that improves existing functionality KAFKACONNECT
Projects
None yet
Development

No branches or pull requests

1 participant