Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create a demo of reading from kafka in a data job #3140

Open
murphp15 opened this issue Feb 21, 2024 · 0 comments
Open

Create a demo of reading from kafka in a data job #3140

murphp15 opened this issue Feb 21, 2024 · 0 comments
Assignees
Labels
enhancement New feature or request initiative: VDK for Private AI Initiative including the effort to support Private AI usecases of VMWare with VDK

Comments

@murphp15
Copy link
Collaborator

murphp15 commented Feb 21, 2024

What is the feature request? What problem does it solve?
If a user needs to process larger amounts of data with VDK they should be able to deploy multiple pod instances and use kafka consumers to scale out.

In this ticket we want an example of a single pod deployment reading from kafka.
When the topic is drained we then want the pod to die.

@murphp15 murphp15 added the enhancement New feature or request label Feb 21, 2024
@murphp15 murphp15 added this to the multi-pod data-job instance milestone Feb 26, 2024
@gageorgiev gageorgiev assigned gageorgiev and yonitoo and unassigned gageorgiev Feb 28, 2024
@stefan-pulov stefan-pulov added the initiative: VDK for Private AI Initiative including the effort to support Private AI usecases of VMWare with VDK label Mar 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request initiative: VDK for Private AI Initiative including the effort to support Private AI usecases of VMWare with VDK
Projects
None yet
Development

No branches or pull requests

4 participants