Repository navigation
feat: Kafka agent - #1028
feat: Kafka agent#1028labrenbe wants to merge 29 commits into
Conversation
|
|
||
| [workspace.dependencies] | ||
| stackable-operator = { git = "https://github.com/stackabletech/operator-rs.git", tag = "stackable-operator-0.118.0", features = ["crds", "webhook"] } | ||
| stackable-operator = { git = "https://github.com/stackabletech/operator-rs.git", branch = "feat/platform-access", features = ["crds", "webhook"] } |
There was a problem hiding this comment.
TODO: revert to a tag when ready.
| .with_labels(recommended_labels_for_agent_resources(cluster)) | ||
| .build(), | ||
| ) | ||
| .image_pull_secrets_from_product_image(&cluster.image) |
There was a problem hiding this comment.
Assuming that the agent image can use the same credentials as the product image is fine as long as they come from the same registry. If they don't, there should be an optional field platformAccess.image.pullSecrets that allows the user to specify these independently; when unset, the agent falls back to spec.image.pullSecrets as it currently does.
There was a problem hiding this comment.
This is currently following the precedence of opa-operator. If that pull secret doesn't work then objectOverrides can still be used to override it. I think it's worth a discussion how much customization we want to allow for the agent without overrides.
9b6f2a7 to
146602d
Compare
| ) | ||
| .expect("the volume name is statically defined and must not collide"); | ||
| container_builder | ||
| .add_volume_mount(&*DISCOVERY_VOLUME_NAME, DISCOVERY_DIR) |
There was a problem hiding this comment.
Just a question if this is intended behavior: if the address in the discovery dir is an external one (external ListenerClass), is it intended that the agent connects from outside using that external path?
There was a problem hiding this comment.
This is a very good call-out. I thought there were limitations that prevent us from using the headless services, but I realized my assumption was wrong. Changed the agent to use the headless service instead which is how it should be done anyway in 14f9ee8.
| ensure!( | ||
| metadata_manager == crate::crd::MetadataManager::KRaft, | ||
| ManagementAccessRequiresKraftSnafu | ||
| ); |
There was a problem hiding this comment.
This is pretty arbitrary. Can we document / comment why?
There was a problem hiding this comment.
I decided to focus on KRaft for the first version of the admin client as supporting clusters with zookeeper would add extra complexity. I will document that this is a current limitation of the first version of the kafka-agent. Similarly we decided not to support Kerberos for version 1 in stackabletech/issues#888.
Description
Part of stackabletech/issues#888.
Depends on decisions
Depends on PRs
Deploys an agent alongside Kafka clusters which is an independent controller that reconciles and manages internal resources like
KafkaTopic.This PR only adds an empty agent shell that doesn't do anything yet.
Definition of Done Checklist
Author
Reviewer
Acceptance
type/deprecationlabel & add to the deprecation scheduletype/experimentallabel & add to the experimental features tracker