[janhoy]

🤖 AI assisted 🤖

Fixes #690

The operator now exposes metrics about its own work on the existing /metrics endpoint, next to the generic controller-runtime, workqueue, client-go and Go runtime metrics. All of them are prefixed solr_operator_.

Gauges, computed at scrape time from the operator's cache so they reflect exactly what the operator sees and disappear with the resource: managed_resources per kind and namespace; solrcloud_info (Solr version), solrcloud_desired_replicas, solrcloud_ready_replicas, solrcloud_up_to_date_replicas and solrcloud_ready per SolrCloud; solrcloud_cluster_operations by operation and running/queued state; solrbackup_state (running/succeeded/failed) per SolrBackup; solrprometheusexporter_ready per exporter.

Counters, incremented by the controllers: reconciles_total per resource and result (controller-runtime only counts per controller, this tells which resource); solrcloud_cluster_operations_total by operation and completed/failed/paused result; solrbackup_runs_total by result; solr_api_requests_total per SolrCloud and result. Counter series are removed when the operator sees the resource has been deleted. The reconcile and queue-depth stats asked for in the issue are already covered by controller_runtime_* and workqueue_*, so the docs point at those instead of duplicating them.

The helm chart gets a prometheus section mirroring the solr chart: prometheus.scrapeAnnotations.enable (default true, since the operator pods have always carried prometheus.io/scrape) now also adds prometheus.io/port and prometheus.io/path, and prometheus.podMonitor.* creates a PodMonitor for the operator pods for a Prometheus Operator setup. The PodMonitor sets honorLabels, so the namespace label of the metrics keeps naming the Solr resource's namespace rather than the operator pod's; the docs say the same about honor_labels for annotation-based scrape configs. It is not gated on CRD detection, so offline rendering works.

New docs page getting-started/operator-metrics.adoc covers scraping, the metrics reference, the useful generic metrics, multi-replica behaviour and example alert queries. Upgrade note and chart changelog included.

Testing

Unit tests for the counters and the reconciler wrapper, fake-client tests for the collector including read errors, and an envtest spec that registers the collector on the real manager cache like main.go does and checks that series appear and disappear with a SolrCloud. make lint, make check-docs and make unit-tests pass. E2E tests were not run.

Overlaps with #880 only in values.yaml (both add keys under prometheus) and the chart changelog.

[Copilot]

🟡 Changes recommended

Unresolved metrics defects can misattribute resource namespaces, double-count completed backup runs after failed status patches, and incorrectly report readiness when replicas are omitted.

3 open issues

• Use the API default for omitted replica counts · New
• Increment the finished-run counter only after patch success · New
• Preserve resource namespace labels in Prometheus · New

What changed in this PR

Adds application-level monitoring for the Solr Operator, addressing #690 through resource gauges and operational counters on the existing /metrics endpoint.

Changes:

• Adds cache-backed resource metrics and counters for reconciles, cluster operations, backups, and Solr API requests.
• Adds configurable scrape annotations and an optional Helm-managed PodMonitor.
• Adds metric tests, scraping guidance, example queries, and upgrade notes.

File	Description
main.go	Registers the cache-backed metrics collector.
helm/​solr-operator/​values.yaml	Adds scrape annotation and PodMonitor settings.
helm/​solr-operator/​templates/​podmonitor.yaml	Defines the optional operator PodMonitor.
helm/​solr-operator/​templates/​deployment.yaml	Makes scrape annotations configurable.
helm/​solr-operator/​README.md	Documents monitoring and Helm settings.
helm/​solr-operator/​Chart.yaml	Records monitoring additions in the changelog.
go.mod	Promotes Prometheus libraries to direct dependencies.
docs/​modules/​upgrade-notes/​pages/​upgrade-notes.adoc	Explains monitoring-related upgrade changes.
docs/​modules/​getting-started/​pages/​running-the-operator.adoc	Links to the metrics guide.
docs/​modules/​getting-started/​pages/​operator-metrics.adoc	Documents scraping, metrics, and example queries.
docs/​modules/​getting-started/​nav.adoc	Adds the metrics guide to navigation.
controllers/​util/​solr_api/​v2.go	Counts v2 Solr API request outcomes.
controllers/​util/​solr_api/​api.go	Counts Solr Collections API request outcomes.
controllers/​suite_test.go	Registers resource metrics for integration tests.
controllers/​solrprometheusexporter_controller.go	Instruments exporter reconciles.
controllers/​solrcloud_controller.go	Instruments reconciles and cluster operation outcomes.
controllers/​solrbackup_controller.go	Instruments reconciles and backup completion.
controllers/​resource_metrics.go	Collects resource counts, readiness, and operation state.
controllers/​resource_metrics_test.go	Tests resource gauges and cache read failures.
controllers/​resource_metrics_integration_test.go	Tests metrics against the manager cache.
controllers/​metrics/​metrics.go	Defines counters, reconcile instrumentation, and cleanup.
controllers/​metrics/​metrics_test.go	Tests counter recording and deleted-resource cleanup.

_{Review effort: Balanced}