feat(metrics): OTLP push export for gateway metrics #4489

User Story

I run OpenShell gateways and feed their telemetry into an existing observability stack with an OTLP pipeline into a trace and metrics backend. I want the gateway to push its operational metrics over OTLP, alongside the current Prometheus scrape endpoint, so that collectors which do not scrape can still receive gateway metrics through the same OTLP path the gateway already uses for traces.

Problem Statement

The gateway emits its metrics families (openshell_server_grpc_requests_total, openshell_server_grpc_request_duration_seconds, the HTTP equivalents, the readiness gauge, and gateway-interceptor counters) only through the Prometheus /metrics scrape endpoint. Gateway trace export already supports OTLP push via the openshell-otel crate and the [openshell.gateway.otlp] config surface, but metrics have no push path. Operators whose collectors are push-only, or who want metrics to travel the same OTLP route as traces, have no supported option today.

Impact / Why This Matters

In push-only or collector-centric deployments, operators must run a separate Prometheus scrape path solely for OpenShell, duplicating transport and trust configuration they already solved for trace OTLP. This is an adoption barrier for environments standardized on OTLP collectors.

Proposed Design

Offer OTLP metric export as an opt-in, in addition to (not replacing) the existing /metrics scrape endpoint. Scrape remains the default, consistent with the "scrape first, push as operator choice" position from #909. Configuration reuses the existing [openshell.gateway.otlp] surface (endpoint, TLS, service name) rather than inventing a parallel one. Default off.

Suggested UX (if applicable)

[openshell.gateway.otlp]
endpoint = "https://collector:4317"
service_name = "production-gateway"
metrics = true   # opt-in; traces unaffected, scrape endpoint still available

Acceptance Criteria

• Gateway metrics can be exported via OTLP push to a configured endpoint, opt-in, default off.
• The Prometheus /metrics scrape endpoint remains available and is the default.
• OTLP metric export reuses the existing [openshell.gateway.otlp] config and TLS surface.
• Documented in the gateway configuration reference.

Alternatives Considered

Prometheus remote-write: adds a second export protocol and dependency when the gateway already speaks OTLP for traces. A collector-side scrape-to-OTLP bridge: works, but pushes the integration burden onto every operator rather than offering a first-class path.

Agent Investigation

Verified on main: no OTLP MetricExporter/PeriodicReader exists in crates/; metrics are Prometheus scrape-only. Trace OTLP export is present in openshell-otel (OtlpTraceConfig, build_provider) and wired through Helm into gateway.toml.

Context

Re-homed from the now-closed #2507 (sub-issue 4, "OTLP metrics export"), which was auto-closed as stale after its trace-export critical path landed. Part of the #1055 Enterprise Observability umbrella. Related: #3664 (securing the metrics endpoint), #909 (metrics catalog, closed).

cc @krishicks (original #2507 author / observability WG lead)