[uditdasari]

This turns on the two channel pool metrics that grpc-gcp gained in #14947 and #14985 and sends them to Cloud Monitoring as Spanner built-in metrics.

What is reported

• channel_pool/channel_count: the number of channels in the data pool per state (active, draining), with the labels lane, pool_mode and pool_index. The instance is reported as unknown because one pool serves many databases and instances, the same as the existing eef metrics. The project is the built-in metrics fallback project.
• channel_pool/channel_load: for each call, the number of calls that are open on the chosen channel, as a histogram with the boundaries 1, 2, 3, 5, 8, 12, 16, 20, 25, 30, 40, 50, 75, 100, 150, 200. project_id, instance_id and database come from the resource name of the request, lane and pool_mode come from the pool.

Which pools report

• Only the main data pool. The instance admin and database admin stubs now get their own channel provider with these metrics off. They already had their own pools, so this does not add channels.
• With DirectPath fallback, the DirectPath pool and the CloudPath pool both report under one pool_index and are told apart by lane.
• The experimental fallback recovery layout reports without lane, because its channels switch between the two paths.

When nothing is reported

• The application sets its own channel provider, grpc-gcp is disabled, built-in metrics are disabled, the client uses no credentials, the emulator is used, or SPANNER_DISABLE_CHANNEL_POOL_BUILTIN_METRICS=true.
• Application settings cannot change these metrics. Spanner always overwrites the built-in metrics settings in the grpc-gcp options and adds the per-call labels after the CallContextConfigurators of the application.

Cost

• channel_count adds nothing per call.
• channel_load adds one cached lookup of the resource name per call and carries the labels in the call options. Recording the value was measured in feat(grpc-gcp): add built-in channel_load metric for client library pools #14985 at about 38 ns per call with one thread, and up to about 1 µs per call when 8 threads write the same series at the same time.

Testing

• GapicSpannerRpcTest and SpannerMetadataProviderTest for the options and the resource name parsing. ChannelPoolBuiltInMetricsTest runs a client against the mock server and checks that only the data pool reports, the lanes of the DirectPath fallback pools, the fallback recovery layout without lane, the switches that turn the metrics off, that application settings cannot change the metrics, and the labels and boundaries of channel_load.
• End to end on a GCE VM against a real database, with the exported data checked afterwards: CloudPath and DirectPath clients, a static pool, a dynamic pool that scaled up from 4 to 8 channels and down from 8 to 6 (with 2 draining), two clients in one process (pool_index 1 and 2), two databases, and the long-term series.

[gemini-code-assist]

Code Review

This pull request introduces built-in metrics for the gRPC-GCP channel pool in the Cloud Spanner Java client, including channel count and channel load metrics, along with corresponding configuration options and metadata attributes. The review feedback correctly identifies a compilation error in SpannerMetadataProvider.java due to missing imports for Cache and CacheBuilder. Additionally, it suggests refactoring the cache to use LoadingCache and CacheLoader, which simplifies the retrieval logic and eliminates the need for manual exception handling via getUnchecked.