[Spicy-cream]

What problem does this PR solve?

URMA can fall back to TCP during connection setup, but established failures do not guide subsequent connection attempts. A failed server SEND can close the control socket while the client immediately reconnects over URMA to the same peer.

What is changed and the side effects?

• Add process-level LEGACY, AUTO and TCP_ONLY modes. AUTO applies a configurable peer cooldown, defaulting to 5 seconds; LEGACY remains the default. New connections use TCP during cooldown and can select URMA afterward.
• Report completion, JFC and WR-posting failures once per connection. Ignore reports from earlier policy epochs, count access errors separately and preserve EAGAIN for backpressure.
• Report established client control-socket EOF/read failures so server-side faults also trigger peer cooldown. Normal server-side EOF from short clients closes the connection without cooling down the peer.
• Select CTP/RTP paths from device capabilities, release short-connection resources, serialize TCP reads and initialize/validate send SGEs. Optional aligned copied sends add one payload copy.
• Add a recovery client, process-local SEND fault injection, independent regression tests, documentation and a URMA mock CI workflow.

Mode changes affect new connections. Short connections can resume URMA after cooldown; established single/pooled TCP connections can continue serving RPCs. Existing v2/v3 wire formats remain compatible.

Validation

• Current community CI checks pass. The URMA mock workflow passes all 15 regression groups with RDMA disabled and URMA enabled.
• The server-SEND regression reproduces the missing client cooldown before the fix. All 8 CTP/RTP, v2/v3 and event/polling combinations pass after it, covering normal short closes, TCP cooldown, URMA recovery and 64 KiB attachment validation.
• Library/example builds, C++14 compilation, injection-runner self-tests and fetched-SDK-header configuration checks passed locally.
• Prior URMA RM/RTP hardware runs passed TCP_ONLY, v2/v3 AUTO, runtime mode switching and single/pooled 64 KiB/1 MiB attachment checks. Client SEND-post injection passed event short/single/pooled and polling short runs with 1 MiB attachments: one expected RPC failure per run, subsequent RPC recovery and zero content-validation failures. Short connections resumed URMA after TCP cooldown.

Related discussions and PRs

• Discussion #3711: runtime recovery and business QoS configuration.
• Discussion #3432: common handshake design for RDMA, URMA and UBSHM; URMA integration is proposed in feat: integrate URMA with the common transport handshake #3645.
• Related URMA fixes: Fix URMA send buffer reclamation order #3626 (send-buffer reclamation order) and Fix URMA polling shutdown lifetime #3634 (polling shutdown lifetime).

Check List

• Build the library/examples and run related regressions.
• Add independent tests, preserve existing community UT sources and update documentation.
• Follow four-space indentation and C++ nullptr conventions.

[Spicy-cream]

This is the necessary work for us to complete a plan,could you have a look? @chenBright @wwbmmm

[Spicy-cream]

This PR is only for URMA, should we have also done the RDMA part?

[wwbmmm]

On the RDMA question: URMA-only is fine for this PR. RDMA has no equivalent runtime recovery today -- it only falls back to TCP while the handshake/queue-pair bring-up is failing (rdma_endpoint.cpp), and an established QP error is treated as unrecoverable. So there is no parity gap this PR has to close. If the mechanism proves useful, the generalization worth doing afterwards is a transport-agnostic "a transport failure cools this peer for the next N seconds / new connections degrade to TCP" hook shared by RDMA and URMA, rather than a second process-global copy of the policy inside RDMA. That is also an argument for keeping the policy interface small and transport-neutral where it is cheap to do so.