[tzphh]

Purpose

Fix #10715.
Add partition.expiration-retain-latest to retain the latest partition when all partitions are eligible for expiration, keeping data available when no new data arrives.

The option defaults to false and supports the built-in values-time and update-time strategies on normal tables.

Tests

73 tests passed across PartitionExpireTest,PartitionExpireTableTest, and CoreOptionsTest.

Coverage includes repeated expiration, replacement after a new partition is
committed, expiration limits and batching, stable timestamp ties, and
Metastore partition deletion.

[Akash3121]

LGTM
I traced both built-in expiration strategies through partition-entry aggregation, max-count limiting, batching, metastore deletion, and repeated expiration runs. Retaining the candidate before limits are applied preserves the latest partition across repeated runs, while a newly committed later partition correctly releases the old one. The deterministic tie-breaker and malformed-value behavior also look appropriate. The added tests cover the important lifecycle and boundary cases.

[JingsongLi]

The fallback-data use case is supported, and the targeted options/partition-expiration reactor passes with normal Maven checks. I also traced repeated expiration, limits/batches, committed replacement partitions, metastore deletion, malformed values and default compatibility.

An additional real-catalog test finds that the Flink age-based expiration action bypasses the new protection: core expiration retains the one row; executing the action with retain-latest explicitly true leaves zero rows. The inline comment describes the missing option propagation. The action was compiled from this head against the head core/API; its inherited catalog/action dependencies are byte-identical to the base.

For rollout, all expiration owners must be upgraded/restarted before enabling the safeguard; older owners ignore the option. Pause expiration before rolling those owners back. There is no new on-disk format.

[JingsongLi]

[P1] Propagate the safeguard into the Flink expiration action

The supported ExpirePartitionsAction copies table configuration into its table, but constructs PartitionExpireStrategy from a separate map containing only strategy/formatter/pattern. Consequently the new strategy constructors always see retain-latest=false there, even if it is persisted on the table and explicitly supplied through the action's table-conf map.

I reproduced this with a real filesystem table containing only dt=20010101, TTL 1 d, formatter yyyyMMdd, and retain-latest=true. table.store().newPartitionExpire(...).expire(...) leaves its one row; the actual Flink ExpirePartitionsAction.executeLocally() with the same safeguard explicitly true deletes it and leaves an empty table. This is age-based expiration, not the explicit DROP/TRUNCATE exception documented below.

Please build the action's strategy options from the table/configured options before applying its CLI overrides, or explicitly propagate this option, and add an action-level regression that keeps the sole expired partition.

[tzphh]

Thanks for catching this! Fixed by propagating partition.expiration-retain-latest from the merged table options to the action's expiration strategy. Added action-level regression tests covering persisted settings, --table_conf overrides, and repeated expiration of the sole expired partition.

[JingsongLi]

+1