[kinyoklion]

Summary

The flag overrides feature defined by the OVERRIDE specification needs a file source that reloads reliably: coalesced change notifications, retry after a failed read, retention of the last good data, sequential reloads, support for files that do not exist yet, and files that are rewritten in place. This change adds that infrastructure to the integrations package for the file-based override source that follows. The existing file data source keeps its current behavior.

• FileDataReloader serializes reloads, debounces change signals with a 100 ms settle window, keeps the last good data on failure by not applying, retries a failed load after one second, reports an identical repeated failure once, and skips a reload whose content did not change. Close never waits for an in-flight read. An exception from the apply callback is reported like any other load failure and retried, and nothing from that load is remembered. A load that holds an emptied file back (below) applies nothing, reports no failure, and arms its retry for the time the loader asks for; a failure still arms the normal retry. The loader is told of every result that becomes the data in effect, applied or skipped as unchanged, through Loader.accepted.
• FileDataPoller detects changes by examining each file's modification time and size on a fixed interval, including a file that appears or disappears. A change callback that throws is logged and polling continues. Its observe is package-private, so that the loader reads a file's state the same way, through symbolic links.
• FileDataWatcher watches parent directories through the file system watch service and signals once after start so a change made between an initial load and the start of watching is not missed. A directory that is missing at start or deleted later is registered again on a one-second schedule once it exists, and one change is signaled so the reload picks up what was written meanwhile.
• OverrideFileLoader reads a list of files with the override source's rules: a file that does not exist contributes no entries, entries keep the versions that the documents specify, a value-only entry becomes a flag that is on and serves the value by fallthrough, a document that parses but holds a flag the model rejects is reported as a file data error, and the result carries a per-file summary and a content digest of each file's path, whether it exists, and its content.

Format, encoding and empty files

The override loader chooses the parser by the file name: .yaml and .yml, in any case, are YAML, and any other file, including one with no extension, is JSON. The OVERRIDE spec (requirement 1.3.3, sdk-specs #272) recommends the extension and lets an SDK whose file data source inspects the content keep doing so; the override source is unreleased, so it follows the extension, which lets the format decide whether an empty file is valid.

• The text encoding is detected from a byte order mark (UTF-8, UTF-16 or UTF-32) and is UTF-8 otherwise, for both formats, through SnakeYAML's UnicodeReader.
• An empty or white-space JSON file fails with the file has no content, which is not a JSON document, and the last good overrides stay. An empty JSON placeholder therefore fails the whole reload. A JSON document that is not an object, such as null or [], fails with the JSON document is not an object.
• An empty YAML file, or YAML with only a comment or a --- marker, is a document with no entries.
• Compatibility with choosing by content: a JSON file in UTF-16, or in UTF-8 with a byte order mark, still loads. JSON indented with tabs, or with the escape \/, loads from a .json file; SnakeYAML, which reads YAML 1.1, rejects it in a .yaml file.

In-place rewrites

Applications and editors rewrite a file in place: they truncate it and then write the new content. A load in between finds the file empty. A YAML file can legitimately be empty, so its parser cannot tell an emptied file from one in the middle of a rewrite. The loader keeps, for each file, whether it contributed content to the data in effect (updated by accepted), and, for a file whose content is in effect and that loads have found empty, since when it has been empty without being written, its state then (modification time and size, read through any symbolic link, compared only for equality), and when a load first held it back.

• A file whose content is not in effect, such as a placeholder or a file whose empty state was applied, is parsed right away, and so is any file when retries are off.
• Otherwise a new state restarts the file's delay, and the file is held back while it has stayed empty for less than the retry delay R, and for at most 5 R from the first load that held it back. The clock is read after the file's own read, so a slow read of another file does not shorten its delay. The hold asks for its retry when the first held file's delay or bound is over, plus R/10, so that a timer that fires a little early still finds the time over.
• A file's bound starts over only at a load that holds nothing back, for every file that load read, whether the load is applied, skipped as unchanged, or fails. A read with content, even content that fails to parse, or a missing file ends the file's empty state, so its next emptying starts a new delay even where a coarse modification time makes it look the same.
• A load reads and classifies every configured file, also after a read or parse failure, so files emptied together start their delays in the same load. It then fails with the first failure; then holds files back; and only then merges the entries, so a duplicate key or an entry that the data model rejects cannot keep a later file from being read. A holding load keeps the data in effect, including changes to the other files, because a reload applies all the files or none.
• Elapsed times are differences of System.nanoTime() readings, which do not overflow.

A rewrite is covered when the gap between truncating and writing is shorter than the retry delay. Not changed: a read that sees YAML cut off between two entries, after flagValues:, or inside an unquoted value parses as valid YAML and is applied until the next reload; JSON cut off anywhere fails to parse. Go still detects the format from the content; Node chooses by extension case-sensitively, while this change, like .NET, ignores case.

The existing file data source

FileDataSourceImpl, FileSynchronizer, FileInitializer, FileDataSourceBase, and FileDataSourceParsing.FileDataException are unchanged from feat/overrides. FileDataSourceParsing gains, for the override source: two FlagFactory methods that keep document versions; selectForPath and parseForPath (extension and encoding); the NO_DOCUMENT result that the YAML parser returns for content with no document, which has all-null maps as before; JsonFlagFileParser.parse(Reader); and the two JSON rules above, which the file data source cannot reach because it parses a file as JSON only when it starts with {. parse(InputStream) keeps reading JSON in the platform's default encoding for the file data source. The file data source still requires every configured file to exist, assigns a load version to every entry, expands a value-only entry to an on flag with a single fallthrough variation, reloads on every watch event with no debounce, no retry timer, and no skip, and reports every failure. Two new test classes pin that behavior: FileSynchronizerReloadBehaviorTest (identical rewrite emits another change set, every failed reload is reported and logged as the plain description at error level, no retry without a file change, missing file logged with the path) and FileDataLoadingBehaviorTest (a model-rejected document escapes as a SerializationException, and FileDataException.getDescription requires a cause). The baseline test files pass unmodified.

Testing

• OverrideFileLoaderTest, on real files rewritten in place (InPlaceWrites), with a test clock that crosses the long overflow: the format by extension, empty and non-object JSON, YAML with no document, the encoding (UTF-8, UTF-16LE and UTF-16BE, with and without a byte order mark, a non-ASCII value), the hold at 0.95 R and its end at exactly R, the requested retries, a new write restarting the delay (also through a symbolic link), the per-file bound with one file kept written and with files rewritten in turn or emptied in staggered turns, the bound starting over at a load that holds nothing back and not otherwise, a file read during another file's failure, the fix of a file saved with a mistake or a duplicate key, the first failure reported, the digest covering paths and presence.
• FileDataReloaderTest (scripted loader): a hold applies nothing, reports nothing, and is retried after the time the loader asks for; a failure arms the normal retry delay; accepted is called for applied and unchanged results only.
• FileDataReloaderEmptiedFileTest (the reloader with the real loader on real files): with a test clock and an hour-long retry delay, the hold's end and the requested retry delays on both arming paths; with the real clock, an emptied file held back and the rewrite applied with no empty application in between, a file that stays empty applied by the retry, two quick rewrites, a change to another file while one is empty, a retry finding a second truncation, a file moved between configured paths, the same bytes moved from .yaml to .json.
• The in-place rewrite handling was reviewed in eight rounds as fix: Keep the overrides in effect while an override file is rewritten in place #248 (general and adversarial reviews, Bugbot). Mutation check with a clean build per mutant: 57 mutants, each failing at least one test. The override test classes ran 12 times in a row with no failure.

SDK-3246

_{Opened from termag}

---

Note

Overview
Adds reload infrastructure for an upcoming file-based flag override source: FileDataReloader (serialized loads, 100 ms debounce, 1 s failure retry, skip unchanged via content hash, last-good data on failure), FileDataWatcher (directory watches with missing-dir retry), FileDataPoller (mtime/size polling as a fallback), and OverrideFileLoader (override-specific merge rules, SHA-256 digest, in-place rewrite handling via EmptiedFileException).

FileDataSourceParsing gains extension-based format selection, BOM-aware decoding for overrides, stricter JSON empty/non-object errors, YAML NO_DOCUMENT, and FlagFactory helpers that preserve document versions. The existing file data source behavior is unchanged; new tests document its reload and error semantics vs the override loader.

^{Reviewed by Cursor Bugbot for commit 3b81e47. Bugbot is set up for automated code reviews on this repo. Configure here.}

[kinyoklion]

bugbot review

[cursor]

Stale Bugbot comment from a previous run.

[kinyoklion]

The file data poller test that expected exactly one callback per rewrite is replaced by one that checks the poller goes quiet once the file is stable, following launchdarkly/go-server-sdk#473. A plain rewrite is not atomic (the write and the modification-time update are separate steps), so a poll that lands between them legitimately reports an extra change; the poller promises generous detection and the reloader absorbs the excess. The test writes in the ordinary way, drains the callbacks of the write, and then requires silence, which still fails for a poller that does not update its baseline.

Written by Claude Code on behalf of @kinyoklion.

[kinyoklion]

bugbot review

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 3b81e47. Configure here.}