feat(evaluations)!: promote Evaluations from Code to core

AGENTS.md

@@ -86,10 +86,10 @@ Every public module has a committed API report at `packages/<package>/api-report
 
 PRs are squash-merged with the PR title as the commit message, and release-please builds each package's changelog from those messages.
 
-- **A change to an experimental feature uses that feature's scope:** `evaluations` (Evaluations from Code). `feat`, `fix` and `perf` commits with that scope are listed under **Experimental** (`changelog-sections` in `release-please-config.json`), for example `feat(evaluations): score rows in parallel`. Any other scope, such as `client` or `skills`, lists the change as a core feature or fix. Agent Skills is core, so `feat(skills):` lands under Features.
-- **Use exactly one scope.** `feat(evaluations,client):` matches no Experimental row, so it lands under Features.
+- **A change to an experimental feature uses that feature's scope.** `feat`, `fix` and `perf` commits with that scope are listed under **Experimental** (`changelog-sections` in `release-please-config.json`). No feature is experimental now. Any other scope, such as `client`, `skills` or `evaluations`, lists the change as a core feature or fix. Agent Skills and Evaluations from Code are core, so `feat(skills):` and `feat(evaluations):` land under Features.
+- **Use exactly one scope.** `feat(beta,client):` matches no Experimental row, so it lands under Features.
 - **A PR that changes both core and experimental behaviour** should be split. If it can't be, add a `BEGIN_COMMIT_OVERRIDE` block to its description with one conventional-commit line per change. An override also corrects a merged PR's title, as long as the release PR hasn't merged yet.
-- **A breaking experimental change** uses `!` while the package is 0.x: `feat(evaluations)!:` bumps the minor version, which is what an experimental break is allowed. After 1.0, `!` would force a major release. Use `feat(evaluations):` without `!` instead, and say the change is breaking in the title.
+- **A breaking experimental change** uses `!` while the package is 0.x: `feat(beta)!:` bumps the minor version, which is what an experimental break is allowed. After 1.0, `!` would force a major release. Use `feat(beta):` without `!` instead, and say the change is breaking in the title.
 - **When a feature is promoted to core,** remove its rows from `changelog-sections`.
 
 The rule is specified in `TESTING.md` §0.3 in `launchdarkly/ai-sdks-monorepo`.

packages/ai/README.md

@@ -70,10 +70,10 @@ Experimental features are not re-exported from this package. They live in
 `launchdarkly-ai-server`, the package this one depends on, and need a `launchdarkly-ai-server`
 release that includes them. Import them from `launchdarkly_ai_server.experimental`; if that
 import fails, upgrade `launchdarkly-ai-server` itself, since upgrading this package alone can
-leave an older `launchdarkly-ai-server` in place. The experimental feature is:
+leave an older `launchdarkly-ai-server` in place. No feature is experimental now.
 
-- **Evaluations from Code**, in `launchdarkly_ai_server.experimental.evaluations`; see the
-  [evaluations guide](https://github.com/launchdarkly/python-ai-sdk/blob/main/packages/client/README.md#run-an-evaluation-from-code-experimental).
+Evaluations from Code is core, so this package re-exports it. See the
+[evaluations guide](https://github.com/launchdarkly/python-ai-sdk/blob/main/packages/client/README.md#run-an-evaluation-from-code).
 
 ---
 

packages/ai/tests/test_reexports.py

@@ -27,9 +27,9 @@ def test_reexported_values_are_identical_references(self) -> None:
                 f"launchdarkly_ai_python.{name} is not the same object as launchdarkly_ai_server.{name}"
             )
 
-    def test_experimental_evaluations_are_not_reexported(self) -> None:
-        importlib.import_module("launchdarkly_ai_server.experimental.evaluations")
+    def test_evaluations_are_reexported(self) -> None:
+        server = importlib.import_module("launchdarkly_ai_server")
         barrel = importlib.import_module("launchdarkly_ai_python")
 
         for name in ("init_evaluations", "EvalTool", "Judge", "Scorer", "DatasetRow"):
-            assert not hasattr(barrel, name), name
+            assert getattr(barrel, name) is getattr(server, name), name

packages/client/README.md

@@ -46,11 +46,7 @@ No code changes are required — `init_client()` detects the packages at runtime
 
 `init_client()` also sets the `launchdarkly.project_id` resource attribute to the resolved SDK key, not the project's ID.
 
-### Run an evaluation from code (experimental)
-
-> **Experimental.** Import Evaluations from Code from `launchdarkly_ai_server.experimental.evaluations`;
-> none of these names is exported from the package root. They may change in a minor release,
-> so review the changelog when you upgrade.
+### Run an evaluation from code
 
 The evaluations harness reads an LD-hosted dataset, creates a new evaluation and API-source run, and invokes your handler once per row. Rows can then be scored by LaunchDarkly judges and local scorer functions; see [Score rows with judges and scorers](#score-rows-with-judges-and-scorers). Each success or error queues a `$ld:ai:offline-evals:generation` custom event containing the evaluation, run, dataset, and row identifiers plus output or error (`errorMessage` is included for `ERROR` rows), nested `usage.inputTokens`/`usage.outputTokens`, timing, and stable hashes. Dataset-owned input, expected output, metadata, and variables are not duplicated in the event. Each queued event is logged at `INFO` on the `launchdarkly_ai_server.evaluations.runner` logger with its RFC3339 UTC `emittedAt` timestamp and stable `eventId`, making it possible to compare SDK emission time with ClickHouse arrival time once that logger is enabled. The same `emittedAt` value is included in the event payload. Events are flushed before the summary is fetched and the call returns; handlers are never rerun to retry event delivery. Pass/fail is derived from LaunchDarkly's run summary.
 
@@ -63,7 +59,7 @@ import asyncio
 import sys
 
 from launchdarkly_ai_openai_messages import create_openai_messages_handler
-from launchdarkly_ai_server.experimental.evaluations import init_evaluations
+from launchdarkly_ai_server import init_evaluations
 
 
 async def main() -> int:
@@ -124,7 +120,7 @@ Pass `criteria` to `run()` to score every generated row. A `Judge` references an
 ```python
 from launchdarkly_ai_claude_messages import create_claude_messages_handler
 from launchdarkly_ai_openai_messages import create_openai_messages_handler
-from launchdarkly_ai_server.experimental.evaluations import (
+from launchdarkly_ai_server import (
     DatasetRow, Judge, Scorer, init_evaluations,
 )
 
@@ -272,7 +268,7 @@ async def main():
 Pass `tools` to `run()` as a list of `EvalTool`. Construct one to define a tool in code. Await `evals.tools.get()` to use a tool that already exists in your project's AI library, which reads the tool and pins its version at that point. One list can hold both kinds.
 
 ```python
-from launchdarkly_ai_server.experimental.evaluations import EvalTool, init_evaluations
+from launchdarkly_ai_server import EvalTool, init_evaluations
 
 
 def lookup_order(order_id: str) -> str:

packages/client/agents.md

@@ -39,7 +39,7 @@ No other `launchdarkly-ai-*` package may define or duplicate these. They import
 | `src/launchdarkly_ai_server/utils.py` | `parse_template`, `parse_json_with_possible_fences`, `create_handler`, `parse_usage`, `make_track_data`, `to_ld_context` |
 | `src/launchdarkly_ai_server/registry.py` | `Registry`, `global_registry`, `compose`, `resolve_handlers`, `resolve_tools` |
 | `src/launchdarkly_ai_server/judges.py` | `run_judges`, `build_judge_tasks`, `run_judge` |
-| `src/launchdarkly_ai_server/evaluations/` | `init_evaluations`, the private management API operations, and `EvaluationsModule.run()` orchestration. Experimental: public names are exported only from `experimental/evaluations.py`, never from the package root |
+| `src/launchdarkly_ai_server/evaluations/` | `init_evaluations`, the private management API operations, and `EvaluationsModule.run()` orchestration. Public names are exported from the package root |
 | `src/launchdarkly_ai_server/__init__.py` | Public barrel — the only surface handler packages import from |
 
 ---
@@ -78,7 +78,13 @@ from launchdarkly_ai_server import execute_and_track, execute_and_stream, wrap_t
 
 # Entry points
 from launchdarkly_ai_server import config, graph, resolve_graph
-from launchdarkly_ai_server.experimental.evaluations import init_evaluations
+
+# Evaluations from Code
+from launchdarkly_ai_server import (
+    init_evaluations, EvaluationsModule, AIConfig, GenerationConfig, GenerationMode,
+    GenerationOverrides, DatasetRow, EvalTool, Criterion, Judge, Scorer,
+    EvalRunResult, RunSummary, EvaluationsError,
+)
 
 # Agent Skills
 from launchdarkly_ai_server import (
@@ -91,12 +97,13 @@ from launchdarkly_ai_server import (
 )
 ```
 
-An experimental feature (today only Evaluations from Code) is exported only from its module
-under `launchdarkly_ai_server.experimental`, and its names may change in a minor release. No
-core public type, function or `init_client` option may name it. Agent Skills is core and is
-exported from the package root; `shutdown` still reaches its state only through an internal
-hook (it clears the skill store), and an error there is logged, never raised into
-`shutdown`.
+An experimental feature is exported only from its module under
+`launchdarkly_ai_server.experimental`, and its names may change in a minor release. No feature
+is experimental now. No core public type, function or `init_client` option may name an
+experimental feature. Agent Skills and Evaluations from Code are core and are exported from
+the package root. `lifecycle` imports no Evaluations from Code module. For Agent Skills,
+`shutdown` still reaches its state only through an internal hook (it clears the skill store),
+and an error there is logged, never raised into `shutdown`.
 
 `MAX_SKILL_CONTENT_BYTES` and `SKILL_OBJECT_KIND` are deliberately **not** exported; both stay
 internal to `skills_core`:
@@ -163,7 +170,7 @@ Handlers may return any of these — the client normalizes them before emitting
 
 ## SDK-run evaluations
 
-Evaluations from Code is experimental. Its public names are exported from `launchdarkly_ai_server.experimental.evaluations` and pinned by `TestPackageExports` in `tests/test_evaluations.py`; none may be added to the package root. `init_evaluations` reports the helper `experimental.initEvaluations`.
+Evaluations from Code is core. Its public names are exported from the package root and pinned by `TestPackageExports` in `tests/test_evaluations.py` and by `tests/test_public_surface.py`. `init_evaluations` reports the helper `client.initEvaluations`.
 
 `init_evaluations()` creates an evaluations harness using `LD_API_TOKEN` and the management API host (`api_base_uri`, then `LD_API_BASE_URI`; a blank value at either level counts as unset). Do not reuse `LD_BASE_URI`, or name the option `base_uri`: that host configures SDK delivery and may point at a relay proxy. Evaluation-run links use the separate `ui_base_uri` option, then `LD_UI_BASE_URI`, then `https://app.launchdarkly.com`; do not derive their host from `LD_API_BASE_URI`. An event transport is resolved in `init_evaluations()`, which raises before any network I/O when it finds neither an SDK key (`sdk_key` or `LD_SDK_KEY`) nor an already-initialized event-capable client: generation events are the only ingest path for row results, so a run without a transport could never complete. The lifecycle module's bring-your-own-client path (`init_client(client=...)`) therefore satisfies the check on its own, and `run()` reuses that singleton through `_resolve_client`; `run()` raises if the client disappears before it emits. Both polling arguments reject NaN, which would otherwise never compare past a deadline and hang the run. The harness always queues one `$ld:ai:offline-evals:generation` custom event per row through the standard SDK event transport and flushes before returning. No feature flag gates event emission. The harness polls the run summary endpoint until a nonzero `total_rows` has `pending_rows == 0` and `passed + failed + error` rows accounting for the total, polling every `poll_interval_seconds` (default 2s) until `poll_timeout_seconds` (default 180s); both are `run()` arguments so large datasets can widen them. The summary endpoint does not return run state, so `RunSummary` exposes row counts only.