feat: realtime auth + lifecycle events, hexagon structure, multi-instance fanout (ADR-0001-0009)

.gitignore

@@ -5,7 +5,8 @@ build/
 !**/src/main/**/build/
 !**/src/test/**/build/
 
-.kotlin/**
+.kotlin
+.playwright-mcp
 
 ### STS ###
 .apt_generated

AGENTS.md

@@ -17,6 +17,7 @@ Kotlin 2.4 · Spring Boot 4.1 (WebFlux) · Java 24 toolchain · Gradle wrapper 9
 | Compile | `./gradlew classes` (verified) |
 | Build + tests | `./gradlew build` (what CI runs; needs Docker for Testcontainers) |
 | Tests only | `./gradlew test` |
+| Coverage | `./gradlew koverHtmlReport` → `build/reports/kover/html` (generated gRPC stubs excluded) |
 | Run app | `SPRING_PROFILES_ACTIVE=local ./gradlew bootRun` |
 | Infra | `./init_volumes.sh && docker compose up -d` (Kafka KRaft, Keycloak+Postgres on :8090, MongoDB rs0 on :27017) |
 | Config | `cp src/main/resources/config/env.template.properties src/main/resources/config/env.properties` and fill values — required, `application.properties` resolves `${...}` placeholders from it |
@@ -25,19 +26,28 @@ Requires JDK 24 (`sdkman` GraalVM works). Protobuf/gRPC stubs generate from `src
 
 ## Layout
 
-- `src/main/kotlin/com/softeno/template/app/` — reusable blocks: `common/` (BaseDocument, errors, principal utils), `config/` (security, Mongo auditing, exception handling, OTel), `kafka/`, plus example domains `user/` and `permission/`.
-- `src/main/kotlin/com/softeno/template/sample/` — removable feature demos: `http/` (controller, functional router, external WebClient + circuit breaker), `proto/` (gRPC), `websocket/` (coroutine & reactive handlers).
-- `src/test/kotlin/.../IntegrationTest.kt` — `BaseIntegrationTest` boots the app under profile `integration` (permit-all test `SecurityConfig`) with Testcontainers Kafka (`apache/kafka-native`) and Mongo (`mongo:6.0.4`), random ports, WireMock for external HTTP, MockK/springmockk, fixtures in `fixture/`.
-- `http/` — IntelliJ HTTP client requests; copy `http-client.env.template.json` → `http-client.env.json` (gitignored).
+- `src/main/kotlin/com/softeno/template/` — three feature packages + infrastructure (ADR-0003):
+  - `users/` — **application core** (dummy template implementation — app user ids are Mongo ids, never Keycloak `sub`s): `UserService` works on db documents (no domain wrapper classes; permissions are embedded `PermissionSnapshot`s kept fresh by `UserPermissionSync` fan-out on permission update/delete — the fan-out publishes NOTHING per user; `PermissionService` emits ONE `UserAction.SYNCED` (rendered `USERS_SYNCED: <permissionId>`) after it completes, so clients refetch once against a fully-synced set (ADR-0005/ADR-0006)), `api/` REST+GraphQL, `db/` documents+repositories, `permission/` sub-domain; the write path ENDS at `UserEventsPublisher` (Kafka tx; `UserEvent` with `UserAction` CREATED/UPDATED/DELETED/SYNCED — ADR-0006). Events are cache-invalidation hints, never state transfer: a crash between Mongo commit and Kafka publish only delays client convergence (client backstops revalidate on stream reconnect), so there is deliberately no outbox — introduce one if a consumer ever needs the events for real work. Core never imports notifications/sample.
+  - `notifications/` — **optional feature** (deleting it keeps the app compiling): `Notification.kt` port (`broadcast` + `revokeUser`, ADR-0004), `Message.kt` envelope, `ticket/` one-time-ticket infra (`TicketStore` port + `MongoTicketStore` adapter — `findAndRemove` atomic redeem + TTL-indexed expiry, multi-instance since ADR-0009; `TicketRecord`/`TicketOrigin`, `TicketAuthentication`, `TicketProperties` — WebSocket-only since ADR-0007), `ws/` adapter (handlers, session registry + reverse index + sweeper, `api/WsTicketController.kt` mint, `security/` redeem filter + `SessionBinding`, toggled by `com.softeno.ws.enabled` — on by default, `matchIfMissing`; off ⇒ no `/ws` mapping, no `/ws-ticket`, adapter absent from the port list (`WsDisabledIT`)), `sse/` adapter (`Service.kt` emitter, `SseSubscriptionRegistry` reverse index + ceiling sweeper + `closeAll()` on `ContextClosedEvent` so streams die with the acceptor instead of outliving it — otherwise clients sit on a connected stream of a dying server (`SseShutdownIT`); bearer-auth only — the ticket path was removed by ADR-0007), `kafka/` listeners (user events → `List<Notification>` broadcast; Keycloak LOGOUT → `revokeUser` on every adapter; admin events → `AdminDirective.of` interpretation (`RevokeUser` on `ACTION users/{id}/logout` / `DELETE users/{id}`, else `Ignore`) → same revocation path, covering admin-initiated session termination that fires no user LOGOUT event; the DTO is a dumb wire shape — a new admin-event reaction is a new directive case + controller branch, not matching logic in the DTO). All listener groups are **per-instance** (`${group}-…-${com.softeno.kafka.instance-id}`, ADR-0008): every instance consumes every event and acts only on its own connections — never put a *writing* consumer on these groups; new side-effect consumers need a shared (competing) group.
+  - `sample/` — removable demos: `http/` (`internal/` coroutine service/controller/router; `external/` WebClient + circuit breaker), `proto/` (gRPC). The WebSocket demo moved to `notifications/ws/`.
+  - `common/` (BaseDocument, errors, paging utils) + `config/` (security, Mongo auditing, exception handling, OTel, kafka wiring incl. `WebToolConfig` static page serving) — infrastructure, imports nothing from feature packages (except `config/kafka` wiring the core's `UserEvent`; `config/graphql` holds the global GraphQL exception handler).
+- Repo-root `web/` — static dev tools served at `/web/**` by `config/http/WebToolConfig.kt` (permitAll): `chat.html` (ticket-auth WebSocket reference client — re-mint on error, `REAUTH_REQUIRED` reconnect, close codes). The page always calls the backend at :8080, so it also works served from :3000, the origin the stock Keycloak `frontend` client allows. (`sse.html` was removed by ADR-0007 — SSE lives in the main FE app now.)
+- `docs/adr/` — architecture decision records; immutable once accepted, superseded by new numbered ADRs (see *Keeping docs current*).
+- **Test layout**: integration tests end in `*IT.kt` and live in the package of the feature they test; unit tests stay `*Test.kt` (the suffix is a convention — Gradle's JUnit Platform runs both under `test`). Shared test infrastructure lives in `src/test/kotlin/.../testsupport/`: `BaseIntegrationTest` boots the app under profile `integration` (permit-all test `SecurityConfig`) with Testcontainers Kafka (`apache/kafka-native`) and Mongo (`mongo:6.0.4`), random ports, and wipes users/permissions after each test; `R` is the static random test-data utility — tests generate their data via `R` and assert against those values; `TestBearerWebFilter` simulates bearer auth via the `X-Test-Sub` header under the `integration` profile. Fixtures/abilities live in their context's package (e.g. `users/permission/PermissionFixture.kt`, `notifications/kafka/KeycloakEventFixture.kt`, `sample/http/external/ExternalApiAbility.kt`, `notifications/ws/WsTicketAuthAbility.kt` — the browser-like WS client harness). `notifications/ws/WsTicketAuthIT` is Testcontainers-free (random port + `X-Test-Sub` header auth + `FakeTicketStore` via nested `@TestConfiguration` — never give it a Mongo-backed dependency: there is no Mongo in that context, calls would hang; the Mongo adapter is covered by `TicketStoreIT`). WireMock serves external HTTP on :4500.
+- `http/` — IntelliJ HTTP client requests; copy `http-client.env.template.json` → `http-client.env.json` (gitignored). `ws.http` demos ticket mint + `WEBSOCKET` chat frames.
 - `.github/workflows/` — `ci.yml` (build+test all branches), `build-and-upload.yml` (jar → multi-arch image `xcodeassociated/spring-reactive-template:latest` + dependency graph).
 
 ## Conventions
 
-- **Dual idioms**: capabilities exist in both Reactor (`Flux/Mono`, `reactive/` packages) and coroutines (`Flow/suspend`) variants. When adding a feature, follow the existing pair or note which style it is.
-- **Feature removal contract**: each optional feature = own package + a commented dependency group in `build.gradle` + its properties keys (`com.softeno.kafka.*`, `com.softeno.ws.*`, `spring.grpc.*`, …) + rules in `app/config/security/SecurityConfig.kt`. Known `app → sample` coupling points that must be removed together with their feature: `app/kafka/KafkaSampleHandler.kt` and `app/user/notification/Service.kt` (websocket `Message`/emitters), `app/config/http/WebExceptionHandler.kt` and `app/user/api/GraphqlController.kt` (`externalGraphQl` bean, external client config). Avoid adding new cross-imports beyond these.
-- Security rules live in the single main `SecurityConfig.kt` (path matchers, `realm_access` roles → authorities, method security via `@EnableMethodSecurity`); the test-profile config must keep paths permit-all.
-- Bean-level toggles use `@ConditionalOnProperty` (e.g. `com.softeno.ws.type=coroutine|reactive`).
-- Entities implement `app/common/BaseDocument.kt` (id, `@Version`, audited createdBy/Date); auditing pulls the JWT `sub` via `ReactiveMongoAuditorConfig`.
+- **App users ≠ Keycloak users**: the `users/` domain is a **dummy template implementation** (Mongo `UserDocument`). Never mix the two identities — live SSE/WS sessions are bound to the Keycloak JWT `sub`, while app user ids are Mongo ids; don't treat one as the other (e.g. app-user lifecycle events carry app ids and are broadcasts only, never session revocation).
+
+- **Full verification**: any addition or change runs the entire test suite (`./gradlew build`, what CI runs) before handoff — never just the affected package's tests.
+
+- **Coroutine-first** (no dual idioms): endpoints, services, and SSE/WS features are coroutine-based (`Flow`/`suspend`); the reactive Reactor API exists only as infrastructure beneath them (WebFlux transport, reactive Mongo repositories, `WebClient`, `WebFilter`/RouterFunction signatures — which the transport requires). Do not add parallel `reactive/` flavors of features.
+- **Feature removal contract**: each optional feature = own package + a commented dependency group in `build.gradle` + its properties keys (`com.softeno.kafka.*`, `com.softeno.ws.*`, `com.softeno.ticket.*`, `com.softeno.sse.*`, `spring.grpc.*`, …) + rules in `config/security/SecurityConfig.kt`. The core→notifications boundary is Kafka (ADR-0003) and the listeners drive transports only through the `Notification` port (ADR-0004) — so there are **no `app → sample` coupling points left**. Remaining known couplings: `config/http/WebExceptionHandler.kt` (external client config — the `externalGraphQl` bean now lives inside `sample/` itself) and the `userEventsPublisher.publish` calls in `UserService` (create/update/delete — delete with Kafka). `notifications/ticket/` is WebSocket-only infra since ADR-0007 — remove with the `ws/` adapter.
+- Security rules live in the single main `SecurityConfig.kt` (path matchers, `realm_access` roles → authorities, method security via `@EnableMethodSecurity`); the test-profile config must keep paths permit-all. `/ws/**` is `.authenticated()` via the one-time ticket redeemed at the upgrade (`WsTicketRedeemFilter` — browsers cannot send headers on an upgrade; ADR-0001); `/update/**` (SSE) is `.authenticated()` via bearer JWT only (ADR-0007); `/ws-ticket` mint endpoint and `/sample-secured/**` + `/minio/**` are `.authenticated()`; `/web/**` is permitAll static (dev-tool pages).
+- Bean-level toggles use `@ConditionalOnProperty` (WebSocket adapter: `com.softeno.ws.enabled` — **enabled by default**, `matchIfMissing = true`; setting `false` removes the entire WS feature: handler + `/ws` mapping, message service, `/ws-ticket` mint endpoint, redeem filter, and the adapter simply drops out of the `List<Notification>` fan-out. Contract pinned by `WsDisabledIT`: with the flag off the endpoints 404 and no WebSocket connection can be established while SSE/Kafka keep working).
+- Entities implement `common/BaseDocument.kt` (id, `@Version`, audited createdBy/Date); auditing pulls the JWT `sub` via `ReactiveMongoAuditorConfig`. Services work on db documents — no domain wrapper classes (ADR-0003).
 - Profiles: `local` (disables OTLP export), `docker` (set by Dockerfile entrypoint), `integration` (tests). Never commit secrets; only `env.template.properties` belongs in git.
 
 ## Gotchas
@@ -46,8 +56,11 @@ Requires JDK 24 (`sdkman` GraalVM works). Protobuf/gRPC stubs generate from `src
 - The Keycloak image `xcodeassociated/keycloak-kafka` is a third-party image that forwards auth events to Kafka topics consumed by `KafkaKeycloakController`.
 - macOS ARM only: build conditionally adds `netty-resolver-dns-native-macos` (via `osdetector`).
 - `HELP.md` is a stale Spring Initializr leftover (gitignored).
+- Coroutine `saveAll` does not apply version-checked updates to existing documents (verified, Spring Data Mongo 5.1) — `UserPermissionSync` saves per entity; keep it that way.
+- Embedded document shapes have no migration framework: after changing `UserDocument`'s nested types, wipe the dev collection (e.g. `mongosh db.userDocument.deleteMany({})`) or old rows 500 on read. Same trap on the dev Kafka topic: `sample_topic_2` may hold pre-typed-refactor payloads (`{"content":"USER_CREATED: …"}`) — since ADR-0008 production/dev consumers start at `latest`, fresh groups no longer replay them (the IT profile keeps `earliest` for deterministic consumption — a brand-new IT group against a polluted topic WOULD replay; containers are ephemeral, so it never happens). Pre-ADR-0006 payloads (`{"userId": …}` without `action`) read as `CREATED` via `UserEvent.effectiveAction` (the field is nullable because Jackson does NOT apply Kotlin ctor defaults without the Kotlin module — a missing `action` would otherwise NPE; pinned by `UserEventsListenerTest`).
 
 ## Keeping docs current
 
 - **`README.md` must always be up to date.** Before finishing any development task, check whether it changed anything the README documents (purpose, features, feature-removal table, project structure, commands, ports, setup steps, infrastructure) and update the README in the same change if so. Never hand off with a stale README.
 - **`AGENTS.md` must always be up to date too.** Before finishing any development task, check whether it changed anything this file documents (stack versions, Gradle tasks/commands, feature set or its removal contract, security config structure, CI/deployment) and update it in the same change if so. Verify commands you document actually run; mark anything untested as such.
+- **`docs/adr/` must stay accurate — but ADRs are immutable.** Never rewrite the context/decision of an accepted ADR to reflect new reality. When a documented decision or flow changes (e.g. the WebSocket ticket-auth flow in `ADR-0001`), write a new numbered ADR explaining the new context and decision, set the old ADR's status to `Superseded by ADR-NNNN` (the status line is the only editable part), and update the README ADR table plus any linked sections in the same change.