bug(ocsf): schema downgrade labels events 1.1 or 1.3 that do not conform to those versions #4284

User Story

I use OpenShell with Splunk as the OCSF sink and set ocsf_schema_version so that older SIEM integrations accept the events. I need events labelled 1.1 or 1.3 to actually conform to those schemas, so that the SIEM's data models and any audit relying on them are correct.

Problem Statement

downgrade_event() (crates/openshell-ocsf/src/format/downgrade.rs) strips a few profile-gated fields and rewrites metadata.version, but never checks the target version's requirements. OCSF 1.8 relaxed several requirements that 1.1 and 1.3 enforce, and OpenShell's emitters rely on that relaxation:

The result is events labelled 1.1 or 1.3 that the official validator rejects. The setting applies to the supervisor JSONL (ocsf_schema_version, released in v0.1.0) and the gateway JSONL ([openshell.gateway.ocsf_log] schema_version, on main). docs/observability/ocsf-json-export.mdx states the core structure is identical across 1.1 to 1.8 and lists AWS Security Lake as supported.

Impact / Why This Matters

Lenient sinks such as Splunk accept the events, so operators believe they hold conformant audit logs when they do not. Strict sinks that validate against 1.1 can reject them, and the operator sees this only in the sink's ingestion errors. Leaving the setting unset is not a workaround for sinks that only accept older versions.

Proposed Design

• An event OpenShell labels with an older version must validate against that version. Never insert placeholder data to satisfy a requirement.
• Events that can be converted are downgraded as today, also removing attributes the target does not define.
• Events that cannot be converted are written unchanged at native 1.8, counted, and reported once per event class with the missing attribute.
• Where the missing data genuinely exists, emitters include it: the generated response on deny paths, the workload peer as src_endpoint.
• The docs list which event types convert to each target version.

Suggested UX (if applicable)

No configuration change. A warning when an event stays at native 1.8:

WARN ocsf downgrade: HTTP Activity kept at 1.8.0 for target 1.1.0 (missing http_response); further occurrences counted in openshell_ocsf_downgrade_kept_native_total

Acceptance Criteria

• Every event labelled 1.1.0 or 1.3.0 passes POST https://schema.ocsf.io/api/<version>/validate.
• Events that cannot be converted keep metadata.version 1.8.0 and no downgraded_from marker.
• A metric counts events kept at native 1.8, by class and target version, on both supervisor and gateway outputs.
• Deny-path HTTP events carry http_response, and L7 events carry src_endpoint when the peer is known.
• Regression tests validate downgraded events against vendored 1.1.0 and 1.3.0 schemas.
• ocsf-json-export.mdx no longer claims structural identity and documents the per-class conversion table.

Alternatives Considered

• Remove the downgrade and move version mapping to the exporter proposed in feat(observability): define gateway-to-exporter extension contract #3224. Long term this is the likely home, but the exporter does not exist yet and users on older sinks need correct output now. This fix keeps the contract honest until then.
• Fill missing required fields with placeholders. Rejected: it produces fake audit data.
• Drop non-convertible events. Rejected as the default: it silently loses security records.

Agent Investigation

Verified against the OCSF schema tags v1.1.0, v1.3.0 and v1.8.0 and the official validator API. Related: #2717, #3264, #3554, #2640 (adds the trace object, which also needs removing for 1.3 and below), #3224. Depends on #4283, since even complete events currently fail on device.os.type_id.