Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 16 additions & 0 deletions .changeset/31592-oracle-raw16-reference-exact.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
---
'@objectstack/spec': minor
---

feat(spec): an Oracle `RAW(16)` uuid column backs a `lookup`, `master_detail` or `user` field exactly again; its `text` pairing stays lossy

Clause-②: yes (widening)
Metadata-protocol: semantic

`isCompatible(rawType, fieldType, 'oracle')` now answers `true` instead of `'lossy'` for a `RAW(16)` column against `lookup`, `master_detail` and `user`. The SQL driver binds a comparison value, a written value and a by-id key for a `RAW(16)` column as `HEXTORAW(?)`, so a reference field over a `SYS_GUID()` key is filtered, written and resolved by the uuid string it reads. Before that bind existed, every pairing over the column was judged `'lossy'`.

What a caller sees:

- **Draft and validation.** A field of those three types over a `RAW(16)` column is no longer reported as a `type_mismatch` warning by object validation.
- **`text` is unchanged.** A `text` field over a `RAW(16)` column still answers `'lossy'`, and a draft still marks such a column for review. Its pattern operators (`$contains`, `$startsWith`, `$like` and the rest) are sent as text patterns, and Oracle compares them with the column's hex text (32 upper-case digits, no hyphens), not with the uuid string a read returns.
- **Other dialects and other `RAW` widths** answer as before.
21 changes: 15 additions & 6 deletions content/docs/data-modeling/external-datasources.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -555,7 +555,7 @@ object whose `remoteName` is `OWNER.TABLE`.
| `DATE` | `datetime` | An Oracle `DATE` carries a time of day. A `date` field over it is judged lossy: the time is dropped. |
| `TIMESTAMP WITH TIME ZONE` | `datetime` | Compared and returned as UTC instants. |
| `CLOB` | `textarea` | |
| `RAW(16)` | `text` | Read as a lower-case UUID string. Every field pairing over it is judged lossy, so a draft marks it for review: it cannot be filtered or written by that string yet (see below). |
| `RAW(16)` | `text` | Read as a lower-case UUID string, and filtered and written by that string (see below). A `lookup`, `master_detail` or `user` field over it is exact. A `text` field is judged lossy, so a draft marks it for review: `$contains` and the other pattern operators do not match it as the string it reads. |

The driver pins every session to UTC. Thin mode decodes `DATE` and `TIMESTAMP`
values with the server process's local clock, and the driver repairs that on read.
Expand All @@ -573,11 +573,20 @@ the process's own time zone. Run the server with `TZ=UTC` and there is no such h
`MERGE`. Use create and update.
- **JSON-array membership** (`$contains` on a multi-value field) is refused. There is
no Oracle construct modelled for it.
- **Filtering or writing a `RAW(16)` column by its UUID string is not supported.**
The driver presents the value as a string on read, but it does not convert the
string back to `RAW` for a comparison or a write. This includes a `lookup` keyed
by a `SYS_GUID()` column. Binding through `HEXTORAW` is tracked in
[#31390](https://github.com/objectstack-ai/objectstack/issues/31390).
- **A `RAW(16)` column is filtered and written by its UUID string.** The driver
reads which of an object's columns are `RAW(16)` from the catalog, and binds a
comparison value (`=`, `$ne`, `$in`, the range operators), a written value and a
record's key for those columns as `HEXTORAW(?)`. That covers a `SYS_GUID()` key and
a `lookup` keyed by one. A UUID is accepted with hyphens (the form a read returns)
or as its 32 hex digits, in either case. Any other value is refused before the
statement is sent: in a filter with `INVALID_FILTER`, in a write with
`VALIDATION_ERROR`. So is a create that gives no id on an object keyed by such a
column, because the id the driver would mint is not a UUID. A `VARCHAR2` column
that holds UUIDs is still compared as text.
- **Pattern operators on a `RAW(16)` column** (`$contains`, `$startsWith`, `$like`
and the rest) are sent as text patterns. Oracle compares them with the column's
hex text: 32 upper-case digits, no hyphens. A fragment of the UUID as it reads
back matches only when it has no hyphen and the operator ignores case.
- A `NUMBER` beyond 2^53 is read as a JavaScript number and loses precision.
- **Paged reads are stable when the object's `columnMap` maps a column to `id`.**
That column breaks ties after the caller's own sort. Without a mapped key, a paged
Expand Down
12 changes: 7 additions & 5 deletions docs/adr/0015-external-datasource-federation.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
**Deciders**: ObjectStack Protocol Architects
**Builds on**: [ADR-0005](./0005-metadata-customization-overlay.md) (one Zod source of truth per metadata type)
**Consumers**: `@objectstack/spec`, `@objectstack/runtime`, `@objectstack/objectql`, `@objectstack/plugins/driver-sql`, `@objectstack/services/service-ai`, `@objectstack/cli`, `../objectui`
**Amended by**: [ADR-0143](./0143-outbound-connector-auth-is-open-tier.md) (2026-10-27), attribution only — see the note below. D7 (2026-10-29, ruling 6486136301 on #27298) — an API-backed datasource may borrow a declared connector instance's credential through `external.connector`; see the D7 note below. #27688 (2026-10-30), example only — §8's `credentialsRef` is a `sys_secret:` handle, the only form the shipped secret binder resolves; see the note under §8. #27917 (2026-10-30), dialect list only — §4.6 names the `odata` vocabulary the matrix gained with #27745. #28208 (2026-10-31, ruling 6525963988 on #27687) — a datasource defined in code reads its credential at connect from `OS_DATASOURCE_<NAME>_PASSWORD`, behind the host switch `OS_DATASOURCE_ENV_CREDENTIALS`; see the note under §10 and the correction in the D7 note. 2026-11-03 (maintainer ruling in the security channel, D1 = B, D2 = (c2)+(a)) — under a walled tenancy posture every external object states its organization scope, or the boot is refused by name; see the note under §10 and the sentence added to the D7 note. #31328 (2026-11-08) — §4.6 names the `oracle` vocabulary, with how `NUMBER(p,s)` and `RAW(16)` are read, and its federation-only scope; §5.1 gains the federation-only arm of the DDL gate. #31327 (2026-11-08) — a second federation-only driver, SQL Server (`mssql`): §4.6 names the `mssql` vocabulary and its one dialect-only lossy rule, and the two dialects' lossy moves share one table; §5.1's federation-only arm names both drivers.
**Amended by**: [ADR-0143](./0143-outbound-connector-auth-is-open-tier.md) (2026-10-27), attribution only — see the note below. D7 (2026-10-29, ruling 6486136301 on #27298) — an API-backed datasource may borrow a declared connector instance's credential through `external.connector`; see the D7 note below. #27688 (2026-10-30), example only — §8's `credentialsRef` is a `sys_secret:` handle, the only form the shipped secret binder resolves; see the note under §8. #27917 (2026-10-30), dialect list only — §4.6 names the `odata` vocabulary the matrix gained with #27745. #28208 (2026-10-31, ruling 6525963988 on #27687) — a datasource defined in code reads its credential at connect from `OS_DATASOURCE_<NAME>_PASSWORD`, behind the host switch `OS_DATASOURCE_ENV_CREDENTIALS`; see the note under §10 and the correction in the D7 note. 2026-11-03 (maintainer ruling in the security channel, D1 = B, D2 = (c2)+(a)) — under a walled tenancy posture every external object states its organization scope, or the boot is refused by name; see the note under §10 and the sentence added to the D7 note. #31328 (2026-11-08) — §4.6 names the `oracle` vocabulary, with how `NUMBER(p,s)` and `RAW(16)` are read, and its federation-only scope; §5.1 gains the federation-only arm of the DDL gate. #31327 (2026-11-08) — a second federation-only driver, SQL Server (`mssql`): §4.6 names the `mssql` vocabulary and its one dialect-only lossy rule, and the two dialects' lossy moves share one table; §5.1's federation-only arm names both drivers. #31592 (2026-11-09) — §4.6: with the `HEXTORAW` bind of #31390 on `main`, an Oracle `RAW(16)` column backs a `lookup`, `master_detail` or `user` field exactly again; only its `text` pairing stays `'lossy'`.

> **Amendment note D7 (2026-10-29, ruling 6486136301, letter A) — a second credential channel.**
> §10 says credentials reach a datasource only through `credentialsRef`. That stays true for a
Expand Down Expand Up @@ -429,9 +429,11 @@ read before the matrix strips them (SQL Server's `(max)` length does not: an
integral `NUMBER(p)` / `NUMBER(p,0)` (a negative scale counts its `p − s`
digits) is `integer` up to nine digits and `bigint` beyond, `NUMBER(*,0)` —
Oracle's `INTEGER` — included; a bare `NUMBER` is `decimal`. `RAW(16)` is
`uuid`, any other `RAW` `binary`; on `oracle` every pairing over that `uuid` is
judged `'lossy'`, not exact, because the driver reads it as a UUID string but
cannot yet bind that string back to `RAW` for a filter or a write (#31390). `DATE` carries a time of day, so it is
`uuid`, any other `RAW` `binary`. The driver reads it as a UUID string and binds
that string back through `HEXTORAW` for a comparison, a write and a by-id key
(#31390), so a `lookup`, `master_detail` or `user` field over it is exact; a
`text` field over it stays `'lossy'` on `oracle`, because its pattern operators
match the column's hex text, not the string it reads. `DATE` carries a time of day, so it is
`datetime`. A pre-23ai flag column (`NUMBER(1)` / `CHAR(1)`) is judged as the
integer or text it is, never coerced to `boolean`. The `oracle` and `mssql` driver ids
serve federation only: a datasource naming either with `schemaMode: 'managed'`
Expand All @@ -443,7 +445,7 @@ sync would have nothing to send.
names, and maps `timestamp` to binary, because on SQL Server it is the
`rowversion` counter and not a date-time. A dialect moves a pairing to
`'lossy'` in one table, `DIALECT_LOSSY_RULES`, and only ever toward `'lossy'`:
`oracle` demotes the exact pairings of `uuid` (above), and `mssql` adds one, a
`oracle` demotes the exact `text` pairing of `uuid` (above), and `mssql` adds one, a
text column backing a `json`, `tags` or `multiselect` field, because SQL Server
keeps JSON in `nvarchar(max)` and the SQL driver stores those fields there as
JSON text (#31327).
Expand Down
14 changes: 10 additions & 4 deletions packages/spec/src/data/type-compat.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -392,12 +392,18 @@ describe('the oracle dialect (ADR-0015 §4.6)', () => {

it('reads RAW(16) as a UUID and every other RAW width as binary', () => {
expect(canonicalizeSqlType('RAW(16)', 'oracle')).toBe('uuid');
// Read as a UUID string, but the driver cannot yet filter or write it by
// that string, so every pairing over it is flagged for review on oracle.
for (const field of ['text', 'lookup', 'master_detail', 'user'] as const) {
expect(isCompatible('RAW(16)', field, 'oracle'), field).toBe('lossy');
// The driver filters, writes and keys a RAW(16) column by the UUID string
// it reads (HEXTORAW), so a reference field over it is exact on oracle...
for (const field of ['lookup', 'master_detail', 'user'] as const) {
expect(isCompatible('RAW(16)', field, 'oracle'), field).toBe(true);
}
// ...and a text field stays flagged for review: its pattern operators
// match the column's hex text, not the string it reads.
expect(isCompatible('RAW(16)', 'text', 'oracle')).toBe('lossy');
expect(suggestFieldTypeForSqlType('RAW(16)', 'oracle')).toBe('text');
// The demotion names its field types; a type the uuid row does not pair
// exactly is not promoted by it.
expect(isCompatible('RAW(16)', 'number', 'oracle')).toBe(false);
// Other dialects' uuid columns keep their exact pairings.
expect(isCompatible('uuid', 'lookup', 'postgres')).toBe(true);
expect(canonicalizeSqlType('RAW(32)', 'oracle')).toBe('binary');
Expand Down
31 changes: 19 additions & 12 deletions packages/spec/src/data/type-compat.ts
Original file line number Diff line number Diff line change
Expand Up @@ -208,8 +208,8 @@ const DIALECT_ALIASES: Partial<Record<SqlDialect, Record<string, CanonicalSqlTyp
*
* `RAW(16)` is the conventional Oracle column for a UUID / GUID (`SYS_GUID()`
* returns one), so it maps to `uuid`; every other width is `binary`. The
* field pairings over it are `'lossy'` on oracle, not exact: see
* {@link DIALECT_LOSSY_RULES}.
* reference pairings over it are exact; a `text` field over it is `'lossy'`
* on oracle: see {@link DIALECT_LOSSY_RULES}.
*
* `undefined` for any other spelling — the caller then strips parameters and
* reads the alias tables as for every dialect.
Expand Down Expand Up @@ -427,13 +427,20 @@ export function suggestFieldTypeForSqlType(rawType: string, dialect?: SqlDialect
* — never exact, never suggested — so a draft carries a `// REVIEW:` marker
* and validation warns instead of passing silently or failing:
*
* - `demoteExact` — the canonical type's EXACT pairings, which this dialect
* can only half-serve, answer `'lossy'` here.
* `oracle` / `uuid`: a `RAW(16)` column reads back as a UUID string, but the
* driver binds a string comparand or write value as text, and Oracle refuses
* text against RAW. A field over it (a `lookup` keyed by a `SYS_GUID()`
* column included) displays correctly and cannot yet be filtered or written
* by its value. Binding through `HEXTORAW` is #31390.
* - `demoteExact` — the EXACT pairings of the canonical type named here,
* which this dialect can only half-serve, answer `'lossy'` here.
* `oracle` / `uuid` → `text`: the SQL driver reads a `RAW(16)` column as a
* lower-case hyphenated UUID string, and binds a comparison comparand, a
* written value and a by-id key for it as `HEXTORAW(?)` over that string's
* hex digits (#31390). A reference field over it — a `lookup` or
* `master_detail` keyed by a `SYS_GUID()` column, a `user` — is filtered,
* written and resolved by the very string it reads, so those pairings stay
* exact. A `text` field is still half-served: its pattern operators
* (`$contains`, `$startsWith`, `$like`, …) are not bound through
* `HEXTORAW`, so they match the column as the text Oracle converts a
* `RAW` to (its 32 hex digits, upper case, no hyphens), not as the string
* it reads; a fragment of that string with a hyphen, or in the case it is
* read in, answers no row instead of a refusal.
* - `addLossy` — field types the base row answers `false` for, which this
* dialect's column can carry with loss.
* `mssql` / `text`: SQL Server (before its 2025 `json` type) keeps JSON in
Expand All @@ -446,12 +453,12 @@ export function suggestFieldTypeForSqlType(rawType: string, dialect?: SqlDialect
* column shape the driver relies on (#31327).
*/
interface DialectLossyRule {
readonly demoteExact?: true;
readonly demoteExact?: readonly FieldType[];
readonly addLossy?: readonly FieldType[];
}

const DIALECT_LOSSY_RULES: Partial<Record<SqlDialect, Partial<Record<CanonicalSqlType, DialectLossyRule>>>> = {
oracle: { uuid: { demoteExact: true } },
oracle: { uuid: { demoteExact: ['text'] } },
mssql: { text: { addLossy: ['json', 'tags', 'multiselect'] } },
};

Expand All @@ -468,7 +475,7 @@ export function isCompatible(
const canonical = canonicalizeSqlType(rawType, dialect);
const entry = CANONICAL_TO_FIELD[canonical];
const rule = dialect ? DIALECT_LOSSY_RULES[dialect]?.[canonical] : undefined;
if (entry.exact.includes(fieldType)) return rule?.demoteExact ? 'lossy' : true;
if (entry.exact.includes(fieldType)) return rule?.demoteExact?.includes(fieldType) ? 'lossy' : true;
if (entry.lossy.includes(fieldType) || rule?.addLossy?.includes(fieldType)) return 'lossy';
// Unknown remote types are permissive-but-lossy against text/json only,
// already encoded in CANONICAL_TO_FIELD.unknown.
Expand Down
Loading