fix(driver-sql): a date field still on a date-time column filters as its calendar day on SQLite and MySQL; operator note for Postgres days stored early

.changeset/33001-postgres-early-days-operator-note.md

@@ -0,0 +1,28 @@
+---
+'@objectstack/driver-sql': patch
+---
+
+docs(driver-sql): how an operator finds and corrects the Postgres `date` values that releases before 18.4.0 stored one day early after a `datetime` → `date` publish; the platform does not repair them
+
+Clause-②: no
+Metadata-protocol: none
+
+**Not repaired automatically.** Affected: Postgres, a database server whose time zone is ahead of UTC (for example `Asia/Shanghai`), a field published from `datetime` to `date`, and a day written into it after that publish by a release before 18.4.0. Such a day was stored as midnight in the server's zone (`2026-11-12` as `2026-11-11T16:00:00Z` at `+08:00`), so it reads, filters and buckets as the previous day. 18.4.0 writes new days correctly and reads every door the same way, but it cannot tell those rows from a value written while the field was a `datetime` at the same instant, so it changes no stored value.
+
+To find candidates, run against the object's table, with ZONE the server's zone (`SHOW TimeZone`) and FIELD the field:
+
+```sql
+SELECT id, FIELD FROM TABLE
+WHERE (FIELD AT TIME ZONE 'ZONE')::time = '00:00'
+  AND (FIELD AT TIME ZONE 'UTC')::time != '00:00';
+```
+
+Check each row against what was entered (a value that really was entered as a date-time at local midnight is also listed), then correct the ones that are early days:
+
+```sql
+UPDATE TABLE
+SET FIELD = ((FIELD AT TIME ZONE 'ZONE')::date)::timestamp AT TIME ZONE 'UTC'
+WHERE id IN (...);
+```
+
+After the update the row reads, filters and buckets as the day that was entered.

.changeset/33002-date-on-instant-column-filter.md

@@ -0,0 +1,10 @@
+---
+'@objectstack/driver-sql': patch
+---
+
+fix(driver-sql): on SQLite and MySQL, a `date` field whose column a `datetime` → `date` publish left as a date-time column now filters as its calendar day, so a row written while the field was a `datetime` is matched by a filter on the day it reads as
+
+Clause-②: no
+Metadata-protocol: none
+
+After a field is published from `datetime` to `date`, the column keeps its date-time type (SQLite `datetime`, MySQL `DATETIME`), and rows written before the publish still hold a time of day. They already read back as their day (`2026-11-12`), but an equality filter on that day (`where: { due: '2026-11-12' }`) compared the bare day against the whole stored instant and missed them, while `$gt` on that day matched them. The schema sync now records such a column on SQLite and MySQL as it already did for a Postgres `timestamptz` column, and every filter door (the query builder and the analytics native-SQL door) reads it as its calendar day: SQLite by the first ten characters of its canonical UTC text, MySQL by `date()` under its UTC session. Writes, reads and the day bucket are unchanged. As with Postgres, an index on such a column no longer serves a filter on the field; only columns a `datetime` → `date` publish left behind are affected.

packages/drivers/driver-sql/src/date-on-instant-column.ts

@@ -1,7 +1,7 @@
 // Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
 
 /**
- * [#32851] A `Field.date` whose Postgres column is still an INSTANT column.
+ * [#32851] A `Field.date` whose column is still an INSTANT column.
  *
  * `createColumn` gives a `date` field a `date` column, and the additive sync
  * never revisits an existing column's type. So a field retyped `datetime` →
@@ -37,14 +37,21 @@
  * (it drops the time of day a `datetime` wrote), which is a data change, not a
  * read rule.
  *
- * MySQL needs none of this: its session is pinned to UTC (`withUtcSession`)
- * and `DATETIME` is zone-naive, so a bare day IS UTC midnight on every door.
- * SQLite stores text. Only Postgres records the registry this module reads.
+ * [#33002] SQLite and MySQL keep the instant column too (SQLite's `datetime`,
+ * MySQL's `DATETIME`), and there the zone was never the problem: MySQL's
+ * session is pinned to UTC (`withUtcSession`) and `DATETIME` is zone-naive,
+ * SQLite stores the canonical UTC text. But a row written while the field was
+ * a `datetime` still holds its time of day, so it READ as its day while an
+ * equality filter compared the bare day against the whole instant and missed
+ * it. So a FILTER on such a column reads it as its calendar day on every
+ * dialect ({@link isInstantColumnType} records it, `filterColumnExpr` reads
+ * it); the write keeps the bare day there (it already is UTC midnight), and
+ * the day bucket already answered the day (`strftime` / `date_format`).
  */
 
 /**
- * The registry: per table, the `Field.date` fields whose physical Postgres
- * column is a `timestamp` type. Recorded by the schema sync from the column
+ * The registry: per table, the `Field.date` fields whose physical column is an
+ * instant type ({@link isInstantColumnType}). Recorded by the schema sync from the column
  * catalog it already reads, and EMPTY until a sync has looked — a driver that
  * never synced the table (deferred DDL, a guest datasource) keeps the plain
  * `date` path it had before.
@@ -66,13 +73,25 @@ export function isDateOnInstantColumn(
 }
 
 /**
- * Whether a Postgres catalog type (as knex `columnInfo()` reports it:
- * `timestamp with time zone`, `timestamp without time zone`) holds instants.
- * Only `timestamptz` is recorded: it is the column this driver builds for a
- * `datetime` field, and the one whose bare-day cast reads the session zone.
+ * Whether a catalog type (as knex `columnInfo()` reports it) is the instant
+ * column this driver builds for a `Field.datetime` on `dialect`, the one a
+ * `datetime` → `date` publish leaves under a `date` field:
+ *
+ * - Postgres: `timestamp with time zone` only, whose bare-day cast reads the
+ *   session zone (`timestamp without time zone` is never built here);
+ * - MySQL / MariaDB: `datetime` (and `timestamp`, which an older table may
+ *   carry);
+ * - SQLite: `datetime`, the declared type of the TEXT column holding the
+ *   canonical UTC instant (a `date` field's own column declares `date`).
+ *
+ * Any other dialect records nothing and keeps its `date` path.
  */
-export function isPostgresInstantColumnType(type: unknown): boolean {
-  return String(type ?? '').trim().toLowerCase() === 'timestamp with time zone';
+export function isInstantColumnType(dialect: string, type: unknown): boolean {
+  const t = String(type ?? '').trim().toLowerCase();
+  if (dialect === 'postgres') return t === 'timestamp with time zone';
+  if (dialect === 'mysql') return t === 'datetime' || t === 'timestamp';
+  if (dialect === 'sqlite') return t === 'datetime';
+  return false;
 }
 
 /** The fixed-width calendar day `temporalStorageForm(…, 'date')` answers. */
@@ -89,10 +108,30 @@ export function utcMidnightOfDay(day: unknown): unknown {
 }
 
 /**
- * The SQL that reads an instant column as its UTC calendar day — the day the
- * read door presents. `columnSql` is an identifier placeholder (`??`) or an
- * already-quoted column reference.
+ * The SQL that reads a Postgres instant column as its UTC calendar day — the
+ * day the read door presents. `columnSql` is an identifier placeholder (`??`)
+ * or an already-quoted column reference.
  */
 export function utcCalendarDaySql(columnSql: string): string {
   return `((${columnSql}) AT TIME ZONE 'UTC')::date`;
 }
+
+/**
+ * [#33002] The SQL that reads a MySQL `DATETIME` instant column as its calendar
+ * day. The column is zone-naive and the session is UTC, so `date()` answers the
+ * day the read presents.
+ */
+export function mysqlCalendarDaySql(columnSql: string): string {
+  return `date(${columnSql})`;
+}
+
+/**
+ * [#33002] The SQL that reads a SQLite instant column as its calendar day:
+ * the first ten characters of its canonical UTC text. `canonicalSql` is the
+ * column already put in canonical form (`sqliteCanonicalDatetimeSql`), so a
+ * legacy epoch-millisecond value answers its UTC day too, and a bare day
+ * (`YYYY-MM-DD`, what the write stores after the publish) answers itself.
+ */
+export function sqliteCalendarDaySql(canonicalSql: string): string {
+  return `substr(${canonicalSql}, 1, 10)`;
+}

packages/drivers/driver-sql/src/sql-driver-32851-date-on-instant-column.test.ts

@@ -23,12 +23,15 @@
  * as its UTC calendar day. See `date-on-instant-column.ts`.
  *
  * Every cell writes and reads the same rows: SQLite and MySQL never had the
- * defect (SQLite stores text; MySQL's session is UTC and `DATETIME` is
- * zone-naive) and must still answer the same day. The row written while the
- * field was a `datetime` is asserted through the filter and the bucket on
- * Postgres only: on SQLite and MySQL a bare-day comparand does not match a
- * stored instant with a time of day, the same on the pre-fix tree, and that is
- * not what this card changes.
+ * zone defect (SQLite stores text; MySQL's session is UTC and `DATETIME` is
+ * zone-naive) and must still answer the same day.
+ *
+ * [#33002] The row written while the field was a `datetime` is asserted
+ * through the filter and the bucket on EVERY cell. Before #33002, on SQLite
+ * and MySQL a bare-day comparand did not match that stored instant with a time
+ * of day although it read as that day (measured on SQLite at `e540baf8`:
+ * `due = '2026-11-12'` found only the row written after the publish); the
+ * filter now reads such a column as its calendar day there too.
  */
 
 import { describe, it, expect, beforeAll, afterAll } from 'vitest';
@@ -115,6 +118,9 @@ function measure(cell: DialectCell): void {
       expect(labelsOf(range as any[])).toEqual(expect.arrayContaining(['created', 'updated']));
       const before = await driver.find(TABLE, { where: { due: { $lt: DAY } } });
       expect(labelsOf(before as any[])).not.toContain('created');
+      expect(labelsOf(before as any[])).not.toContain('era');
+      const after = await driver.find(TABLE, { where: { due: { $gt: DAY } } });
+      expect(labelsOf(after as any[])).toEqual([]);
     });
 
     it('a `datetime` field beside it still reads and filters as its instant', async () => {
@@ -124,7 +130,7 @@ function measure(cell: DialectCell): void {
       expect(labelsOf(hit as any[])).toEqual(['created']);
     });
 
-    it.runIf(pg)('the row written as a `datetime` reads, filters and buckets as one day', async () => {
+    it('the row written as a `datetime` reads, filters and buckets as one day', async () => {
       const era: any = await driver.findOne(TABLE, { where: { id: ids.era } });
       expect(era.due).toBe(DAY);
 
@@ -138,11 +144,15 @@ function measure(cell: DialectCell): void {
       expect((grouped as any[]).map((g) => [g.due, Number(g.n)])).toEqual([[DAY, 3]]);
     });
 
-    it.runIf(pg)('the outside-the-builder doors (analytics native SQL) answer the same day', async () => {
-      const column = driver.temporalFilterColumnSql(TABLE, 'due', '"due"');
+    it('the outside-the-builder filter door (analytics native SQL) answers the same day', async () => {
+      const q = cell.id === 'mysql' ? '`' : '"';
+      const column = driver.temporalFilterColumnSql(TABLE, 'due', `${q}due${q}`);
       const comparand = driver.temporalFilterValue(TABLE, 'due', DAY);
-      const hit = rowsOf(await driver.execute(`select label from "${TABLE}" where ${column} = ? order by label`, [comparand]));
+      const hit = rowsOf(await driver.execute(`select label from ${q}${TABLE}${q} where ${column} = ? order by label`, [comparand]));
       expect(hit.map((r) => r.label)).toEqual(['created', 'era', 'updated']);
+    });
+
+    it.runIf(pg)('the outside-the-builder bucket door (analytics native SQL) answers the same day', async () => {
 
       const expr = driver.dateBucketSql(TABLE, 'due', 'day');
       const echoed = rowsOf(await driver.execute(`select ${expr} as b, count(*) as n from "${TABLE}" group by ${expr}`));

packages/drivers/driver-sql/src/sql-driver-filter-helpers.ts

@@ -67,7 +67,7 @@ import type { OracleUuidBindingRole } from './sql-driver-oracle-dialect.js';
 import { jsonMembershipUnsupportedOnOracleError } from './sql-driver-predicate-emitters.js';
 import type { ReadPresentationKind } from './sql-driver-top-level.js';
 import type { SqlDriver } from './sql-driver.js';
-import { isDateOnInstantColumn, utcCalendarDaySql } from './date-on-instant-column.js';
+import { isDateOnInstantColumn, mysqlCalendarDaySql, sqliteCalendarDaySql, utcCalendarDaySql } from './date-on-instant-column.js';
 
 /**
  * The `SqlDriver` members the bodies of {@link FilterHelpersFacet} read — every one of
@@ -359,11 +359,18 @@ export class FilterHelpersFacet {
   }
 
   /** The body of `SqlDriver#filterColumnExpr`; the method's docblock there is its contract. */
-  static filterColumnExpr(this: Pick<FilterHelpersHost, 'dateFields' | 'dateFieldsOnInstantColumn' | 'needsLegacyDatetimeRepair' | 'needsLegacyTimeRepair' | 'sqliteCanonicalDatetimeSql' | 'sqliteCanonicalTimeSql'>, table: string | null | undefined, field: string, column: string): { sql: string; bindings: any[] } | null {
+  static filterColumnExpr(this: Pick<FilterHelpersHost, 'dateFields' | 'dateFieldsOnInstantColumn' | 'isMysql' | 'isSqlite' | 'needsLegacyDatetimeRepair' | 'needsLegacyTimeRepair' | 'sqliteCanonicalDatetimeSql' | 'sqliteCanonicalTimeSql'>, table: string | null | undefined, field: string, column: string): { sql: string; bindings: any[] } | null {
     // [#32851] A `Field.date` on a Postgres `timestamptz` column compares as its
     // UTC calendar day, the day the read door presents — never cast in the
-    // session's zone. See `date-on-instant-column.ts`.
+    // session's zone. [#33002] On SQLite and MySQL such a column compares as
+    // its calendar day too, so a row written while the field was a `datetime`
+    // (an instant with a time of day) matches the day it reads as. See
+    // `date-on-instant-column.ts`.
     if (isDateOnInstantColumn(this, table, field)) {
+      if (this.isSqlite) {
+        return { sql: sqliteCalendarDaySql(this.sqliteCanonicalDatetimeSql('??')), bindings: [column, column, column, column] };
+      }
+      if (this.isMysql) return { sql: mysqlCalendarDaySql('??'), bindings: [column] };
       return { sql: utcCalendarDaySql('??'), bindings: [column] };
     }
     if (this.needsLegacyDatetimeRepair(table, field)) {
@@ -387,9 +394,13 @@ export class FilterHelpersFacet {
   }
 
   /** The body of `SqlDriver#temporalFilterColumnSql`; the method's docblock there is its contract. */
-  static temporalFilterColumnSql(this: Pick<FilterHelpersHost, 'dateFields' | 'dateFieldsOnInstantColumn' | 'needsLegacyDatetimeRepair' | 'needsLegacyTimeRepair' | 'sqliteCanonicalDatetimeSql' | 'sqliteCanonicalTimeSql'>, objectName: string, field: string, columnSql: string): string {
-    // [#32851] The outside-the-builder twin of `filterColumnExpr`'s arm.
-    if (isDateOnInstantColumn(this, objectName, field)) return utcCalendarDaySql(columnSql);
+  static temporalFilterColumnSql(this: Pick<FilterHelpersHost, 'dateFields' | 'dateFieldsOnInstantColumn' | 'isMysql' | 'isSqlite' | 'needsLegacyDatetimeRepair' | 'needsLegacyTimeRepair' | 'sqliteCanonicalDatetimeSql' | 'sqliteCanonicalTimeSql'>, objectName: string, field: string, columnSql: string): string {
+    // [#32851] [#33002] The outside-the-builder twin of `filterColumnExpr`'s arm.
+    if (isDateOnInstantColumn(this, objectName, field)) {
+      if (this.isSqlite) return sqliteCalendarDaySql(this.sqliteCanonicalDatetimeSql(columnSql));
+      if (this.isMysql) return mysqlCalendarDaySql(columnSql);
+      return utcCalendarDaySql(columnSql);
+    }
     if (this.needsLegacyDatetimeRepair(objectName, field)) {
       return this.sqliteCanonicalDatetimeSql(columnSql);
     }

packages/drivers/driver-sql/src/sql-driver-rotation.ts

@@ -43,7 +43,7 @@ import {
   isDeferredDdlLockWaitRefusal, isMultiValuedColumn, isNowDefaultValue,
 } from './sql-driver-top-level.js';
 import type { SqlDriver } from './sql-driver.js';
-import { isPostgresInstantColumnType } from './date-on-instant-column.js';
+import { isInstantColumnType } from './date-on-instant-column.js';
 
 /**
  * The `SqlDriver` members the bodies of {@link RotationFacet} read — every one of
@@ -563,11 +563,11 @@ export class RotationFacet {
 
         // [#32851] Read from the catalog just fetched, before any column below
         // is added: an added `date` column is built `date`, never an instant.
-        if (this.dialectName === 'postgres') {
-          for (const field of this.dateFields[tableName] ?? []) {
-            if (isPostgresInstantColumnType((columnInfo as Record<string, { type?: unknown }>)[field]?.type)) {
-              this.dateFieldsOnInstantColumn[tableName].add(field);
-            }
+        // [#33002] On every dialect that builds an instant column for a
+        // `datetime` field, not only Postgres.
+        for (const field of this.dateFields[tableName] ?? []) {
+          if (isInstantColumnType(this.dialectName, (columnInfo as Record<string, { type?: unknown }>)[field]?.type)) {
+            this.dateFieldsOnInstantColumn[tableName].add(field);
           }
         }
 

packages/drivers/driver-sql/src/sql-driver-sqlite-serialisation.ts

@@ -69,7 +69,7 @@ export interface SqliteSerialisationHost {
 /** The bodies of the "SQLite serialisation" section of `SqlDriver`: a stateless facet, called by the class's stubs. */
 export class SqliteSerialisationFacet {
   /** The body of `SqlDriver#formatInput`; the method's docblock there is its contract. */
-  static formatInput(this: Pick<SqliteSerialisationHost, 'dateFields' | 'dateFieldsOnInstantColumn' | 'datetimeFields' | 'fileColumnsMoved' | 'isOracle' | 'isSqlite' | 'jsonFields' | 'mediaFields' | 'oracleRawUuidFieldsByObject' | 'oracleUuidBinding' | 'storageDatetimeValue' | 'timeFields' | 'toDateOnly' | 'undeclaredAuditTimestampFields'>, object: string, data: any): any {
+  static formatInput(this: Pick<SqliteSerialisationHost, 'dateFields' | 'dateFieldsOnInstantColumn' | 'datetimeFields' | 'fileColumnsMoved' | 'isMysql' | 'isOracle' | 'isSqlite' | 'jsonFields' | 'mediaFields' | 'oracleRawUuidFieldsByObject' | 'oracleUuidBinding' | 'storageDatetimeValue' | 'timeFields' | 'toDateOnly' | 'undeclaredAuditTimestampFields'>, object: string, data: any): any {
     let copy: any = data;
     let copied = false;
 
@@ -155,10 +155,11 @@ export class SqliteSerialisationFacet {
         const day = this.toDateOnly(v);
         // Oracle binds the day as its UTC-midnight instant — `oracleDayBinding`.
         // [#32851] So does a Postgres `timestamptz` column a `datetime` → `date`
-        // publish left: a bare day would be read in the SESSION's zone.
+        // publish left: a bare day would be read in the SESSION's zone. On
+        // MySQL and SQLite (#33002) the bare day already is UTC midnight.
         const normalized = this.isOracle
           ? oracleDayBinding(day)
-          : isDateOnInstantColumn(this, object, field) ? utcMidnightOfDay(day) : day;
+          : isDateOnInstantColumn(this, object, field) && !this.isMysql && !this.isSqlite ? utcMidnightOfDay(day) : day;
         if (normalized !== v) {
           if (!copied) { copy = { ...copy }; copied = true; }
           copy[field] = normalized;