Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions .changeset/eql-types-as-record-fields.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
---
'@cipherstash/eql': minor
---

`eql-bindings`, the Rust crate released with this package: an EQL v4 type can now be a field of a Stack Encrypt record.

- A `#[derive(EncryptTarget, DecryptSource)]` record with `struct = ..` can have fields such as `email: TextEq` or `profile: JsonTreeDocument`. The field's column is the record's scope and the field's name, so `scope = "members"` and the field `email` store `{"t": "members", "c": "email"}` in `i`.
- With no scope and no binding, a field writes the same terms as the type written on its own under that column, and as a data plan that names the type. A data plan is how the Go module writes the type.
- A record can list a scope or a binding, such as `scope(tenant: u64)` or `bind(row: u128)`. The terms of an EQL field then derive under the column and the scope. The ciphertext is sealed under the column, the scope and the binding. A field of any other type works the same way. `i` stores the column alone, so the reader supplies the scope and the binding again. The same is true of an EQL field of a record nested in another.
- A query from a field's marker takes the record's scope with `.scope(..)`. It gives the term of the values stored under that scope.
- A data plan that names an EQL type takes a scope and a binding in the same way. So a Go struct with an `encrypt_into` field now seals through `cipher.Scope(..).Bind(..)`, where it was refused.
- `JsonTreeDocument` now implements `EncryptTarget` and `DecryptSource` for `JsonPlaintext`. It encrypts under the default options, with the column as its context.

The stored format, the SQL bundle, and the TypeScript types do not change.
2 changes: 1 addition & 1 deletion AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -85,7 +85,7 @@ Every npm package except EQL lives under `languages/typescript/`: packages in `l
- `languages/typescript/packages/utils`: Shared config (`utils/config`) and logger (`utils/logger`)
- `languages/typescript/packages/bench`: Performance / index-engagement benchmarks (private, not published)
- `languages/typescript/packages/protect-ffi`: Native FFI bindings to the CipherStash Client SDK (`@cipherstash/protect-ffi`) — the Rust core that `languages/typescript/packages/stack` encrypts and decrypts through, absorbed from `cipherstash/protectjs-ffi`. Contains a **nested Cargo workspace** (`crates/`) and six per-platform binary packages under `platforms/*`, each published as `@cipherstash/protect-ffi-<platform>` and linked here via `workspace:*`. Also holds the repo's live FFI integration suite at `integration-tests/` — a private workspace member (`@cipherstash/ffi-integration-tests`) enrolled by its own literal entry in `pnpm-workspace.yaml`, needing Docker and credentials, and deliberately carrying **no `test` script** so `pnpm test` cannot reach it. See the "Working on protect-ffi" notes below before touching it — its default `test` and `build` are deliberately Rust-free.
- `packages/eql`: The Encrypt Query Language subtree — the SQL bundle that stores and queries encrypted payloads — absorbed from `cipherstash/encrypt-query-language`. **The directory is the subtree root, not the package.** It was imported at a *verbatim prefix* so its repo-root-relative paths (mise tasks, `Doxyfile`, `sync-generated.mjs`) keep resolving, which puts the npm package `@cipherstash/eql` two levels down at `packages/eql/packages/eql` — the same shape as `languages/typescript/packages/protect-ffi/platforms/*`, and enrolled the same way, by an explicit `packages/eql/packages/*` glob in `pnpm-workspace.yaml`. The subtree root deliberately carries no `package.json`. Also contains a **nested Cargo workspace** at `packages/eql/crates/` (`eql-bindings`, published in lockstep with the npm package, plus `eql-domains` / `eql-codegen` / `eql-tests-macros`, which are not), a SQLx test crate at `packages/eql/tests/sqlx`, an unpublished encryption test crate at `packages/eql/tests/encryption` (real encryption with a fake key source, the executable Rustdoc example at `packages/eql/crates/eql-bindings/src/encryption/example.rs`, plus optional PostgreSQL coverage), an ~900-line `mise.toml` task surface, its own `AGENTS.md`, and `docs/`. See the "Working on EQL" notes below before touching it.
- `packages/eql`: The Encrypt Query Language subtree — the SQL bundle that stores and queries encrypted payloads — absorbed from `cipherstash/encrypt-query-language`. **The directory is the subtree root, not the package.** It was imported at a *verbatim prefix* so its repo-root-relative paths (mise tasks, `Doxyfile`, `sync-generated.mjs`) keep resolving, which puts the npm package `@cipherstash/eql` two levels down at `packages/eql/packages/eql` — the same shape as `languages/typescript/packages/protect-ffi/platforms/*`, and enrolled the same way, by an explicit `packages/eql/packages/*` glob in `pnpm-workspace.yaml`. The subtree root deliberately carries no `package.json`. Also contains a **nested Cargo workspace** at `packages/eql/crates/` (`eql-bindings`, published in lockstep with the npm package, plus `eql-domains` / `eql-codegen` / `eql-tests-macros`, which are not), a SQLx test crate at `packages/eql/tests/sqlx`, an unpublished encryption test crate at `packages/eql/tests/encryption` (real encryption with a fake key source, the executable Rustdoc example at `packages/eql/crates/eql-bindings/src/encryption/example.rs`, the Rust examples of `packages/eql/crates/eql-bindings/README.md` run as doctests, plus optional PostgreSQL coverage), an ~900-line `mise.toml` task surface, its own `AGENTS.md`, and `docs/`. See the "Working on EQL" notes below before touching it.
**Repository ownership:** EQL now lives in `cipherstash/stack`. File and update
EQL issues in this repository, never in the historical
`cipherstash/encrypt-query-language` repository. Old upstream issue and PR
Expand Down
7 changes: 5 additions & 2 deletions conformance/stack-encrypt/declaration-grammar.md
Original file line number Diff line number Diff line change
Expand Up @@ -149,7 +149,8 @@ With `bind`:
part, a list inside the list, is refused (`bind`).
- Every field's stored context is its label, extended, then the parts in
order. No query context includes them, so a query on the plan is refused
(`bind_in_query`). No target field is admitted.
(`bind_in_query`). A target field is bound as any other field is: its
EQL value stores the label's column, and not the parts.
- The key is reserved, like `context_field`: a field named `bind` is refused
(`bind`).

Expand Down Expand Up @@ -223,7 +224,9 @@ must never silently turn into a ciphertext-only field.

For a resolver that supplies EQL types, additional existing rules apply: the
target must be producible, its column label has exactly two segments, and the
plan has neither extensions nor a context field. An explicit kind matches the
plan has no context field. Extensions and `bind` parts come after a target
field's label, as they do for any other field; the EQL value stores the
label's column alone. An explicit kind matches the
target's plaintext kind; otherwise that kind is inferred. Only a document target
accepts JSON options. Sealed-null rules include the target's own indexes.
Those resolver-dependent outcomes and normalization need an EQL profile; this
Expand Down
72 changes: 45 additions & 27 deletions docs/plans/2026-10-24-stack-encrypt-api-plan.md
Original file line number Diff line number Diff line change
Expand Up @@ -253,7 +253,8 @@ The field is then stored as one EQL column, such as

Go records already do this through the dynamic interface. A field there
names its EQL type, such as `"target": "TextEq"`, and the engine runs that
type for the field. The Rust record derive does not support it yet.
type for the field. The Rust record derive does the same (#1658). The
field's column is the record's scope and the field's name.

```rust
use eql_bindings::v4::{integer::IntegerOrd, text::TextEq};
Expand All @@ -265,8 +266,6 @@ struct Member {
age: i32,
}

// Not on `next` yet in Rust: EQL types as fields of a derived record.
// Go records do this today through the dynamic interface.
#[derive(EncryptTarget, DecryptSource)]
#[stash(struct = Member, scope = "members")]
struct EncryptedMember {
Expand Down Expand Up @@ -401,9 +400,8 @@ use stack_encrypt::json::JsonPlaintext;

let column = Identifier::for_column("customers", "profile")?;

// Not on `next` yet: the new traits for EQL types, and the public
// `JsonPlaintext`. Today this needs `keyset.encrypt_as(&value, context)`,
// a `serde_json::Value`, and an owned context.
// Not on `next` yet: a context built with `Scope::new`. Today this
// passes the column itself: `.encrypt_into_with_context(&keyset, column.clone())`.
let profile: JsonPlaintext = json!({ "name": "Ada", "age": 35 }).into();
let encrypted: JsonTreeDocument = profile
.encrypt_into_with_context(&keyset, Scope::new(&column))
Expand Down Expand Up @@ -1483,31 +1481,51 @@ Base64.
#### EQL types as fields

- **What to build.** EQL types such as `TextEq` and `JsonTreeDocument` as
field types of the Rust record derive. In Rust today, an EQL type works
only on its own, through the older structural API, with a
`NonEmpty<Identifier>` context. The EQL types do derive the structural
`EncryptFrom` and `DecryptInto`. A record still cannot use them as
fields.
A record with a `TextEq` field, built against `next` on 7 November 2026,
fails with three errors. `NonEmpty<Identifier>` and
`ExpectedContext<Identifier>` do not implement `From<CallerContext>`, and
the record derive gives each field a `CallerContext`.
- **The smallest fix.** Convert a `CallerContext` of two parts into an
`Identifier`, with the table first and the column second. `Identifier`
already gives the same context as that pair, so the stored bytes do not
change.
field types of the Rust record derive.
- **How it works.** The record derive gives each field a context of its
own: the record's scope, then the field's name. A field type that stores
its context stores that own context, as the type it stores. The stored
type implements `target::ContextFromField`. `Identifier` takes an own
context of two parts, the table first and the column second, and gives
the same context as that pair, so the stored bytes do not change.
- **What `i` stores.** The declared names only: the table and the column.
A record can also list a scope or a binding, supplied by the caller or
stored in the row. The terms of an EQL field derive under the query
context: the column, then the scope. The ciphertext is sealed under the
stored context: the column, then the scope, then the binding. A field
of any other type works the same way. `i` does not store the scope or
the binding, so the reader supplies them again, and a query supplies
the scope. An EQL field of a record nested in another stores its own
column in the same way.
- **Why.** This gives the [EQL field example](#store-a-field-as-an-eql-type)
and the [JSON example](#encrypt-a-json-document).
- **Done already.** The dynamic interface supports EQL fields, so Go
records use them today. A field there names its EQL type, and the engine
looks the type up by name through `eql-bindings` (the `TargetResolver`
in `dynamic/record.rs`). `Identifier` already gives the same context as
the `(table, column)` pair that a record derive builds.
- **Tracked in.** Issue #1486 is open, in part. No dedicated issue. Pull request #1504 defers the EQL owned implementations. `eql-bindings` builds against the crates.io `stack-encrypt`, so a `stack-encrypt` release must come first.
- **Done already.** In #1658:
- A Rust record with EQL fields encrypts and decrypts through the new
API, with or without a scope and a binding, and when it is nested in
another record.
- For a field of each kind (equality, storage only, order, and a JSON
document), a record with no scope writes the terms that a data plan
writes for the same declaration. A data plan is the path Go takes.
Each path opens the other's values
(`packages/eql/tests/encryption/tests/record_fields.rs`).
- `JsonTreeDocument` implements `EncryptTarget<JsonPlaintext>` and
`DecryptSource<JsonPlaintext>`.
- A query from a field's marker works for the equality and match
types, with the record's scope. A plan's query for an EQL type takes
a scope too, through the plan chain that #1488 removes. The order
types do not implement `Answers` yet, so a marker cannot query them.
- A data plan that names an EQL type takes a scope and a binding too.
`TargetResolver` receives them after the field's label, and
`TargetError::Extended` is removed. Under the same scope and binding,
a data plan and a derived record write the same terms, and each
opens the other's values. A Go struct with an `encrypt_into` field
seals through `cipher.Scope(..).Bind(..)`.
- **Tracked in.** Issue #1658. `eql-bindings` builds against the crates.io `stack-encrypt`, so a `stack-encrypt` release with `ContextFromField` must come before the `eql-bindings` release that uses it.
- **Finished when** these are all true:
- a Rust record with EQL fields encrypts, decrypts, and queries through
the new API;
- it writes the same bytes as a Go record with the same declaration.
the new API, including order queries;
- it writes the same terms as a Go record with the same declaration,
with and without a scope.

#### A ready-made EQL resolver

Expand Down
22 changes: 22 additions & 0 deletions languages/golang/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,28 @@ listed under **Breaking**. Releases are tagged `languages/golang/vX.Y.Z`.

## [Unreleased]

### Changed

- **An `encrypt_into` field takes a scope and a binding.** A cipher with a
scope or a binding refused a struct with a field that names an EQL type,
as `ErrEncoding`. It now seals the field as it seals any other. The scope
and the binding come after the column in the field's context: the search
terms derive under the column and the scope, and the value is sealed
under the binding too. The EQL value stores the table and the column
alone, in `i`. Open it through a cipher with the same scope and binding,
and query it through a cipher with the same scope. `Plan.Validate` takes
a plan with `Extension` or `Bind` and a target field. Values stored
through a cipher with no scope and no binding do not change.
- A `JsonSearch` document is found and read in the same way. A query,
and a read prepared with `Select` or `Subtree`, derive under the
column and the scope.
- `Cipher.JSONOpen` opens a read's rows through a cipher with a binding,
where it refused one. Every row of one call opens under that binding.
- `Select` and `Subtree` open with the cipher that prepared the read,
and preparing a read takes no binding. So they read a document sealed
with a scope, and do not open one sealed with a binding. Open that
document whole, through the cipher with the binding.

## [0.1.0-rc.4] - 2026-11-13

### Upgrading from rc.3
Expand Down
2 changes: 1 addition & 1 deletion languages/golang/cmd/stashgen/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -443,7 +443,7 @@ Put the column on the left of `@@` (`WHERE name @@ $1::eql_v4.query_text_match`)
Separate columns work today for four indexes: `equality`, `match`, `ore` and `ope`.

A field with `encrypt_into` is stored under its table and column, which is what an EQL value records in its `i`.
A cipher extended with a tenant part (`cipher.Extend(...)`) has no column for the extended label, so it refuses a struct with an `encrypt_into` field; use `index=` columns for a tenant-extended struct until that rule is settled.
A cipher with a scope or a binding (`cipher.Scope(tenant).Bind(rowID)`) seals a struct with an `encrypt_into` field as it seals any other. The scope and the binding come after the column in the field's context, and `i` still holds the table and the column alone. Open the value through a cipher with the same scope and binding, and query it through a cipher with the same scope.

## When stashgen stops

Expand Down
18 changes: 14 additions & 4 deletions languages/golang/encrypt/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -599,10 +599,20 @@ the column's `eql_v4.query_text_eq` operand takes; `TextEq` answers `eq`
alone, and a `TextMatch` field's `Matches` returns an
`encrypt.EQLMatchQuery`: the `eql.TextMatchQuery` operand with `Verify` and
`TargetLimit`. `mise run
wasm:guest:build:eql` builds that guest beside the other. A cipher extended
with a tenant part refuses a struct with an `encrypt_into` field: an EQL
value is stored under a table and a column, and the extended label has no
column.
wasm:guest:build:eql` builds that guest beside the other.

A cipher with a scope or a binding (`cipher.Scope(tenant).Bind(rowID)`)
seals a struct with an `encrypt_into` field as it seals any other struct.
The scope and the binding come after the column in the field's context.
The EQL value stores the table and the column alone, in `i`, so:

- open the value through a cipher with the same scope and binding;
- query it through a cipher with the same scope. A binding is in no query;
- read part of a `JsonSearch` document (`Select`, `Subtree`) through a
cipher with the same scope. These readers open with the cipher that
prepared the read, and preparing a read takes no binding. So they do not
open a document sealed with a binding: open that document whole, through
the cipher with the binding.

### Encrypted JSON documents

Expand Down
11 changes: 6 additions & 5 deletions languages/golang/encrypt/bindings_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -130,16 +130,17 @@ func TestAFieldNamedBindIsRefused(t *testing.T) {
}
}

// An EQL type stores its column alone, so a binding has no column to sit
// under: the plan is refused, as an extended one is.
func TestABindingIsRefusedOnAnEQLType(t *testing.T) {
// An EQL type stores its column alone. A binding comes after the column in
// the field's context, so a plan with an EQL field takes one, as it takes a
// scope. The whole behaviour is in TestAnEQLFieldTakesAScopeAndABinding.
func TestABindingIsTakenOnAnEQLType(t *testing.T) {
p := &record.Plan{
Context: []string{"users"},
Bind: []any{uint64(9)},
Fields: []record.Field{{Name: "email", Kind: record.String, Target: "TextEq"}},
}
if err := p.Validate(); err == nil {
t.Fatal("a bound EQL plan validated")
if err := p.Validate(); err != nil {
t.Fatalf("a bound EQL plan: %v", err)
}
}

Expand Down
Loading
Loading