Skip to content

stack-encrypt: a derived record cannot have an EQL type as a field, and JsonTreeDocument has only the old traits #1658

Open

Description

@coderdan

Background

Stack Encrypt (packages/stack-encrypt) is the Rust engine under every CipherStash language SDK. EQL (Encrypt Query Language) is the SQL library we install into a customer's PostgreSQL database. It stores each encrypted value as a JSON payload in a typed column, such as public.eql_v4_text_eq. The eql-bindings crate (packages/eql/crates/eql-bindings) has one Rust type for each EQL v4 column type, such as TextEq, IntegerOrd or JsonTreeDocument.

A Rust program describes a table row as a record. #[derive(EncryptTarget, DecryptSource)] on a struct = .. record encrypts each field of the plaintext into the type of the matching field. Each field is encrypted under its own context: the record's scope and the field's name, such as ("members", "email"). An EQL type stores its column in the payload as i, a table and a column (Identifier). The two describe the same thing, and give the same context (eql-bindings/src/encryption.rs, the comment on impl Describe for Identifier).

The Stack Encrypt API plan item "EQL types as fields" asks for this. #1585 gave the scalar EQL types the new traits, and left this item and the JSON tree types out of scope.

Problem

1. A record cannot have an EQL type as a field. This record, from the plan's example "Store a field as an EQL type", does not compile:

#[derive(EncryptTarget, DecryptSource)]
#[stash(struct = Member, scope = "members")]
struct EncryptedMember {
    #[stash(passthrough)]
    id: u64,
    email: TextEq,
    age: IntegerOrd,
}

For each EQL field, the derive gives the field a CallerContext and cannot convert it:

the trait bound `NonEmpty<Identifier>: From<CallerContext>` is not satisfied
the trait bound `ExpectedContext<Identifier>: From<CallerContext>` is not satisfied

A plain conversion is not enough. A CallerContext can hold more than a table and a column: a record can list a scope (scope(tenant: u64)) or a binding (bind(row: u128)). An EQL payload has nowhere to store those, and does not need to. i holds the declared names only: the table and the column. A scope and a binding are used and not stored, as they are for a field of any other type.

Go records already have EQL fields. A field in a data plan names its EQL type ("target": "TextEq"), and the engine looks the type up by name. So today a Rust program cannot write the same row a Go program writes, except through the dynamic interface. A data plan refuses a scope or a binding on such a field (TargetError::Extended), so a Go struct with an encrypt_into field cannot be sealed through cipher.Scope(..).Bind(..).

2. JsonTreeDocument has only the old traits. JsonTreeDocument (public.eql_v4_json_search) implements only EncryptFrom and DecryptInto (eql-bindings/src/v4/json_tree.rs:507 and :524). Every other EQL v4 type implements the new EncryptTarget and DecryptSource. #1488 removes the old traits, and it cannot do that while JsonTreeDocument depends on them.

Proposal

  1. Let a field type that stores its context store the field's own context: the record's scope and the field's name. A type that cannot store that own context is refused before any key is requested, as a PlanError that names it.
  2. Implement this for the EQL types. An EQL field stores the field's (scope, field) pair as its Identifier, in i. A scope or a binding that the record lists is used and not stored. The terms derive under the column and the scope, and the ciphertext is sealed under the binding too. The reader supplies the scope and the binding again, and a query supplies the scope.
  3. On decrypt, check the field's (scope, field) pair against the i the payload stores, as a standalone decrypt with an expected column does.
  4. Implement EncryptTarget<JsonPlaintext> and DecryptSource<JsonPlaintext> on JsonTreeDocument, with the column as its context, and the same default options as today. It can then be a record field too.
  5. Do the same for a data plan, the path Go takes. A field that names an EQL type takes the scopes and the bindings of the plan and of the call.

The stored bytes do not change. With no scope and no binding, an EQL field in a record writes what the same type writes on its own under Identifier::for_column(scope, field), and what a Go record writes.

Out of scope:

Acceptance

  • The plan's EncryptedMember record compiles, and encrypts and decrypts through the new API.
  • With no scope and no binding, each EQL field writes the same terms as the same type encrypted on its own under Identifier::for_column(scope, field), and each value opens through the other path.
  • A record written through the derive opens through a data plan that declares the same fields with EQL targets (the path Go uses), and the other way round. The same holds under a scope and a binding.
  • A query from a field marker (EncryptedMember::EMAIL.eq(..)) gives the term stored in the row. For a record with a scope, the query takes the scope.
  • A record that lists a scope or a binding, supplied by the caller or stored in the row, encrypts and decrypts with an EQL field. i stores the table and the column alone. The value does not open under another scope or binding, or without them.
  • An EQL field of a record nested in another stores its own column.
  • A Go struct with an encrypt_into field seals, queries and opens through a cipher with a scope and a binding.
  • Decrypting a record whose EQL field stores another column is refused before any key is requested.
  • JsonTreeDocument round-trips through encrypt_into_with_context and JsonPlaintext::decrypt_from_with_context, with the same bytes layout as the old path, and works as a record field.

Relationship to other work

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

SDKenhancementNew feature or requestrustPull requests that update Rust code

Type

No type

Projects

No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions