Skip to content

Crypto-shredding for personal data. The log is append-only, so the personal fields of an event are encrypted with a key that belongs to the person they describe, the subject. Forgetting a subject destroys that key: the events stay where they are and keep their amounts, dates and ids, while their personal fields read as the replacement each field declares.

const AccountEvent = Schema.TaggedUnion({
AccountOpened: {
owner: PersonalData.field(Schema.String, { redacted: "[forgotten]" }),
currency: Schema.Literals(["CHF", "EUR"])
},
TransferSent: {
to: Schema.String,
// Encrypted with the key of the receiver rather than the account holder.
beneficiary: PersonalData.field(Schema.String, { redacted: "[forgotten]", subject: "to" }),
amount: Schema.Number
}
})

A field belongs to the aggregate id of its stream unless it names a sibling property that holds its subject. Schemas stay plain: encryption happens where AggregateService writes and reads the store, which then needs Keys.

Each subject has its own data key. The key store keeps it only wrapped by a MasterKey, which can live in a KMS, so the master key can change without touching events. forget destroys the data key, refuses keys for that subject from then on and records the request in the log.

Encryption is AES-256-GCM through a Cipher: WebCrypto by default, which also runs in browsers, or node:crypto from PersonalDataNode.

Defined in: src/PersonalData.ts:455

The data keys of subjects, created on first use and cached for a while.

new Keys(_): Keys

Defined in: node_modules/.pnpm/effect@4.0.6/node_modules/effect/dist/Context.d.ts:121

never

Keys

Context.Service<Keys, { /** What encrypts and decrypts with the keys. */ readonly cipher: Cipher /** The key to encrypt with, created when the subject has none. Fails for a forgotten subject. */ encryptionKey(subject: string): Effect.Effect<DataKey, SubjectForgotten | PersonalDataError> /** * The key to decrypt with, `None` when the subject was forgotten. A subject * without any key fails: losing the key store must not pass for forgetting. */ decryptionKey(subject: string, keyId: string): Effect.Effect<Option.Option<CipherKey>, PersonalDataError> isForgotten(subject: string): Effect.Effect<boolean, PersonalDataError> /** Destroys the key of a subject. Prefer {@link forget}, which also records it in the log. */ forget(subject: string): Effect.Effect<void, PersonalDataError> /** See {@link rewrap}. */ rewrap(options?: RewrapOptions): Effect.Effect<RewrapResult, PersonalDataError> }>()(“effect-cqrs/PersonalData/Keys”).constructor


Defined in: src/PersonalData.ts:376

Wraps the data keys of subjects, so the key store only ever holds them encrypted. Implement it on top of a KMS to keep the master key out of the process, or use layerMasterKey with a key from configuration.

  • Shape<"effect-cqrs/PersonalData/MasterKey", { keyId: string; unwrap(wrapped, context): Effect<Uint8Array<ArrayBuffer>, PersonalDataError>; wrap(dataKey, context): Effect<Uint8Array<ArrayBuffer>, PersonalDataError>; }, this>

new MasterKey(_): MasterKey

Defined in: node_modules/.pnpm/effect@4.0.6/node_modules/effect/dist/Context.d.ts:121

never

MasterKey

Context.Service<MasterKey, { /** The key new data keys are wrapped with. */ readonly keyId: string wrap( dataKey: Uint8Array<ArrayBuffer>, context: { readonly subject: string; readonly keyId: string } ): Effect.Effect<Uint8Array<ArrayBuffer>, PersonalDataError> /** Unwraps a key wrapped by `masterKeyId`, which may be an earlier master key. */ unwrap( wrapped: Uint8Array<ArrayBuffer>, context: { readonly subject: string; readonly keyId: string; readonly masterKeyId: string } ): Effect.Effect<Uint8Array<ArrayBuffer>, PersonalDataError> }>()("effect-cqrs/PersonalData/MasterKey").constructor


Defined in: src/PersonalData.ts:68

A key that couldn’t be read, created or used, or a field that didn’t decrypt.

  • object & YieldableError<this>

new PersonalDataError(…args): PersonalDataError

Defined in: node_modules/.pnpm/effect@4.0.6/node_modules/effect/dist/Schema.d.ts:10979

…[props: object, options?: MakeOptions]

PersonalDataError

Schema.TaggedError<PersonalDataError>()("PersonalDataError", { operation: Schema.String, subject: Schema.String, message: Schema.String, cause: Schema.optional(Schema.Defect()) }).constructor

optional cause?: unknown

Defined in: src/PersonalData.ts:72

Schema.TaggedError<PersonalDataError>()("PersonalDataError", { operation: Schema.String, subject: Schema.String, message: Schema.String, cause: Schema.optional(Schema.Defect()) }).cause

message: string = Schema.String

Defined in: src/PersonalData.ts:71

Schema.TaggedError<PersonalDataError>()("PersonalDataError", { operation: Schema.String, subject: Schema.String, message: Schema.String, cause: Schema.optional(Schema.Defect()) }).message

readonly operation: string = Schema.String

Defined in: src/PersonalData.ts:69

Schema.TaggedError<PersonalDataError>()("PersonalDataError", { operation: Schema.String, subject: Schema.String, message: Schema.String, cause: Schema.optional(Schema.Defect()) }).operation

readonly subject: string = Schema.String

Defined in: src/PersonalData.ts:70

Schema.TaggedError<PersonalDataError>()("PersonalDataError", { operation: Schema.String, subject: Schema.String, message: Schema.String, cause: Schema.optional(Schema.Defect()) }).subject


Defined in: src/PersonalData.ts:63

  • object & YieldableError<this>

new SubjectForgotten(…args): SubjectForgotten

Defined in: node_modules/.pnpm/effect@4.0.6/node_modules/effect/dist/Schema.d.ts:10979

…[props: object, options?: MakeOptions]

SubjectForgotten

Schema.TaggedError<SubjectForgotten>()("SubjectForgotten", { subject: Schema.String }).constructor

readonly subject: string = Schema.String

Defined in: src/PersonalData.ts:64

Schema.TaggedError<SubjectForgotten>()("SubjectForgotten", { subject: Schema.String }).subject

Defined in: src/PersonalData.ts:402

Where keys are kept. Keep it apart from the events, with backups that expire.

readonly optional afterCommit?: <R>(effect) => Effect<void, never, R>

Defined in: src/PersonalData.ts:420

Runs an effect when the caller’s transaction commits, or at once outside one.

R

Effect<void, never, R>

Effect<void, never, R>

readonly create: (subject, key) => Effect<KeyRecord, PersonalDataError>

Defined in: src/PersonalData.ts:408

Stores a key unless the subject already has a record, and returns the record the subject has afterwards: a racing writer’s key, or Forgotten.

string

"Active"

string

string

Uint8Array<ArrayBuffer>

The data key, wrapped by the master key.

Effect<KeyRecord, PersonalDataError>

readonly forget: (subject, forgottenAt) => Effect<void, PersonalDataError>

Defined in: src/PersonalData.ts:413

Deletes the key and records that the subject is forgotten, in one step.

string

number

Effect<void, PersonalDataError>

readonly get: (subject) => Effect<Option<KeyRecord>, PersonalDataError>

Defined in: src/PersonalData.ts:403

string

Effect<Option<KeyRecord>, PersonalDataError>

readonly optional inTransaction?: Effect<boolean, never, never>

Defined in: src/PersonalData.ts:418

Whether a call now joins a transaction of the caller, whose writes a rollback can still undo. What is read or created there is cached only once it commits.

readonly optional replaceWrapped?: (subject, from, to) => Effect<boolean, PersonalDataError>

Defined in: src/PersonalData.ts:440

Stores a key wrapped anew, if the subject still has the key from: not forgotten, nor wrapped again meanwhile. Returns whether it did.

string

string

string

string

Uint8Array<ArrayBuffer>

Effect<boolean, PersonalDataError>

readonly optional wrappedByOthers?: (masterKeyId, afterSubject, limit) => Effect<readonly object[], PersonalDataError>

Defined in: src/PersonalData.ts:426

Up to limit subjects after afterSubject, in order, whose key is wrapped by another master key than masterKeyId. With Backend.replaceWrapped, what rewrap needs.

string

string

number

Effect<readonly object[], PersonalDataError>


Defined in: src/PersonalData.ts:296

AES-256-GCM, which encrypts personal fields and wraps data keys. What it produces is a random IV of ivLength bytes, the ciphertext and a tag of tagLength bytes, so data written by one implementation reads with any other.

The default, webCrypto, works in Node and in browsers but is asynchronous. PersonalDataNode.layerCipher runs on node:crypto synchronously, which costs a fraction per field.

readonly asynchronous: boolean

Defined in: src/PersonalData.ts:301

Whether an operation waits on something outside the fiber, as WebCrypto does. Events of a stream then decrypt several at a time.

decrypt(key, sealed, additionalData): Effect<Uint8Array<ArrayBuffer>, UnknownError>

Defined in: src/PersonalData.ts:310

Fails when the data was altered, or belongs to another key or other additional data.

CipherKey

Uint8Array<ArrayBuffer>

Uint8Array<ArrayBuffer>

Effect<Uint8Array<ArrayBuffer>, UnknownError>

encrypt(key, plaintext, additionalData): Effect<Uint8Array<ArrayBuffer>, UnknownError>

Defined in: src/PersonalData.ts:304

CipherKey

Uint8Array<ArrayBuffer>

Uint8Array<ArrayBuffer>

Effect<Uint8Array<ArrayBuffer>, UnknownError>

importKey(bytes): Effect<CipherKey, UnknownError>

Defined in: src/PersonalData.ts:303

Imports a key of 32 bytes. The caller zeroes the bytes afterwards.

Uint8Array<ArrayBuffer>

Effect<CipherKey, UnknownError>


Defined in: src/PersonalData.ts:282

A key imported by a Cipher, which only that cipher can use.

readonly [CipherKeyTypeId]: typeof CipherKeyTypeId

Defined in: src/PersonalData.ts:283


Defined in: src/PersonalData.ts:449

readonly key: CipherKey

Defined in: src/PersonalData.ts:451

readonly keyId: string

Defined in: src/PersonalData.ts:450


Defined in: src/PersonalData.ts:113

Where a personal field sits in an encoded payload, and how to find its subject.

readonly path: readonly string[]

Defined in: src/PersonalData.ts:114

readonly redacted: Json

Defined in: src/PersonalData.ts:117

readonly subject: string | undefined

Defined in: src/PersonalData.ts:116

The sibling property holding the subject, or undefined for the aggregate id.


Defined in: src/PersonalData.ts:482

readonly optional cacheSize?: number

Defined in: src/PersonalData.ts:484

How many subjects to keep in memory, with their unwrapped keys. Defaults to 10,000.

readonly optional cacheTtl?: Input

Defined in: src/PersonalData.ts:490

How long what the store said about a subject is used before it is asked again. It bounds how long another process keeps using the key of a subject forgotten here. Defaults to one minute.


Defined in: src/PersonalData.ts:1087

Where a payload is stored, which the ciphertext of its fields is bound to.

readonly aggregateId: string

Defined in: src/PersonalData.ts:1089

readonly aggregateName: string

Defined in: src/PersonalData.ts:1088

readonly type: string

Defined in: src/PersonalData.ts:1090


Defined in: src/PersonalData.ts:472

readonly optional batchSize?: number

Defined in: src/PersonalData.ts:474

How many keys are read from the store at a time. Defaults to 100.


Defined in: src/PersonalData.ts:477

readonly rewrapped: number

Defined in: src/PersonalData.ts:479

The keys now wrapped by the current master key.


Defined in: src/PersonalData.ts:864

readonly optional migrate?: boolean

Defined in: src/PersonalData.ts:871

Creates the table on startup when it doesn’t exist. Defaults to true. Turn it off to run migrations with your own tooling instead.

readonly optional table?: string

Defined in: src/PersonalData.ts:866

Defaults to cqrs_personal_data_keys.


Defined in: src/PersonalData.ts:1174

A payload, with the subjects whose personal data it holds.

readonly payload: Json

Defined in: src/PersonalData.ts:1175

readonly subjects: readonly string[]

Defined in: src/PersonalData.ts:1176

KeyRecord = { _tag: "Active"; keyId: string; masterKeyId: string; wrapped: Uint8Array<ArrayBuffer>; } | { _tag: "Forgotten"; forgottenAt: number; }

Defined in: src/PersonalData.ts:391

What the key store holds for a subject.

{ _tag: "Active"; keyId: string; masterKeyId: string; wrapped: Uint8Array<ArrayBuffer>; }

readonly _tag: "Active"

readonly keyId: string

readonly masterKeyId: string

readonly wrapped: Uint8Array<ArrayBuffer>

The data key, wrapped by the master key.


{ _tag: "Forgotten"; forgottenAt: number; }


Sealed = typeof Sealed.Type

Defined in: src/PersonalData.ts:1247

Text encrypted once with the key of each subject, so destroying any one of the keys makes it unreadable.

Cipher: Reference<Cipher>

Defined in: src/PersonalData.ts:296

The cipher that keys, master keys and fields use, webCrypto unless another is provided.


const forget: (…args) => Effect<void, EventStoreError | PersonalDataError, Keys | EventStore>

Defined in: src/PersonalData.ts:1045

Forgets a subject: destroys its key, so its personal fields read as their replacements everywhere, and records a SubjectForgotten event in the stream of the subject, with the current causation and metadata as the audit trail. Running it again records nothing new.

The key goes first, so that no event says a subject is forgotten while its key still exists. A store that can’t be reached is tried a few more times; when it still fails, the subject is forgotten but the event is missing, and onForget handlers haven’t run. Call forget again until it succeeds.

Other processes stop using the key once their cache expires, see KeysOptions.cacheTtl.

…[subject: string]

Effect<void, EventStoreError | PersonalDataError, Keys | EventStore>


const ivLength: 12 = 12

Defined in: src/PersonalData.ts:317


const keyLength: 32 = 32

Defined in: src/PersonalData.ts:319


const layerMasterKeyEphemeral: Layer.Layer<MasterKey>

Defined in: src/PersonalData.ts:642

A random master key that lasts as long as the layer. Keys it wrapped can’t be read by another process or after a restart, so it suits tests and demos.


const make: (…args) => Effect<{ cipher: Cipher; decryptionKey(subject, keyId): Effect<Option<CipherKey>, PersonalDataError>; encryptionKey(subject): Effect<DataKey, PersonalDataError | SubjectForgotten>; forget(subject): Effect<void, PersonalDataError>; isForgotten(subject): Effect<boolean, PersonalDataError>; rewrap(options?): Effect<RewrapResult, PersonalDataError>; }, never, MasterKey>

Defined in: src/PersonalData.ts:664

Builds Keys on a key store. Reads of the store are cached per subject, and concurrent misses for one subject share a single read, unwrap and, for a new subject, a single key creation.

…[backend: Backend, options?: KeysOptions]

Effect<{ cipher: Cipher; decryptionKey(subject, keyId): Effect<Option<CipherKey>, PersonalDataError>; encryptionKey(subject): Effect<DataKey, PersonalDataError | SubjectForgotten>; forget(subject): Effect<void, PersonalDataError>; isForgotten(subject): Effect<boolean, PersonalDataError>; rewrap(options?): Effect<RewrapResult, PersonalDataError>; }, never, MasterKey>


const makeMasterKey: (…args) => Effect<{ keyId: string; unwrap(wrapped, context): Effect<Uint8Array<ArrayBuffer>, PersonalDataError>; wrap(dataKey, context): Effect<Uint8Array<ArrayBuffer>, PersonalDataError>; }, PersonalDataError, never>

Defined in: src/PersonalData.ts:543

A master key held by the process, given as 32 bytes. Keys wrapped by earlier master keys stay readable while those are listed in previous. The bytes are copied, so the caller can zero its own.

…[options: object]

Effect<{ keyId: string; unwrap(wrapped, context): Effect<Uint8Array<ArrayBuffer>, PersonalDataError>; wrap(dataKey, context): Effect<Uint8Array<ArrayBuffer>, PersonalDataError>; }, PersonalDataError, never>


const makeSqlBackend: (…args) => Effect<{ afterCommit: <R>(effect) => Effect<void, never, R>; create: (subject, key) => Effect<KeyRecord, PersonalDataError, never>; forget: (subject, forgottenAt) => Effect<void, PersonalDataError, never>; get: (subject) => Effect<Option<KeyRecord>, PersonalDataError, never>; inTransaction: Effect<boolean, never, never>; replaceWrapped: (subject, from, to) => Effect<boolean, PersonalDataError, never>; wrappedByOthers: (masterKeyId, afterSubject, limit) => Effect<object[], PersonalDataError, never>; }, PersonalDataError, SqlClient>

Defined in: src/PersonalData.ts:918

A key store in a SQL table, on SQLite or PostgreSQL. The table is created on startup unless migrate is false. Provide it a client of its own to keep keys in another database than the events, with other permissions and backups.

Forgetting overwrites the row. The database may keep the old bytes for a while, in free pages, dead rows or its write-ahead log, and backups keep them as long as they are kept: let backups of this table expire.

…[options?: SqlOptions]

Effect<{ afterCommit: <R>(effect) => Effect<void, never, R>; create: (subject, key) => Effect<KeyRecord, PersonalDataError, never>; forget: (subject, forgottenAt) => Effect<void, PersonalDataError, never>; get: (subject) => Effect<Option<KeyRecord>, PersonalDataError, never>; inTransaction: Effect<boolean, never, never>; replaceWrapped: (subject, from, to) => Effect<boolean, PersonalDataError, never>; wrappedByOthers: (masterKeyId, afterSubject, limit) => Effect<object[], PersonalDataError, never>; }, PersonalDataError, SqlClient>


const openText: (…args) => Effect<Option<string>, PersonalDataError, never>

Defined in: src/PersonalData.ts:1271

Opens text from sealText, None when one of its subjects was forgotten.

…[keys: object, sealed: object, purpose: string]

Effect<Option<string>, PersonalDataError, never>


const Sealed: Struct<{ data: String; subjects: $Array<Struct<{ keyId: String; subject: String; }>>; }>

Defined in: src/PersonalData.ts:1247

Text encrypted once with the key of each subject, so destroying any one of the keys makes it unreadable.


const sealText: (…args) => Effect<{ data: string; subjects: object[]; }, PersonalDataError | SubjectForgotten, never>

Defined in: src/PersonalData.ts:1254

Seals text such as a snapshot of a state that holds the personal data of subjects.

…[keys: object, subjects: readonly string[], text: string, purpose: string]

Effect<{ data: string; subjects: object[]; }, PersonalDataError | SubjectForgotten, never>


const streamName: "effect-cqrs/personal-data" = "effect-cqrs/personal-data"

Defined in: src/PersonalData.ts:1026

The stream that records forgotten subjects, one stream per subject.


const subjectForgottenType: "SubjectForgotten" = "SubjectForgotten"

Defined in: src/PersonalData.ts:1029

The type of the event forget records. Its payload is empty.


const tagLength: 16 = 16

Defined in: src/PersonalData.ts:318


const webCrypto: Cipher

Defined in: src/PersonalData.ts:329

AES-256-GCM on WebCrypto.

field<S>(schema, options): S

Defined in: src/PersonalData.ts:95

Marks a field as personal data. redacted is what the field reads as once its subject is forgotten, and has to be a valid value of the field. subject names a property next to the field that holds the id of the person it describes, by default the aggregate id of the stream.

S extends PlainSchema

S

S["Type"]

string

S


fieldsByTag(schema, where): ReadonlyMap<string, readonly Field[]>

Defined in: src/PersonalData.ts:259

The personal fields of each event in a union of tagged events. Events without any are left out.

Top

string

ReadonlyMap<string, readonly Field[]>


fieldsOf(schema, where): readonly Field[]

Defined in: src/PersonalData.ts:252

The personal fields of one schema, such as an old version of an event.

Top

string

readonly Field[]


layerMasterKey(options): Layer<MasterKey, PersonalDataError>

Defined in: src/PersonalData.ts:587

Redacted<Uint8Array<ArrayBufferLike>>

string

readonly object[]

Layer<MasterKey, PersonalDataError>


layerMasterKeyConfig(options?): Layer<MasterKey, PersonalDataError | ConfigError>

Defined in: src/PersonalData.ts:600

A master key from configuration: an id, and 32 bytes in base64. By default PERSONAL_DATA_MASTER_KEY_ID and PERSONAL_DATA_MASTER_KEY.

Earlier master keys, which keys created before a rotation are still wrapped by, come from PERSONAL_DATA_MASTER_KEY_PREVIOUS by default, as id:base64 pairs separated by commas. Leave it unset when there are none, and empty it once rewrap has moved every key to the current master key.

Config<Redacted<string>>

Config<string>

Config<Redacted<string>>

Layer<MasterKey, PersonalDataError | ConfigError>


layerMemory(options?): Layer<Keys, never, MasterKey>

Defined in: src/PersonalData.ts:861

Keys kept in process memory, for tests and demos.

KeysOptions

Layer<Keys, never, MasterKey>


layerSql(options?): Layer<Keys, PersonalDataError, SqlClient | MasterKey>

Defined in: src/PersonalData.ts:1007

Keys in a SQL table, see makeSqlBackend.

SqlOptions & KeysOptions

Layer<Keys, PersonalDataError, SqlClient | MasterKey>


makeMemoryBackend(): Backend

Defined in: src/PersonalData.ts:823

A key store in process memory, gone after a restart along with access to everything it encrypted.

Backend


migrations(options?): readonly Migration[]

Defined in: src/PersonalData.ts:883

The migrations of the key table of makeSqlBackend, the same on SQLite and PostgreSQL, for teams that run migrations themselves. They are safe to run again.

string

readonly Migration[]


open(keys, fields, location, payload): Effect<WithSubjects, PersonalDataError>

Defined in: src/PersonalData.ts:1186

Decrypts the personal fields of a stored payload, putting the replacement in place of the fields of forgotten subjects. A field stored in plaintext, from before it was marked personal, is left as it is. A field has to be encrypted for the subject it would be sealed for now: the aggregate id, or the sibling it names.

Cipher

What encrypts and decrypts with the keys.

(subject, keyId) => Effect<Option<CipherKey>, PersonalDataError>

The key to decrypt with, None when the subject was forgotten. A subject without any key fails: losing the key store must not pass for forgetting.

(subject) => Effect<DataKey, PersonalDataError | SubjectForgotten>

The key to encrypt with, created when the subject has none. Fails for a forgotten subject.

(subject) => Effect<void, PersonalDataError>

Destroys the key of a subject. Prefer forget, which also records it in the log.

(subject) => Effect<boolean, PersonalDataError>

(options?) => Effect<RewrapResult, PersonalDataError>

See rewrap.

readonly Field[]

Location

Json

Effect<WithSubjects, PersonalDataError>


rewrap(options?): Effect<RewrapResult, PersonalDataError, Keys>

Defined in: src/PersonalData.ts:1022

Wraps every subject key that an earlier master key wrapped with the current one, so the earlier one can be dropped from previous. Keys stay the same, so nothing encrypted with them changes, and running it again only rewraps what is left. Keys created meanwhile are wrapped by the current master key already.

RewrapOptions

Effect<RewrapResult, PersonalDataError, Keys>


seal(keys, fields, location, payload): Effect<WithSubjects, PersonalDataError | SubjectForgotten>

Defined in: src/PersonalData.ts:1127

Encrypts the personal fields of an encoded payload. Fields without a value are left alone. aggregateId is the subject of fields that don’t name one. Returns the subjects whose keys were used.

Cipher

What encrypts and decrypts with the keys.

(subject, keyId) => Effect<Option<CipherKey>, PersonalDataError>

The key to decrypt with, None when the subject was forgotten. A subject without any key fails: losing the key store must not pass for forgetting.

(subject) => Effect<DataKey, PersonalDataError | SubjectForgotten>

The key to encrypt with, created when the subject has none. Fails for a forgotten subject.

(subject) => Effect<void, PersonalDataError>

Destroys the key of a subject. Prefer forget, which also records it in the log.

(subject) => Effect<boolean, PersonalDataError>

(options?) => Effect<RewrapResult, PersonalDataError>

See rewrap.

readonly Field[]

Omit<Location, "aggregateId"> & object

Json

Effect<WithSubjects, PersonalDataError | SubjectForgotten>