19.13. Transparent Data Encryption #

19.13.1. Overview
19.13.2. Key Management
19.13.3. Key Rotation
19.13.4. Cryptography
19.13.5. Operational Notes

Transparent data encryption (TDE) is an optional feature that encrypts, transparently to the user, any user data stored in the database system. User data includes the actual data stored in tables and other objects, as well as system catalog data such as the names of objects.

19.13.1. Overview #

Specifically, TDE encrypts:

  • The files underlying tables, sequences, indexes, including TOAST tables and system catalogs, and including all forks. (We shall call these the data files below.)

  • The write-ahead log (WAL).

  • Various temporary files that are used during query processing and the operation of the database system.

The following are not encrypted or otherwise disguised by TDE:

  • Metadata internal to the operation of the database system that does not contain user data, such as the transaction status (e.g., pg_subtrans, pg_xact).

  • The file names and file system structure in the data directory. (That means, the overall size of the database system, the number of databases, the number of tables, their relative sizes, as well as file-system metadata such as last access time are all visible without decryption.)

  • Data in foreign tables.

  • The server diagnostics log.

  • Configuration files.

Transparent data encryption can be enabled when a database cluster is initialized using initdb; see there for details.

19.13.2. Key Management #

The key for transparent data encryption (the data key) is normally generated by initdb and stored in a file pg_encryption/key.bin under the data directory. (This file actually contains several keys that are used for different purposes at run time; see below for details. But for the purpose of this section, it contains a single sequence of random bytes.)

If nothing else were done, this file would contain the key in plaintext. This would be insecure because someone who got ahold of the encrypted data directory would have access to the plaintext key, thus defeating the purpose of encryption. Therefore, this setup is only suitable for testing purposes.

To secure the data key properly, it should be wrapped by encrypting it with another key. There are broadly two possibilites to arrange this:

  • The data key is protected by a passphrase. This means that a wrapping key is derived from the passphrase and it used to encrypt the data key.

  • The wrapping key is stored elsewhere, for example in a key management system. (This second key is also called the key-wrapping key or master key.)

PostgreSQL leaves the configuration of this up to the user, which allows tailoring the setup to local requirements and integrating with existing key management software or similar. To configure the data key protection, a pair of external commands needs to be specified that take care of the wrapping (encrypting) and unwrapping (decryption) respectively.

For example, suppose you want to protect the data key by a passphrase. This can be done using the openssl command-line utility. The following is an example invocation that sets this up:

initdb -D datadir -y --key-wrap-command='openssl enc -e -aes-128-cbc -pbkdf2 -out %p' --key-unwrap-command='openssl enc -d -aes-128-cbc -pbkdf2 -in %p'

This will wrap the randomly generated data key (done internally by initdb) by encrypting it using the AES-128-CBC algorithm. The encryption uses a key derived from a passphrase using the PBKDF2 key derivation function and a randomly generated salt. The passphrase will be prompted for on the terminal. (See the openssl-enc manual page for details of these options. Available options vary across versions.) The placeholder %p is replaced by the name of the file where the wrapped key should be stored. The unwrap command performs the opposite operation. initdb does not itself need the unwrap operation but it will store it in the postgresql.conf of the initialized cluster, which will use it when it starts up.

The key wrap command will receive the plaintext key on standard input and needs to put the wrapped key at the file system location specified by the %p placeholder. The key unwrap command needs to read the wrapped key from the file system location specified by the %p placeholder and write the unwrapped key to the standard output.

Some utility programs operate directly on the data directory (or copies thereof, such as backups), such as pg_rewind and pg_upgrade. These programs will, depending on the circumstances, also need to be told about the key unwrap command. They each have command-line options for this purpose.

To simplify operations, the key wrap and unwrap commands can also be set in environment variables. These are accepted by all affected applications if no corresponding command-line options have been given. Example:

PGDATAKEYWRAPCMD='openssl enc -e -aes-128-cbc -pbkdf2 -out %p'
PGDATAKEYUNWRAPCMD='openssl enc -d -aes-128-cbc -pbkdf2 -in %p'
export PGDATAKEYWRAPCMD PGDATAKEYUNWRAPCMD

Key unwrap commands that prompt for passwords on the terminal will not work when the server is started by pg_ctl or through service managers such as systemd, because the server will be detached from the terminal in those environments. If an interactive password prompt on server start is desired, a more elaborate configuration will be required that fetches the password via some indirect mechanism. For systemd, the program systemd-ask-password is provided for this purpose. The following setup can be used:

PGDATAKEYWRAPCMD="bash -c 'openssl enc -e -aes-128-cbc -pbkdf2 -out %p -pass file:<(sudo systemd-ask-password --no-tty)'"
PGDATAKEYUNWRAPCMD="bash -c 'openssl enc -d -aes-128-cbc -pbkdf2 -in %p -pass file:<(sudo systemd-ask-password --no-tty)'"

Also, an entry like the following is required in /etc/sudoers:

postgres ALL = NOPASSWD: /usr/bin/systemd-ask-password

External key management systems can be used by the same wrap/unwrap mechanisms. Here are some examples for some well-known services (leaving out details such as how to set up credentials).

AWS Key Management Service: 

aws kms create-key
aws kms create-alias --alias-name alias/pg-tde-master-1 --target-key-id "..."

PGDATAKEYWRAPCMD='aws kms encrypt --key-id alias/pg-tde-master-1 --plaintext fileb:///dev/stdin --output text --query CiphertextBlob | base64 -d > %p'
PGDATAKEYUNWRAPCMD='aws kms decrypt --ciphertext-blob fileb://%p --output text --query Plaintext | base64 -d'

or using https://github.com/VirtusLab/crypt:

PGDATAKEYWRAPCMD='crypt encrypt aws --out %p --region us-east-1 --kms alias/pg-tde-master-1'
PGDATAKEYUNWRAPCMD='crypt decrypt aws --in %p --region us-east-1'

Azure Key Vault: 

az keyvault key create --vault-name pg-tde --name pg-tde-master-1

PGDATAKEYWRAPCMD='crypt encrypt azure --vaultURL https://pg-tde.vault.azure.net --name pg-tde-master-1 --version fa2bf368449e432085318c5bf666754c --out %p'
PGDATAKEYUNWRAPCMD='crypt decrypt azure --vaultURL https://pg-tde.vault.azure.net --name pg-tde-master-1 --version fa2bf368449e432085318c5bf666754c --in %p'

This uses https://github.com/VirtusLab/crypt. The Azure CLI cannot be used directly for this, because it lacks some functionality.

Google Cloud KMS: 

gcloud kms keys create pg-tde-master-1 --location=global --keyring=pg-tde --purpose=encryption

PGDATAKEYWRAPCMD='gcloud kms encrypt --plaintext-file=- --ciphertext-file=%p --location=global --keyring=pg-tde --key=pg-tde-master-1'
PGDATAKEYUNWRAPCMD='gcloud kms decrypt --plaintext-file=- --ciphertext-file=%p --location=global --keyring=pg-tde --key=pg-tde-master-1'

or using https://github.com/VirtusLab/crypt:

PGDATAKEYWRAPCMD='crypt encrypt gcp --out=%p --location=global --keyring=pg-tde --key=pg-tde-master-1 --project your-project-123456'
PGDATAKEYUNWRAPCMD='crypt decrypt gcp --in=%p --location=global --keyring=pg-tde --key=pg-tde-master-1 --project your-project-123456'

HashiCorp Vault: 

# enable once
vault secrets enable transit

# create a key (pick a name)
vault write -f transit/keys/pg-tde-master-1

PGDATAKEYWRAPCMD='base64 | vault write -field=ciphertext transit/encrypt/pg-tde-master-1 plaintext=- > %p'
PGDATAKEYUNWRAPCMD='vault write -field=plaintext transit/decrypt/pg-tde-master-1 ciphertext=- < %p | base64 -d'

Note

Shell commands with pipes, as in some of these examples, are problematic because the exit status of the pipe is that of the last command, and so a failure of the first (more interesting) command would not be reported properly. PostgreSQL will handle this somewhat by recognizing whether the wrap or unwrap command wrote nothing. It is better to make this more robust, however, for example by using the pipefail option available in some shells or the mispipe command available on some operating systems. More complicated commands should probably be put into an external shell script or other program instead of being defined inline.

If no key wrapping is wanted (for testing), then the wrap and unwrap commands must be set to the special value -. This specifies to use the key from the file without further processing. This is distinct from not setting a wrap or unwrap command at all. Having no wrap or unwrap command set when transparent data encryption is used will result in a fatal error when running an affected utility program.

19.13.3. Key Rotation #

To change the master key, manually run the unwrap command specifying the old key and feed the result into the wrap command specifying the new key. Equivalently, if the data key is protected by a passphrase, to change the passphrase, run the unwrap command using the old passphrase and feed the result into the wrap command using the new passphrase. The operations can be performed while the database server is running. The wrapped data key in the file is only used on startup and it not used while the server is running.

For example, building on the above passphrase example using openssl, the following could be used to change the passphrase:

cd $PGDATA/pg_encryption/
openssl enc -d -aes-128-cbc -pbkdf2 -in key.bin | openssl enc -e -aes-128-cbc -pbkdf2 -out key.bin.new
mv key.bin.new key.bin

With this particular method, the decryption and the encryption commands ask for the passphrase on the terminal at the same time, which is awkward and confusing. The following is an alternative:

cd $PGDATA/pg_encryption/
openssl enc -d -aes-128-cbc -pbkdf2 -in key.bin -pass pass:ACTUALPASSPHRASE | openssl enc -e -aes-128-cbc -pbkdf2 -out key.bin.new
mv key.bin.new key.bin

This will leak the old passphrase, but it is going to be replaced anyway. The openssl supports a number of other ways to supply the passphrases.

When using a key management system, the unwrap and wrap commands can be connected similarly, for example:

cd $PGDATA/pg_encryption/
crypt decrypt aws --in key.bin --region us-east-1 | crypt encrypt aws --out key.bin.new --region us-east-1 --kms alias/pg-tde-master-2
mv key.bin.new key.bin

Note that the data key (the key wrapped by the master key) cannot be changed on an existing data directory. If that is required, the data directory would need to be run through an upgrade process using pg_dump, pg_upgrade, or logical replication.

19.13.4. Cryptography #

This section contains details about the cryptographic methods used by the transparent data encryption feature.

The data files are encrypted using AES-128-XTS. The XTS tweak uses the database OID, the relfilenode, and the block number.

The WAL is encrypted using AES-128-CTR. The counter includes the WAL LSN.

Temporary files that are accessed by block are also encrypted using AES-128-XTS. Other temporary files are encrypted using AES-128-CBC.

19.13.5. Operational Notes #

A physical replica is necessarily encrypted (or not encrypted) in the same way and using the same keys as its primary server. If a server uses TDE, a base backup is automatically encrypted. Any WAL fetched from a server using TDE, including streaming replication and archiving, is encrypted.

Logical replication is not affected by TDE. Publisher and subscriber can have different encryption settings. The payload of the logical replication protocol is not encrypted. (Of course, SSL can be used.)

When making a copy of an encrypted database instance for independent use, a new WAL key should be generated for the instance. The command pg_resetwal --new-wal-key can be used for this. Otherwise, the WAL streams of the instances will be vulnerable to a reused key attack. This issue happens when the same WAL stream is continued in more than one independent instance using the same key. Examples of scenarios affected by this are promoting a standby while continuing to use the primary, or promoting more than one standby. (In those cases, more than one independent instance continues writing the same WAL stream with different data.) Merely having (unpromoted) standby servers is not an issue. (In that case, all instances share the same WAL stream.) Also, it is not an issue if a standby is promoted and the former primary is not used as a primary anymore. (In that case, only one WAL stream using a particular key is continued.)