On Tue, Oct 27, 2020 at 10:20:35AM -0400, Bruce Momjian wrote: > I don't know much about how to hook into that stuff so if you have an > idea, I am all ears. I have used OpenSSL with Yubikey via pksc11. You > can see the use of it on slide 57 and following: > > https://momjian.us/main/writings/crypto_hw_config.pdf#page=57 > > Interestingly, that still needed the user to type in a key to unlock the > Yubikey, so we might need PKCS11 and a password for the same server > start. > > I would like to get this moving forward so I will work on the idea of > passing an open /dev/tty file descriptor from pg_ctl to the postmaster > on start.
Here is an updated patch that uses an argument to pass an open /dev/tty
file descriptor to the postmaster. It uses -R for initdb/pg_ctl, -R ###
for postmaster/postgres, and %R for cluster_passphrase_command. Here is
a sample session:
--> $ initdb -R --cluster-passphrase-command '/tmp/pass_fd.sh "%p" %R'
The files belonging to this database system will be owned by user
"postgres".
This user must also own the server process.
The database cluster will be initialized with locale "en_US.UTF-8".
The default database encoding has accordingly been set to "UTF8".
The default text search configuration will be set to "english".
Data page checksums are disabled.
Key management system is enabled.
fixing permissions on existing directory /u/pgsql/data ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... America/New_York
creating configuration files ... ok
running bootstrap script ...
--> Enter database encryption pass phrase:
B1D7B405EDCD97B7351DD3B7AE0637775FFBC6A2C2EEADAEC009A75A58A79F50
ok
performing post-bootstrap initialization ...
--> Enter database encryption pass phrase:
B1D7B405EDCD97B7351DD3B7AE0637775FFBC6A2C2EEADAEC009A75A58A79F50
ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
pg_ctl -D /u/pgsql/data -l logfile start
$ pg_ctl stop
pg_ctl: PID file "/u/pgsql/data/postmaster.pid" does not exist
Is server running?
--> $ pg_ctl -l /u/pg/server.log -R start
waiting for server to start...
--> Enter database encryption pass phrase:
B1D7B405EDCD97B7351DD3B7AE0637775FFBC6A2C2EEADAEC009A75A58A79F50
done
server started
Attached is my updated patch, based on Masahiko Sawada's patch, and my
pass_fd.sh script.
--
Bruce Momjian <br...@momjian.us> https://momjian.us
EnterpriseDB https://enterprisedb.com
The usefulness of a cup is in its emptiness, Bruce Lee
diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
new file mode 100644
index f043433..65422d5
*** a/doc/src/sgml/config.sgml
--- b/doc/src/sgml/config.sgml
*************** COPY postgres_log FROM '/full/path/to/lo
*** 7793,7798 ****
--- 7793,7836 ----
</variablelist>
</sect1>
+ <sect1 id="runtime-config-encryption">
+ <title>Encryption Key Management</title>
+
+ <variablelist>
+ <varlistentry id="guc-cluster-passphrase-command" xreflabel="cluster_passphrase_command">
+ <term><varname>cluster_passphrase_command</varname> (<type>string</type>)
+ <indexterm>
+ <primary><varname>cluster_passphrase_command</varname> configuration parameter</primary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ This option specifies an external command to be invoked when a
+ passphrase for key management system needs to be obtained.
+ </para>
+ <para>
+ The command must print the passphrase to the standard
+ output and have a zero exit code. In the parameter value,
+ <literal>%p</literal> is replaced by a prompt string, and
+ <literal>%R</literal> represents the file descriptor for
+ reading/writing from the terminal; if specified at server start,
+ it is blank. Since it can be blank, it is wise for it to be the
+ last command-line argument specified. (Write <literal>%%</literal>
+ for a literal <literal>%</literal>.) Note that the prompt string
+ will probably contain whitespace, so be sure to quote its use
+ adequately. A single newline is stripped from the end of the
+ output if present. The passphrase must be at least 64 bytes.
+ </para>
+ <para>
+ This parameter can only be set in the
+ <filename>postgresql.conf</filename> file or on the server
+ command line.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </sect1>
+
<sect1 id="runtime-config-client">
<title>Client Connection Defaults</title>
*************** dynamic_library_path = 'C:\tools\postgre
*** 9636,9641 ****
--- 9674,9695 ----
</para>
</listitem>
</varlistentry>
+
+ <varlistentry id="guc-key-management-enabled" xreflabel="key_management_enabled">
+ <term><varname>key_management_enabled</varname> (<type>boolean</type>)
+ <indexterm>
+ <primary>Key management configuration parameter parameter</primary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ Reports whether encryption key management
+ is enabled for this cluster. See <xref
+ linkend="app-initdb-cluster-passphrase-command"/> for more
+ information.
+ </para>
+ </listitem>
+ </varlistentry>
<varlistentry id="guc-data-directory-mode" xreflabel="data_directory_mode">
<term><varname>data_directory_mode</varname> (<type>integer</type>)
diff --git a/doc/src/sgml/database-encryption.sgml b/doc/src/sgml/database-encryption.sgml
new file mode 100644
index ...db84b0d
*** a/doc/src/sgml/database-encryption.sgml
--- b/doc/src/sgml/database-encryption.sgml
***************
*** 0 ****
--- 1,292 ----
+ <!-- doc/src/sgml/database-encryption.sgml -->
+
+ <chapter id="database-encryption">
+ <title>Database Encryption</title>
+
+ <indexterm zone="database-encryption">
+ <primary>Server Side Encryption</primary>
+ </indexterm>
+
+ <para>
+ The purpose of database encryption is to protect the confidential data
+ stored in a database from being revealed.
+ </para>
+
+ <sect1 id="encryption-key-management">
+ <title>Encryption Key Management</title>
+
+ <para>
+ <productname>PostgreSQL</productname> supports internal
+ <firstterm>Encryption Key Management System</firstterm>, which is designed
+ to manage the life cycles of cryptographic keys within the
+ <productname>PostgreSQL</productname>. This includes dealing with their
+ generation, storage, usage and rotation.
+ </para>
+
+ <para>
+ Encryption key management system is enabled when
+ <productname>PostgreSQL</productname> is built with
+ <literal>--with-openssl</literal> and
+ <xref linkend="app-initdb-cluster-passphrase-command"/> is specified during
+ <command>initdb</command>. The cluster passphrase provided by the
+ <option>--cluster-passphrase-command</option> option during
+ <command>initdb</command> and the one generated by
+ <xref linkend="guc-cluster-passphrase-command"/> in the
+ <filename>postgresql.conf</filename> must match, otherwise, the database
+ cluster will not start up. Note that the cluster passphrase command
+ passed to <command>initdb</command> must return a passphrase of at least
+ 64 bytes and less than 1024 bytes. For example.
+ <programlisting>
+ initdb -D dbname --cluster-passphrase-command="cat /path/to/passphrase-file"
+ </programlisting>
+ </para>
+
+ <para>
+ Encryption keys managed by <productname>PostgreSQL</productname>'s
+ key management system are only used for the internal usage such as
+ transparent data encryption in a future release. These encryption keys
+ can never be taken out of database in plaintext form. Also, the
+ encryption key management system creates some internal encryption keys.
+ There is no interface to add and remove these keys.
+ </para>
+
+ <sect2 id="key-encryption-key">
+ <title>Key Encryption Key(<acronym>KEK</acronym>)</title>
+
+ <para>
+ During the <command>initdb</command> process, the cluster passphrase
+ provided by <option>--cluster-passphrase-command</option>
+ is derived into a <firstterm>Key Encryption Key
+ (<acronym>KEK</acronym>)</firstterm>. <acronym>KEK</acronym>
+ encapsulates cryptographic keys managed inside
+ <productname>PostgreSQL</productname> described in <xref
+ linkend="key-derivations"/> using by a way of authenticated encryption
+ described in <xref linkend="key-wrapping"/> before storing the keys
+ to a persistent storage. <acronym>KEK</acronym> must be stored in
+ a trusted key store, such as key vault software or services, or a
+ hardware security module.
+ </para>
+
+ <para>
+ When a <productname>PostgreSQL</productname> server
+ with enabled encryption key management is started, the
+ <varname>cluster_passphrase_command</varname> parameter in
+ <filename>postgresql.conf</filename> will be evaluated and the cluster
+ passphrase will be derived into <acronym>KEK</acronym> in similar
+ ways as initdb.
+ </para>
+
+ <para>
+ After that, the cryptographic keys will be retrieved from
+ <filename>pg_cryptokeys</filename> directory to be restored and
+ integrity-checked by the key management system using the <acronym>KEK</acronym>.
+ If this process fails, it is likely that the cluster passphrase supplied
+ to the cluster is not the same as that supplied to the
+ <command>initdb</command> process. The cluster will refuse to start in this
+ case and user has to manually correct the cluster passphrase.
+ </para>
+
+ <para>
+ <acronym>KEK</acronym> is not stored physically within the
+ <productname>PostgreSQL</productname> server as they are designed
+ to be derived from the correctly configured cluster passphrase.
+ </para>
+ </sect2>
+
+ <sect2 id="key-derivations">
+ <title>Key Derivations</title>
+
+ <para>
+ Encryption key management systems can manage multiple
+ cryptographic keys that have different purposes and usages within
+ <productname>PostgreSQL</productname>. Currently, the Postgres
+ encryption key management system manages no cryptographic keys.
+ </para>
+ </sect2>
+
+ <sect2 id="key-wrapping">
+ <title>Key Protection</title>
+
+ <para>
+ Key management system persists cryptographic keys to the disk after wrapping
+ them by <acronym>KEK</acronym>. This section describes how key maangement
+ system wrap and unwrap key.
+ </para>
+
+ <para>
+ The key management system uses Encryption with Associated Data
+ (<acronym>AEAD</acronym>) to wrap cryptographic keys, which is
+ a form of encryption. In addition to providing a way to protect
+ confidential data from being revealed, it provides a way to check
+ the integrity and authenticity of some associated data. It uses
+ the Encrypt-Then-MAC approach, based on the Advanced Encryption
+ Standard (<acronym>AES</acronym>) in Cipher Block Chaining
+ (<acronym>CBC</acronym>) mode. It uses a random initialization
+ vector (<acronym>IV</acronym>) and a <literal>HMAC-SHA</literal>
+ message authentication code (<acronym>MAC</acronym>).
+ </para>
+
+ <para>
+ Key management system uses two kinds of cryptographic keys for key wrapping:
+ </para>
+
+ <para>
+ <variablelist>
+ <varlistentry>
+ <term><literal>Encryption Key</literal></term>
+ <listitem>
+ <para>
+ Encryption key is 256 bits long randomly generate key. It is primarily used
+ as a key for encapsulate or restore data with <acronym>AES256</acronym>
+ algorithm.
+ </para>
+ </listitem>
+ </varlistentry>
+ <varlistentry>
+ <term><literal>MAC Key</literal></term>
+ <listitem>
+ <para>
+ <acronym>MAC</acronym> key is a 512-bit randomly generated key.
+ <acronym>SHA512</acronym> is the algorithm used along with the
+ <acronym>MAC</acronym> key to compute a cryptographic hash for integrity
+ purposes.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </para>
+
+ <para>
+ Key management systems's key wrapping algorithm is as follows:
+
+ <orderedlist>
+ <listitem>
+ <simpara>Generate random <acronym>IV</acronym>.</simpara>
+ </listitem>
+ <listitem>
+ <simpara>Add padding to the plaintext following PKCS#7 described in
+ <ulink url="https://tools.ietf.org/html/rfc2315";>RFC2315</ulink>.</simpara>
+ </listitem>
+ <listitem>
+ <simpara>Encrypt padded plain text with the <acronym>IV</acronym>
+ using <acronym>AES256</acronym> in <acronym>CBC</acronym>
+ mode.</simpara>
+ </listitem>
+ <listitem>
+ <simpara>Compute <acronym>HMAC</acronym> over the encrypted data.</simpara>
+ </listitem>
+ <listitem>
+ <simpara>Concatenate <acronym>HMAC</acronym>, <acronym>IV</acronym>
+ and encrypted ciphertext as the result of ciphertext.</simpara>
+ </listitem>
+ </orderedlist>
+ </para>
+
+ <para>
+ The length of the result ciphertext can be inferred from that of the plaintext
+ by following formula:
+ <programlisting>
+ Ciphertext Length = 64 + 16 + 16 * (floor(input_size / 16) + 1)
+ </programlisting>
+ </para>
+ </sect2>
+
+ <sect2 id="key-management-rotation">
+ <title>Key Rotation Process</title>
+
+ <para>
+ Encryption keys in general are not interminable —
+ the longer the same key is in use, the greater the chance
+ of it being breached. Performing key rotation at regular
+ intervals helps meet standardized security practices such as <ulink
+ url="https://www.pcisecuritystandards.org/";>PCI-DSS</ulink> and it is
+ a good practice in security to limit the number of encrypted bytes
+ available for a specific key version. The key lifetimes are based
+ on key length, key strength, algorithm and total number of bytes
+ enciphered. The key management system provides a efficient method to
+ perform key rotation.
+ </para>
+
+ <para>
+ Please be aware that the phrase <literal>"key rotation"</literal> here
+ only refers to the rotation of <acronym>KEK</acronym>. The cryptographic
+ keys managed by encryption key management system are not rotated; they
+ will in fact be the same before and after a <literal>"key rotation"</literal>.
+ This can be justified because the actual keys are never stored anywhere
+ physically, presented to user or captured in logging. What is being
+ rotated here is the <acronym>KEK</acronym> who is responsible for
+ encapsulating and restoring cryptographic keys.
+ </para>
+
+ <para>
+ Since <acronym>KEK</acronym> is derived from a cluster passphrase, the
+ <literal>"key rotation"</literal> ultimately refers to the rotation of
+ cluster passphrase and deriving a new <acronym>KEK</acronym> from the
+ new cluster passphrase. The new <acronym>KEK</acronym> can then be used
+ to encapsulate all encryptions keys and store the new results in
+ <filename>pg_cryptokeys</filename> directory.
+ </para>
+
+ <para>
+ To complete the cluster passphrase rotation, user needs to follow the
+ steps below:
+ </para>
+ <itemizedlist>
+ <listitem>
+ <para>
+ Ensure the <productname>PostgreSQL</productname> server is running
+ correctly with KMS enabled. Passphrase rotation cannot be completed
+ with the server shut down.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ Update <xref linkend="guc-cluster-passphrase-command"/> parameter and
+ load such that the new command will return a new cluster passphrase.
+ </para>
+ </listitem>
+ <listitem>
+ <para>
+ In a session, execute <function>pg_rotate_cluster_passphrase()</function>
+ SQL function to initiate the rotation. The function returns true upon
+ successful key rotation and false if otherwise.
+ <programlisting>
+ =# SELECT pg_rotate_cluster_passphrase();
+ pg_rotate_cluster_passphrase
+ ------------------------------
+ t
+ (1 row)
+ </programlisting>
+ </para>
+ </listitem>
+ </itemizedlist>
+
+ <para>
+ Upon successful cluster passphrase rotation, all managed cryptographic
+ keys will be re-encapsulated by the new <acronym>KEK</acronym>
+ derived from the new cluster passphrase. The new encapsulated keys
+ will be stored in <filename>pg_cryptokeys</filename> directory.
+ Please note that the cryptographic keys are the same as before; the
+ rotation process only changes the <acronym>KEK</acronym> that is used
+ to encapsulate and verify the actual cryptographic keys. This way,
+ there is no need to decrypt all the encrypted data with the old keys
+ and re-encrypt them with the new.
+ </para>
+
+ <para>
+ In case of a crash during the cluster passphrase rotation
+ process, the key management system is able to recover to the
+ previous sets of cryptographic keys the next time server starts
+ up. This is possible because the key rotation and encapsulation
+ process are done on a separate temporary key directory called
+ <filename>pg_cryptokeys_tmp</filename> and it will replace
+ <filename>pg_cryptokeys</filename> and be deleted only when everything
+ is successfully finished. If the server starts with pg_cryptokeys_tmp
+ folder present, it would indicate that previous attempt of cluster
+ passphrase rotation was not completed. In this case, the server will
+ discard <filename>pg_cryptokeys_tmp</filename> folder and load the
+ keys in <filename>pg_cryptokeys</filename> as usual.
+ </para>
+ </sect2>
+ </sect1>
+ </chapter>
diff --git a/doc/src/sgml/filelist.sgml b/doc/src/sgml/filelist.sgml
new file mode 100644
index 38e8aa0..b96f4ac
*** a/doc/src/sgml/filelist.sgml
--- b/doc/src/sgml/filelist.sgml
***************
*** 49,54 ****
--- 49,55 ----
<!ENTITY wal SYSTEM "wal.sgml">
<!ENTITY logical-replication SYSTEM "logical-replication.sgml">
<!ENTITY jit SYSTEM "jit.sgml">
+ <!ENTITY database-encryption SYSTEM "database-encryption.sgml">
<!-- programmer's guide -->
<!ENTITY bgworker SYSTEM "bgworker.sgml">
diff --git a/doc/src/sgml/func.sgml b/doc/src/sgml/func.sgml
new file mode 100644
index f7f401b..709f0dc
*** a/doc/src/sgml/func.sgml
--- b/doc/src/sgml/func.sgml
*************** SELECT m.* FROM pg_statistic_ext join pg
*** 26735,26738 ****
--- 26735,26837 ----
</sect1>
+ <sect1 id="functions-encryption">
+ <title>Encryption Functions</title>
+
+ <sect2 id="functions-data-encryption">
+ <title>Data Encryption Function</title>
+ <para>
+ The functions shown in
+ <xref linkend="functions-encryption-table"/> are for encrypting
+ and decrypting data with the <literal>SQL key</literal> described in
+ <xref linkend="encryption-key-management"/>.
+ </para>
+
+ <table id="functions-encryption-table">
+ <title>Encryption <acronym>SQL</acronym> Functions</title>
+ <tgroup cols="3">
+ <thead>
+ <row>
+ <entry>Function</entry>
+ <entry>Return Type</entry>
+ <entry>Description</entry>
+ </row>
+ </thead>
+ <tbody>
+
+ <row>
+ <entry>
+ <indexterm>
+ <primary>pg_encrypt</primary>
+ </indexterm>
+ <literal><function>pg_encrypt(<parameter>data</parameter> <type>text</type>)</function></literal>
+ </entry>
+ <entry>
+ <type>bytea</type>
+ </entry>
+ <entry>
+ Encrypt the given data with the internal SQL key
+ </entry>
+ </row>
+
+ <row>
+ <entry>
+ <indexterm>
+ <primary>pg_unwrap</primary>
+ </indexterm>
+ <literal><function>pg_decrypt(<parameter>data</parameter> <type>bytea</type>)</function></literal>
+ </entry>
+ <entry>
+ <type>text</type>
+ </entry>
+ <entry>
+ Decrypt the given data with the internal SQL key
+ </entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+ </sect2>
+
+ <sect2 id="functions-key-management">
+ <title>Key Management Functions</title>
+ <para>
+ The function shown in
+ <xref linkend="functions-key-management-table"/> are for encryption
+ key management described in <xref linkend="encryption-key-management"/>.
+ </para>
+
+ <table id="functions-key-management-table">
+ <title>Encryption Key Management <acronym>SQL</acronym> Functions</title>
+ <tgroup cols="3">
+ <thead>
+ <row>
+ <entry>Function</entry>
+ <entry>Return Type</entry>
+ <entry>Description</entry>
+ </row>
+ </thead>
+ <tbody>
+ <row>
+ <entry>
+ <indexterm>
+ <primary>pg_rotate_cluster_passphrase</primary>
+ </indexterm>
+ <literal><function>pg_rotate_cluster_passphrase()</function></literal>
+ </entry>
+ <entry>
+ <type>boolean</type>
+ </entry>
+ <entry>
+ Rotate the cluster passphrase. See
+ <xref linkend="key-management-rotation"/> for details.
+ </entry>
+ </row>
+
+ </tbody>
+ </tgroup>
+ </table>
+ </sect2>
+ </sect1>
+
</chapter>
diff --git a/doc/src/sgml/installation.sgml b/doc/src/sgml/installation.sgml
new file mode 100644
index 0ac1cb9..ed58601
*** a/doc/src/sgml/installation.sgml
--- b/doc/src/sgml/installation.sgml
*************** build-postgresql:
*** 976,983 ****
<listitem>
<para>
Build with support for <acronym>SSL</acronym> (encrypted)
! connections. This requires the <productname>OpenSSL</productname>
! package to be installed. <filename>configure</filename> will check
for the required header files and libraries to make sure that
your <productname>OpenSSL</productname> installation is sufficient
before proceeding.
--- 976,984 ----
<listitem>
<para>
Build with support for <acronym>SSL</acronym> (encrypted)
! connections and key management. This requires the
! <productname>OpenSSL</productname> package to be installed.
! <filename>configure</filename> will check
for the required header files and libraries to make sure that
your <productname>OpenSSL</productname> installation is sufficient
before proceeding.
diff --git a/doc/src/sgml/postgres.sgml b/doc/src/sgml/postgres.sgml
new file mode 100644
index 730d5fd..0ea7da6
*** a/doc/src/sgml/postgres.sgml
--- b/doc/src/sgml/postgres.sgml
*************** break is not needed in a wider output re
*** 171,176 ****
--- 171,177 ----
&wal;
&logical-replication;
&jit;
+ &database-encryption;
®ress;
</part>
diff --git a/doc/src/sgml/ref/initdb.sgml b/doc/src/sgml/ref/initdb.sgml
new file mode 100644
index 385ac25..1d0678a
*** a/doc/src/sgml/ref/initdb.sgml
--- b/doc/src/sgml/ref/initdb.sgml
*************** PostgreSQL documentation
*** 163,168 ****
--- 163,187 ----
</listitem>
</varlistentry>
+ <varlistentry id="app-initdb-cluster-passphrase-command" xreflabel="cluster passphrase command">
+ <term><option>--cluster-passphrase-command=<replaceable class="parameter">command</replaceable></option></term>
+ <listitem>
+ <para>
+ This option specifies an external command to be invoked when a passphrase
+ for key management system needs to be obtained.
+ </para>
+ <para>
+ The command must print the passphrase to the standard output and exit
+ with code 0. In the parameter value, <literal>%p</literal> is
+ replaced by a prompt string. (Write <literal>%%</literal> for a
+ literal <literal>%</literal>.) Note that the prompt string will
+ probably contain whitespace, so be sure to quote adequately. A single
+ newline is stripped from the end of the output if present. The passphrase
+ must be at least 64 bytes.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>-D <replaceable class="parameter">directory</replaceable></option></term>
<term><option>--pgdata=<replaceable class="parameter">directory</replaceable></option></term>
diff --git a/doc/src/sgml/ref/pg_ctl-ref.sgml b/doc/src/sgml/ref/pg_ctl-ref.sgml
new file mode 100644
index 3946fa5..3d34071
*** a/doc/src/sgml/ref/pg_ctl-ref.sgml
--- b/doc/src/sgml/ref/pg_ctl-ref.sgml
*************** PostgreSQL documentation
*** 38,43 ****
--- 38,44 ----
<arg choice="opt"><option>-s</option></arg>
<arg choice="opt"><option>-o</option> <replaceable>options</replaceable></arg>
<arg choice="opt"><option>-p</option> <replaceable>path</replaceable></arg>
+ <arg choice="opt"><option>-R</option></arg>
<arg choice="opt"><option>-c</option></arg>
</cmdsynopsis>
*************** PostgreSQL documentation
*** 72,77 ****
--- 73,79 ----
<arg choice="opt"><option>-t</option> <replaceable>seconds</replaceable></arg>
<arg choice="opt"><option>-s</option></arg>
<arg choice="opt"><option>-o</option> <replaceable>options</replaceable></arg>
+ <arg choice="opt"><option>-R</option></arg>
<arg choice="opt"><option>-c</option></arg>
</cmdsynopsis>
*************** PostgreSQL documentation
*** 372,377 ****
--- 374,390 ----
</para>
</listitem>
</varlistentry>
+
+ <varlistentry>
+ <term><option>-R</option></term>
+ <term><option>--read-terminal</option></term>
+ <listitem>
+ <para>
+ Allow the server to read a password from the terminal.
+ This also disables dot-status progress reporting.
+ </para>
+ </listitem>
+ </varlistentry>
<varlistentry>
<term><option>-s</option></term>
diff --git a/doc/src/sgml/ref/pgupgrade.sgml b/doc/src/sgml/ref/pgupgrade.sgml
new file mode 100644
index b59c569..29b0015
*** a/doc/src/sgml/ref/pgupgrade.sgml
--- b/doc/src/sgml/ref/pgupgrade.sgml
*************** psql --username=postgres --file=script.s
*** 823,828 ****
--- 823,835 ----
is down.
</para>
+ <para>
+ During the upgrade <command>pg_upgrade</command> copies the all internal keys
+ to the new cluster. If you want to upgrade from the old cluster that enables
+ the key management to the new cluster that also enables, you must use the same
+ <varname>cluster_passphrase_command</varname> to both clusters. Otherwise
+ <command>pg_upgrade</command> fails due to mismatching the cluster passphrase.
+ </para>
</refsect1>
<refsect1>
diff --git a/doc/src/sgml/storage.sgml b/doc/src/sgml/storage.sgml
new file mode 100644
index 3234adb..05e9b26
*** a/doc/src/sgml/storage.sgml
--- b/doc/src/sgml/storage.sgml
*************** Item
*** 78,83 ****
--- 78,88 ----
</row>
<row>
+ <entry><filename>pg_cryptokeys</filename></entry>
+ <entry>Subdirectory containing cryptographic keys</entry>
+ </row>
+
+ <row>
<entry><filename>pg_dynshmem</filename></entry>
<entry>Subdirectory containing files used by the dynamic shared memory
subsystem</entry>
diff --git a/src/backend/Makefile b/src/backend/Makefile
new file mode 100644
index 9706a95..4ace302
*** a/src/backend/Makefile
--- b/src/backend/Makefile
*************** SUBDIRS = access bootstrap catalog parse
*** 21,27 ****
main nodes optimizer partitioning port postmaster \
regex replication rewrite \
statistics storage tcop tsearch utils $(top_builddir)/src/timezone \
! jit
include $(srcdir)/common.mk
--- 21,27 ----
main nodes optimizer partitioning port postmaster \
regex replication rewrite \
statistics storage tcop tsearch utils $(top_builddir)/src/timezone \
! jit crypto
include $(srcdir)/common.mk
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
new file mode 100644
index 52a67b1..955fd4a
*** a/src/backend/access/transam/xlog.c
--- b/src/backend/access/transam/xlog.c
***************
*** 44,54 ****
--- 44,56 ----
#include "commands/tablespace.h"
#include "common/controldata_utils.h"
#include "executor/instrument.h"
+ #include "crypto/kmgr.h"
#include "miscadmin.h"
#include "pg_trace.h"
#include "pgstat.h"
#include "port/atomics.h"
#include "postmaster/bgwriter.h"
+ #include "postmaster/postmaster.h"
#include "postmaster/startup.h"
#include "postmaster/walwriter.h"
#include "replication/basebackup.h"
***************
*** 81,86 ****
--- 83,89 ----
#include "utils/timestamp.h"
extern uint32 bootstrap_data_checksum_version;
+ extern uint32 bootstrap_key_management_version;
/* Unsupported old recovery command file names (relative to $PGDATA) */
#define RECOVERY_COMMAND_FILE "recovery.conf"
*************** InitControlFile(uint64 sysidentifier)
*** 4607,4612 ****
--- 4610,4616 ----
ControlFile->wal_log_hints = wal_log_hints;
ControlFile->track_commit_timestamp = track_commit_timestamp;
ControlFile->data_checksum_version = bootstrap_data_checksum_version;
+ ControlFile->key_management_version = bootstrap_key_management_version;
}
static void
*************** ReadControlFile(void)
*** 4894,4899 ****
--- 4898,4906 ----
/* Make the initdb settings visible as GUC variables, too */
SetConfigOption("data_checksums", DataChecksumsEnabled() ? "yes" : "no",
PGC_INTERNAL, PGC_S_OVERRIDE);
+
+ SetConfigOption("key_management", KeyManagementEnabled() ? "yes" : "no",
+ PGC_INTERNAL, PGC_S_OVERRIDE);
}
/*
*************** DataChecksumsEnabled(void)
*** 4937,4942 ****
--- 4944,4959 ----
}
/*
+ * Are key management enabled?
+ */
+ bool
+ KeyManagementEnabled(void)
+ {
+ Assert(ControlFile != NULL);
+ return (ControlFile->key_management_version > 0);
+ }
+
+ /*
* Returns a fake LSN for unlogged relations.
*
* Each call generates an LSN that is greater than any previous value
*************** BootStrapXLOG(void)
*** 5343,5348 ****
--- 5360,5371 ----
/* some additional ControlFile fields are set in WriteControlFile() */
WriteControlFile();
+ /* Enable key manager if required */
+ if (ControlFile->key_management_version > 0)
+ BootStrapKmgr();
+ if (terminal_fd != -1)
+ close(terminal_fd);
+
/* Bootstrap the commit log, too */
BootStrapCLOG();
BootStrapCommitTs();
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
new file mode 100644
index 76b2f50..bd7c04e
*** a/src/backend/bootstrap/bootstrap.c
--- b/src/backend/bootstrap/bootstrap.c
***************
*** 28,39 ****
--- 28,41 ----
#include "catalog/pg_collation.h"
#include "catalog/pg_type.h"
#include "common/link-canary.h"
+ #include "crypto/kmgr.h"
#include "libpq/pqsignal.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
#include "pg_getopt.h"
#include "pgstat.h"
#include "postmaster/bgwriter.h"
+ #include "postmaster/postmaster.h"
#include "postmaster/startup.h"
#include "postmaster/walwriter.h"
#include "replication/walreceiver.h"
***************
*** 51,56 ****
--- 53,59 ----
#include "utils/relmapper.h"
uint32 bootstrap_data_checksum_version = 0; /* No checksum */
+ uint32 bootstrap_key_management_version = 0; /* disabled */
static void CheckerModeMain(void);
*************** AuxiliaryProcessMain(int argc, char *arg
*** 224,230 ****
/* If no -x argument, we are a CheckerProcess */
MyAuxProcType = CheckerProcess;
! while ((flag = getopt(argc, argv, "B:c:d:D:Fkr:x:X:-:")) != -1)
{
switch (flag)
{
--- 227,233 ----
/* If no -x argument, we are a CheckerProcess */
MyAuxProcType = CheckerProcess;
! while ((flag = getopt(argc, argv, "B:c:d:D:eFkr:R:x:X:-:")) != -1)
{
switch (flag)
{
*************** AuxiliaryProcessMain(int argc, char *arg
*** 247,252 ****
--- 250,258 ----
pfree(debugstr);
}
break;
+ case 'e':
+ bootstrap_key_management_version = KMGR_VERSION;
+ break;
case 'F':
SetConfigOption("fsync", "false", PGC_POSTMASTER, PGC_S_ARGV);
break;
*************** AuxiliaryProcessMain(int argc, char *arg
*** 256,261 ****
--- 262,270 ----
case 'r':
strlcpy(OutputFileName, optarg, MAXPGPATH);
break;
+ case 'R':
+ terminal_fd = atoi(optarg);
+ break;
case 'x':
MyAuxProcType = atoi(optarg);
break;
diff --git a/src/backend/crypto/Makefile b/src/backend/crypto/Makefile
new file mode 100644
index ...c273620
*** a/src/backend/crypto/Makefile
--- b/src/backend/crypto/Makefile
***************
*** 0 ****
--- 1,18 ----
+ #-------------------------------------------------------------------------
+ #
+ # Makefile
+ # Makefile for src/backend/crypto
+ #
+ # IDENTIFICATION
+ # src/backend/crypto/Makefile
+ #
+ #-------------------------------------------------------------------------
+
+ subdir = src/backend/crypto
+ top_builddir = ../../..
+ include $(top_builddir)/src/Makefile.global
+
+ OBJS = \
+ kmgr.o
+
+ include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/crypto/kmgr.c b/src/backend/crypto/kmgr.c
new file mode 100644
index ...410789f
*** a/src/backend/crypto/kmgr.c
--- b/src/backend/crypto/kmgr.c
***************
*** 0 ****
--- 1,461 ----
+ /*-------------------------------------------------------------------------
+ *
+ * kmgr.c
+ * Key manager routines
+ *
+ * Key manager is enabled if user requests during initdb. During bootstrap,
+ * we generate internal keys, wrap them with KEK which is derived from the
+ * user-provided passphrase, and store them into each file located at KMGR_DIR.
+ * Once generated, these are not changed. During startup, we decrypt all
+ * internal keys and load them to the shared memory space. Internal keys on
+ * the shared memory are read-only. All wrapping and unwrapping key routines
+ * depends on openssl library for now.
+ *
+ * Copyright (c) 2020, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ * src/backend/crypto/kmgr.c
+ *-------------------------------------------------------------------------
+ */
+
+ #include "postgres.h"
+
+ #include <sys/stat.h>
+ #include <unistd.h>
+
+ #include "funcapi.h"
+ #include "miscadmin.h"
+ #include "pgstat.h"
+
+ #include "common/sha2.h"
+ #include "common/kmgr_utils.h"
+ #include "crypto/kmgr.h"
+ #include "storage/fd.h"
+ #include "storage/ipc.h"
+ #include "storage/shmem.h"
+ #include "utils/builtins.h"
+ #include "utils/guc.h"
+ #include "utils/memutils.h"
+
+ /* Struct stores internal keys in plaintext format */
+ typedef struct KmgrShmemData
+ {
+ /*
+ * Internal cryptographic keys. Keys are stored at its ID'th.
+ */
+ CryptoKey intlKeys[KMGR_MAX_INTERNAL_KEYS];
+ } KmgrShmemData;
+ static KmgrShmemData *KmgrShmem;
+
+ /*
+ * Key lengths in bytes of internal keys.
+ *
+ * No key supported for now.
+ */
+ static int internalKeyLengths[KMGR_MAX_INTERNAL_KEYS];
+
+ /* GUC variables */
+ bool key_management_enabled = false;;
+ char *cluster_passphrase_command = NULL;
+
+ static void KmgrSaveCryptoKeys(const char *dir, CryptoKey *keys);
+ static CryptoKey *generate_crypto_key(int len);
+ static void recoverIncompleteRotation(void);
+
+ /*
+ * This function must be called ONCE on system install.
+ */
+ void
+ BootStrapKmgr(void)
+ {
+ PgKeyWrapCtx *ctx;
+ CryptoKey keys_wrap[KMGR_MAX_INTERNAL_KEYS];
+ char passphrase[KMGR_MAX_PASSPHRASE_LEN];
+ uint8 kekenc[KMGR_ENC_KEY_LEN];
+ uint8 kekhmac[KMGR_MAC_KEY_LEN];
+ int passlen;
+
+ /*
+ * Requirement check. We need openssl library to enable key management
+ * because all encryption and decryption calls happen via openssl function
+ * calls.
+ */
+ #ifndef USE_OPENSSL
+ ereport(ERROR,
+ (errcode(ERRCODE_CONFIG_FILE_ERROR),
+ (errmsg("cluster encryption is not supported because OpenSSL is not supported by this build"),
+ errhint("Compile with --with-openssl to use cluster encryption."))));
+ #endif
+
+ memset(keys_wrap, 0, sizeof(keys_wrap));
+
+ /* Get key encryption key from the passphrase command */
+ passlen = kmgr_run_cluster_passphrase_command(cluster_passphrase_command,
+ passphrase, KMGR_MAX_PASSPHRASE_LEN);
+ if (passlen < KMGR_MIN_PASSPHRASE_LEN)
+ ereport(ERROR,
+ (errmsg("passphrase must be more than %d bytes",
+ KMGR_MIN_PASSPHRASE_LEN)));
+
+ /* Get key encryption key and HMAC key from passphrase */
+ kmgr_derive_keys(passphrase, passlen, kekenc, kekhmac);
+
+ /* Create temporarily key wrap context */
+ ctx = pg_create_keywrap_ctx(kekenc, kekhmac);
+ if (!ctx)
+ elog(ERROR, "could not initialize encryption contect");
+
+ /* Wrap all internal keys by key encryption key */
+ for (int id = 0; id < KMGR_MAX_INTERNAL_KEYS; id++)
+ {
+ CryptoKey *key;
+
+ /* generate an internal key */
+ key = generate_crypto_key(internalKeyLengths[id]);
+
+ if (!kmgr_wrap_key(ctx, key, &(keys_wrap[id])))
+ {
+ pg_free_keywrap_ctx(ctx);
+ elog(ERROR, "failed to wrap cluster encryption key");
+ }
+ }
+
+ /* Save internal keys to the disk */
+ KmgrSaveCryptoKeys(KMGR_DIR, keys_wrap);
+
+ pg_free_keywrap_ctx(ctx);
+ }
+
+ /* Report shared-memory space needed by KmgrShmem */
+ Size
+ KmgrShmemSize(void)
+ {
+ if (!key_management_enabled)
+ return 0;
+
+ return MAXALIGN(sizeof(KmgrShmemData));
+ }
+
+ /* Allocate and initialize key manager memory */
+ void
+ KmgrShmemInit(void)
+ {
+ bool found;
+
+ if (!key_management_enabled)
+ return;
+
+ KmgrShmem = (KmgrShmemData *) ShmemInitStruct("Key manager",
+ KmgrShmemSize(), &found);
+
+ if (!found)
+ memset(KmgrShmem, 0, KmgrShmemSize());
+ }
+
+ /*
+ * Get encryption key passphrase and verify it, then get the internal keys.
+ * This function is called by postmaster at startup time.
+ */
+ void
+ InitializeKmgr(void)
+ {
+ CryptoKey *keys_wrap;
+ char passphrase[KMGR_MAX_PASSPHRASE_LEN];
+ int passlen;
+ int nkeys;
+
+ if (!key_management_enabled)
+ return;
+
+ elog(DEBUG1, "starting up key management system");
+
+ /* Recover the failure of the last passphrase rotation if necessary */
+ recoverIncompleteRotation();
+
+ /* Get the crypto keys from the file */
+ keys_wrap = kmgr_get_cryptokeys(KMGR_DIR, &nkeys);
+ Assert(nkeys == KMGR_MAX_INTERNAL_KEYS);
+
+ /* Get cluster passphrase */
+ passlen = kmgr_run_cluster_passphrase_command(cluster_passphrase_command,
+ passphrase, KMGR_MAX_PASSPHRASE_LEN);
+
+ /*
+ * Verify passphrase and prepare an internal key in plaintext on shared memory.
+ *
+ * XXX: do we need to prevent internal keys from being swapped out using
+ * mlock?
+ */
+ if (!kmgr_verify_passphrase(passphrase, passlen, keys_wrap, KmgrShmem->intlKeys,
+ KMGR_MAX_INTERNAL_KEYS))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("cluster passphrase does not match expected passphrase")));
+ }
+
+ const CryptoKey *
+ KmgrGetKey(int id)
+ {
+ Assert(id < KMGR_MAX_INTERNAL_KEYS);
+
+ return (const CryptoKey *) &(KmgrShmem->intlKeys[id]);
+ }
+
+ /* Generate an empty CryptoKey */
+ static CryptoKey *
+ generate_crypto_key(int len)
+ {
+ CryptoKey *newkey;
+
+ Assert(len < KMGR_MAX_KEY_LEN);
+ newkey = (CryptoKey *) palloc0(sizeof(CryptoKey));
+
+ if (!pg_strong_random(newkey->key, len))
+ elog(ERROR, "failed to generate new crypto key");
+
+ newkey->klen = len;
+
+ return newkey;
+ }
+
+ /*
+ * Save the given crypto keys to the disk. We don't need CRC check for crypto
+ * keys because these keys have HMAC which is used for integrity check
+ * during unwrapping.
+ */
+ static void
+ KmgrSaveCryptoKeys(const char *dir, CryptoKey *keys)
+ {
+ elog(DEBUG2, "saving all cryptographic keys");
+
+ for (int i = 0; i < KMGR_MAX_INTERNAL_KEYS; i++)
+ {
+ int fd;
+ char path[MAXPGPATH];
+
+ CryptoKeyFilePath(path, dir, i);
+
+ if ((fd = BasicOpenFile(path, O_RDWR | O_CREAT | O_EXCL | PG_BINARY)) < 0)
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not open file \"%s\": %m",
+ path)));
+
+ errno = 0;
+ pgstat_report_wait_start(WAIT_EVENT_KEY_FILE_WRITE);
+ if (write(fd, &(keys[i]), sizeof(CryptoKey)) != sizeof(CryptoKey))
+ {
+ /* if write didn't set errno, assume problem is no disk space */
+ if (errno == 0)
+ errno = ENOSPC;
+
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not write file \"%s\": %m",
+ path)));
+ }
+ pgstat_report_wait_end();
+
+ pgstat_report_wait_start(WAIT_EVENT_KEY_FILE_SYNC);
+ if (pg_fsync(fd) != 0)
+ ereport(PANIC,
+ (errcode_for_file_access(),
+ errmsg("could not fsync file \"%s\": %m",
+ path)));
+ pgstat_report_wait_end();
+
+ if (close(fd) != 0)
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not close file \"%s\": %m",
+ path)));
+ }
+ }
+
+
+ /*
+ * Check the last passphrase rotation was completed. If not, we decide which wrapped
+ * keys will be used according to the status of temporary directory and its wrapped
+ * keys.
+ */
+ static void
+ recoverIncompleteRotation(void)
+ {
+ struct stat st;
+ struct stat st_tmp;
+ CryptoKey *keys;
+ int nkeys_tmp;
+
+ /* The cluster passphrase rotation was completed, nothing to do */
+ if (stat(KMGR_TMP_DIR, &st_tmp) != 0)
+ return;
+
+ /*
+ * If there is only temporary directory, it means that the previous
+ * rotation failed after wrapping the all internal keys by the new
+ * passphrase. Therefore we use the new cluster passphrase.
+ */
+ if (stat(KMGR_DIR, &st) != 0)
+ {
+ ereport(DEBUG1,
+ (errmsg("there is only temporary directory, use the newly wrapped keys")));
+
+ if (rename(KMGR_TMP_DIR, KMGR_DIR) != 0)
+ ereport(ERROR,
+ errmsg("could not rename directory \"%s\" to \"%s\": %m",
+ KMGR_TMP_DIR, KMGR_DIR));
+ ereport(LOG,
+ errmsg("cryptographic keys wrapped by new passphrase command are chosen"),
+ errdetail("last cluster passphrase rotation failed in the middle"));
+ return;
+ }
+
+ /*
+ * In case where both the original directory and temporary directory
+ * exist, there are two possibilities: (a) the all internal keys are
+ * wrapped by the new passphrase but rotation failed before removing the
+ * original directory, or (b) the rotation failed during wrapping internal
+ * keys by the new passphrase. In case of (a) we need to use the wrapped
+ * keys in the temporary directory as rotation is essentially completed,
+ * but in case of (b) we use the wrapped keys in the original directory.
+ *
+ * To check the possibility of (b) we validate the wrapped keys in the
+ * temporary directory by checking the number of wrapped keys. Since the
+ * wrapped key length is smaller than one disk sector, which is 512 bytes
+ * on common hardware, saving wrapped key is atomic write. So we can
+ * ensure that the all wrapped keys are valid if the number of wrapped
+ * keys in the temporary directory is KMGR_MAX_INTERNAL_KEYS.
+ */
+ keys = kmgr_get_cryptokeys(KMGR_TMP_DIR, &nkeys_tmp);
+
+ if (nkeys_tmp == KMGR_MAX_INTERNAL_KEYS)
+ {
+ /*
+ * This is case (a), the all wrapped keys in temporary directory are
+ * valid. Remove the original directory and rename.
+ */
+ ereport(DEBUG1,
+ (errmsg("last passphrase rotation failed before renaming direcotry name, use the newly wrapped keys")));
+
+ if (!rmtree(KMGR_DIR, true))
+ ereport(ERROR,
+ (errmsg("could not remove directory \"%s\"",
+ KMGR_DIR)));
+ if (rename(KMGR_TMP_DIR, KMGR_DIR) != 0)
+ ereport(ERROR,
+ errmsg("could not rename directory \"%s\" to \"%s\": %m",
+ KMGR_TMP_DIR, KMGR_DIR));
+
+ ereport(LOG,
+ errmsg("cryptographic keys wrapped by new passphrase command are chosen"),
+ errdetail("last cluster passphrase rotation failed in the middle"));
+ }
+ else
+ {
+ /*
+ * This is case (b), the last passphrase rotation failed during
+ * wrapping keys. Remove the keys in the temporary directory and use
+ * keys in the original keys.
+ */
+ ereport(DEBUG1,
+ (errmsg("last passphrase rotation failed during wrapping keys, use the old wrapped keys")));
+
+ if (!rmtree(KMGR_TMP_DIR, true))
+ ereport(ERROR,
+ (errmsg("could not remove directory \"%s\"",
+ KMGR_DIR)));
+ ereport(LOG,
+ errmsg("cryptographic keys wrapped by old passphrase command are chosen"),
+ errdetail("last cluster passphrase rotation failed in the middle"));
+ }
+
+ pfree(keys);
+ }
+
+ /*
+ * SQL function to rotate the cluster passphrase. This function assumes that
+ * the cluster_passphrase_command is already reloaded to the new value.
+ * All internal keys are wrapped by the new passphrase and saved to the disk.
+ * To update all crypto keys atomically we save the newly wrapped keys to the
+ * temporary directory, pg_cryptokeys_tmp, and remove the original directory,
+ * pg_cryptokeys, and rename it. These operation is performed without the help
+ * of WAL. In the case of failure during rotationpg_cryptokeys directory and
+ * pg_cryptokeys_tmp directory can be left in incomplete status. We recover
+ * the incomplete situation by checkIncompleteRotation.
+ */
+ Datum
+ pg_rotate_cluster_passphrase(PG_FUNCTION_ARGS)
+ {
+ PgKeyWrapCtx *ctx;
+ CryptoKey newkeys[KMGR_MAX_INTERNAL_KEYS];
+ char passphrase[KMGR_MAX_PASSPHRASE_LEN];
+ uint8 new_kekenc[KMGR_ENC_KEY_LEN];
+ uint8 new_kekhmac[KMGR_MAC_KEY_LEN];
+ int passlen;
+
+ if (!key_management_enabled)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("could not rotate cluster passphrase because key management is not supported")));
+
+ memset(newkeys, 0, sizeof(newkeys));
+
+ /* Recover the failure of the last passphrase rotation if necessary */
+ recoverIncompleteRotation();
+
+ /*
+ * I think this has to be done from a stand-alone backend since the
+ * file descriptor was already closed.
+ */
+ passlen = kmgr_run_cluster_passphrase_command(cluster_passphrase_command,
+ passphrase,
+ KMGR_MAX_PASSPHRASE_LEN);
+ if (passlen < KMGR_MIN_PASSPHRASE_LEN)
+ ereport(ERROR,
+ (errmsg("passphrase must be more than %d bytes",
+ KMGR_MIN_PASSPHRASE_LEN)));
+
+ /* Get new key encryption key and encryption context */
+ kmgr_derive_keys(passphrase, passlen, new_kekenc, new_kekhmac);
+ ctx = pg_create_keywrap_ctx(new_kekenc, new_kekhmac);
+ if (!ctx)
+ elog(ERROR, "could not initialize encryption contect");
+
+ for (int id = 0; id < KMGR_MAX_INTERNAL_KEYS; id++)
+ {
+ if (!kmgr_wrap_key(ctx, &(KmgrShmem->intlKeys[id]), &(newkeys[id])))
+ elog(ERROR, "failed to wrap key");
+ }
+
+ /* Create temporary directory */
+ if (MakePGDirectory(KMGR_TMP_DIR) < 0)
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not create temporary directory \"%s\": %m",
+ KMGR_TMP_DIR)));
+ fsync_fname(KMGR_TMP_DIR, true);
+
+ /* Prevent concurrent key rotation */
+ LWLockAcquire(KmgrFileLock, LW_EXCLUSIVE);
+
+ /* Save the key wrapped by the new passphrase to the temporary directory */
+ KmgrSaveCryptoKeys(KMGR_TMP_DIR, newkeys);
+
+ /* Remove the original directory */
+ if (!rmtree(KMGR_DIR, true))
+ ereport(ERROR,
+ (errmsg("could not remove directory \"%s\"",
+ KMGR_DIR)));
+
+ /* Rename to the original directory */
+ if (rename(KMGR_TMP_DIR, KMGR_DIR) != 0)
+ ereport(ERROR,
+ (errmsg("could not rename directory \"%s\" to \"%s\": %m",
+ KMGR_TMP_DIR, KMGR_DIR)));
+ fsync_fname(KMGR_DIR, true);
+
+ LWLockRelease(KmgrFileLock);
+
+ pg_free_keywrap_ctx(ctx);
+ PG_RETURN_BOOL(true);
+ }
diff --git a/src/backend/libpq/be-secure-openssl.c b/src/backend/libpq/be-secure-openssl.c
new file mode 100644
index 8b21ff4..744fa96
*** a/src/backend/libpq/be-secure-openssl.c
--- b/src/backend/libpq/be-secure-openssl.c
***************
*** 30,35 ****
--- 30,36 ----
#endif
#include <openssl/ssl.h>
+ #include <openssl/conf.h>
#include <openssl/dh.h>
#include <openssl/conf.h>
#ifndef OPENSSL_NO_ECDH
diff --git a/src/backend/main/main.c b/src/backend/main/main.c
new file mode 100644
index a4dd233..bd4c2a3
*** a/src/backend/main/main.c
--- b/src/backend/main/main.c
*************** help(const char *progname)
*** 325,330 ****
--- 325,331 ----
printf(_(" -N MAX-CONNECT maximum number of allowed connections\n"));
printf(_(" -o OPTIONS pass \"OPTIONS\" to each server process (obsolete)\n"));
printf(_(" -p PORT port number to listen on\n"));
+ printf(_(" -R fd allow password to be read from numeric file descriptor\n"));
printf(_(" -s show statistics after each query\n"));
printf(_(" -S WORK-MEM set amount of memory for sorts (in kB)\n"));
printf(_(" -V, --version output version information, then exit\n"));
diff --git a/src/backend/postmaster/pgstat.c b/src/backend/postmaster/pgstat.c
new file mode 100644
index 822f0eb..a1faea7
*** a/src/backend/postmaster/pgstat.c
--- b/src/backend/postmaster/pgstat.c
*************** pgstat_get_wait_io(WaitEventIO w)
*** 4134,4139 ****
--- 4134,4148 ----
case WAIT_EVENT_DSM_FILL_ZERO_WRITE:
event_name = "DSMFillZeroWrite";
break;
+ case WAIT_EVENT_KEY_FILE_READ:
+ event_name = "KeyFileRead";
+ break;
+ case WAIT_EVENT_KEY_FILE_WRITE:
+ event_name = "KeyFileWrite";
+ break;
+ case WAIT_EVENT_KEY_FILE_SYNC:
+ event_name = "KeyFileSync";
+ break;
case WAIT_EVENT_LOCK_FILE_ADDTODATADIR_READ:
event_name = "LockFileAddToDataDirRead";
break;
diff --git a/src/backend/postmaster/postmaster.c b/src/backend/postmaster/postmaster.c
new file mode 100644
index 959e3b8..00c82a7
*** a/src/backend/postmaster/postmaster.c
--- b/src/backend/postmaster/postmaster.c
***************
*** 100,105 ****
--- 100,106 ----
#include "common/file_perm.h"
#include "common/ip.h"
#include "common/string.h"
+ #include "crypto/kmgr.h"
#include "lib/ilist.h"
#include "libpq/auth.h"
#include "libpq/libpq.h"
*************** static int SendStop = false;
*** 237,242 ****
--- 238,244 ----
/* still more option variables */
bool EnableSSL = false;
+ int terminal_fd = -1;
int PreAuthDelay = 0;
int AuthenticationTimeout = 60;
*************** PostmasterMain(int argc, char *argv[])
*** 694,700 ****
* tcop/postgres.c (the option sets should not conflict) and with the
* common help() function in main/main.c.
*/
! while ((opt = getopt(argc, argv, "B:bc:C:D:d:EeFf:h:ijk:lN:nOo:Pp:r:S:sTt:W:-:")) != -1)
{
switch (opt)
{
--- 696,702 ----
* tcop/postgres.c (the option sets should not conflict) and with the
* common help() function in main/main.c.
*/
! while ((opt = getopt(argc, argv, "B:bc:C:D:d:EeFf:h:ijk:lN:nOo:Pp:R:r:S:sTt:W:-:")) != -1)
{
switch (opt)
{
*************** PostmasterMain(int argc, char *argv[])
*** 792,797 ****
--- 794,803 ----
/* only used by single-user backend */
break;
+ case 'R':
+ terminal_fd = atoi(optarg);
+ break;
+
case 'S':
SetConfigOption("work_mem", optarg, PGC_POSTMASTER, PGC_S_ARGV);
break;
*************** PostmasterMain(int argc, char *argv[])
*** 1351,1356 ****
--- 1357,1369 ----
autovac_init();
/*
+ * Initialize key manager.
+ */
+ InitializeKmgr();
+ if (terminal_fd != -1)
+ close(terminal_fd);
+
+ /*
* Load configuration files for client authentication.
*/
if (!load_hba())
diff --git a/src/backend/replication/basebackup.c b/src/backend/replication/basebackup.c
new file mode 100644
index b89df01..6ad051d
*** a/src/backend/replication/basebackup.c
--- b/src/backend/replication/basebackup.c
***************
*** 18,23 ****
--- 18,24 ----
#include "access/xlog_internal.h" /* for pg_start/stop_backup */
#include "catalog/pg_type.h"
+ #include "common/kmgr_utils.h"
#include "common/file_perm.h"
#include "commands/progress.h"
#include "lib/stringinfo.h"
*************** struct exclude_list_item
*** 152,157 ****
--- 153,161 ----
*/
static const char *const excludeDirContents[] =
{
+ /* Skip temporary crypto key files */
+ KMGR_TMP_DIR,
+
/*
* Skip temporary statistics files. PG_STAT_TMP_DIR must be skipped even
* when stats_temp_directory is set because PGSS_TEXT_FILE is always
diff --git a/src/backend/storage/ipc/ipci.c b/src/backend/storage/ipc/ipci.c
new file mode 100644
index 96c2aaa..64fe10c
*** a/src/backend/storage/ipc/ipci.c
--- b/src/backend/storage/ipc/ipci.c
***************
*** 23,28 ****
--- 23,29 ----
#include "access/syncscan.h"
#include "access/twophase.h"
#include "commands/async.h"
+ #include "crypto/kmgr.h"
#include "miscadmin.h"
#include "pgstat.h"
#include "postmaster/autovacuum.h"
*************** CreateSharedMemoryAndSemaphores(void)
*** 149,154 ****
--- 150,156 ----
size = add_size(size, BTreeShmemSize());
size = add_size(size, SyncScanShmemSize());
size = add_size(size, AsyncShmemSize());
+ size = add_size(size, KmgrShmemSize());
#ifdef EXEC_BACKEND
size = add_size(size, ShmemBackendArraySize());
#endif
*************** CreateSharedMemoryAndSemaphores(void)
*** 267,272 ****
--- 269,275 ----
BTreeShmemInit();
SyncScanShmemInit();
AsyncShmemInit();
+ KmgrShmemInit();
#ifdef EXEC_BACKEND
diff --git a/src/backend/storage/lmgr/lwlocknames.txt b/src/backend/storage/lmgr/lwlocknames.txt
new file mode 100644
index 774292f..a44805d
*** a/src/backend/storage/lmgr/lwlocknames.txt
--- b/src/backend/storage/lmgr/lwlocknames.txt
*************** XactTruncationLock 44
*** 53,55 ****
--- 53,56 ----
# 45 was XactTruncationLock until removal of BackendRandomLock
WrapLimitsVacuumLock 46
NotifyQueueTailLock 47
+ KmgrFileLock 48
diff --git a/src/backend/tcop/postgres.c b/src/backend/tcop/postgres.c
new file mode 100644
index 411cfad..ab464d9
*** a/src/backend/tcop/postgres.c
--- b/src/backend/tcop/postgres.c
***************
*** 42,47 ****
--- 42,48 ----
#include "catalog/pg_type.h"
#include "commands/async.h"
#include "commands/prepare.h"
+ #include "crypto/kmgr.h"
#include "executor/spi.h"
#include "jit/jit.h"
#include "libpq/libpq.h"
*************** process_postgres_switches(int argc, char
*** 3554,3560 ****
* postmaster/postmaster.c (the option sets should not conflict) and with
* the common help() function in main/main.c.
*/
! while ((flag = getopt(argc, argv, "B:bc:C:D:d:EeFf:h:ijk:lN:nOo:Pp:r:S:sTt:v:W:-:")) != -1)
{
switch (flag)
{
--- 3555,3561 ----
* postmaster/postmaster.c (the option sets should not conflict) and with
* the common help() function in main/main.c.
*/
! while ((flag = getopt(argc, argv, "B:bc:C:D:d:EeFf:h:ijk:lN:nOo:Pp:R:r:S:sTt:v:W:-:")) != -1)
{
switch (flag)
{
*************** process_postgres_switches(int argc, char
*** 3650,3655 ****
--- 3651,3660 ----
strlcpy(OutputFileName, optarg, MAXPGPATH);
break;
+ case 'R':
+ terminal_fd = atoi(optarg);
+ break;
+
case 'S':
SetConfigOption("work_mem", optarg, ctx, gucsource);
break;
*************** PostgresMain(int argc, char *argv[],
*** 3902,3907 ****
--- 3907,3921 ----
BaseInit();
/*
+ * Initialize kmgr for cluster encryption. Since kmgr needs to attach to
+ * shared memory the initialization must be called after BaseInit().
+ */
+ if (!IsUnderPostmaster)
+ InitializeKmgr();
+ if (terminal_fd != -1)
+ close(terminal_fd);
+
+ /*
* Create a per-backend PGPROC struct in shared memory, except in the
* EXEC_BACKEND case where this was done in SubPostmasterMain. We must do
* this before we can use LWLocks (and in the EXEC_BACKEND case we already
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
new file mode 100644
index a62d64e..411f068
*** a/src/backend/utils/misc/guc.c
--- b/src/backend/utils/misc/guc.c
***************
*** 47,52 ****
--- 47,53 ----
#include "commands/vacuum.h"
#include "commands/variable.h"
#include "common/string.h"
+ #include "crypto/kmgr.h"
#include "funcapi.h"
#include "jit/jit.h"
#include "libpq/auth.h"
*************** const char *const config_group_names[] =
*** 745,750 ****
--- 746,753 ----
gettext_noop("Statistics / Monitoring"),
/* STATS_COLLECTOR */
gettext_noop("Statistics / Query and Index Statistics Collector"),
+ /* ENCRYPTION */
+ gettext_noop("Encryption"),
/* AUTOVACUUM */
gettext_noop("Autovacuum"),
/* CLIENT_CONN */
*************** static struct config_bool ConfigureNames
*** 2036,2041 ****
--- 2039,2055 ----
NULL, NULL, NULL
},
+ {
+ {"key_management", PGC_INTERNAL, PRESET_OPTIONS,
+ gettext_noop("Show whether key management is enabled for this cluster."),
+ NULL,
+ GUC_NOT_IN_SAMPLE | GUC_DISALLOW_IN_FILE
+ },
+ &key_management_enabled,
+ false,
+ NULL, NULL, NULL
+ },
+
/* End-of-list marker */
{
{NULL, 0, 0, NULL, NULL}, NULL, false, NULL, NULL, NULL
*************** static struct config_string ConfigureNam
*** 4392,4397 ****
--- 4406,4421 ----
"",
NULL, NULL, NULL
},
+
+ {
+ {"cluster_passphrase_command", PGC_SIGHUP, ENCRYPTION,
+ gettext_noop("Command to obtain passphrase for database encryption."),
+ NULL
+ },
+ &cluster_passphrase_command,
+ "",
+ NULL, NULL, NULL
+ },
{
{"application_name", PGC_USERSET, LOGGING_WHAT,
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
new file mode 100644
index 9cb571f..fc30641
*** a/src/backend/utils/misc/postgresql.conf.sample
--- b/src/backend/utils/misc/postgresql.conf.sample
***************
*** 633,638 ****
--- 633,643 ----
# autovacuum, -1 means use
# vacuum_cost_limit
+ #------------------------------------------------------------------------------
+ # ENCRYPTION
+ #------------------------------------------------------------------------------
+
+ #cluster_passphrase_command = ''
#------------------------------------------------------------------------------
# CLIENT CONNECTION DEFAULTS
diff --git a/src/bin/initdb/initdb.c b/src/bin/initdb/initdb.c
new file mode 100644
index ee3bfa8..d8488b7
*** a/src/bin/initdb/initdb.c
--- b/src/bin/initdb/initdb.c
*************** static bool debug = false;
*** 141,151 ****
--- 141,154 ----
static bool noclean = false;
static bool do_sync = true;
static bool sync_only = false;
+ static bool pass_terminal_fd = false;
+ static char *term_fd_opt = NULL;
static bool show_setting = false;
static bool data_checksums = false;
static char *xlog_dir = NULL;
static char *str_wal_segment_size_mb = NULL;
static int wal_segment_size_mb;
+ static char *cluster_passphrase_cmd = NULL;
/* internal vars */
*************** static const char *const subdirs[] = {
*** 203,208 ****
--- 206,212 ----
"global",
"pg_wal/archive_status",
"pg_commit_ts",
+ "pg_cryptokeys",
"pg_dynshmem",
"pg_notify",
"pg_serial",
*************** test_config_settings(void)
*** 954,965 ****
test_buffs = MIN_BUFS_FOR_CONNS(test_conns);
snprintf(cmd, sizeof(cmd),
! "\"%s\" --boot -x0 %s "
"-c max_connections=%d "
"-c shared_buffers=%d "
"-c dynamic_shared_memory_type=%s "
"< \"%s\" > \"%s\" 2>&1",
backend_exec, boot_options,
test_conns, test_buffs,
dynamic_shared_memory_type,
DEVNULL, DEVNULL);
--- 958,970 ----
test_buffs = MIN_BUFS_FOR_CONNS(test_conns);
snprintf(cmd, sizeof(cmd),
! "\"%s\" --boot -x0 %s %s "
"-c max_connections=%d "
"-c shared_buffers=%d "
"-c dynamic_shared_memory_type=%s "
"< \"%s\" > \"%s\" 2>&1",
backend_exec, boot_options,
+ term_fd_opt ? term_fd_opt : "",
test_conns, test_buffs,
dynamic_shared_memory_type,
DEVNULL, DEVNULL);
*************** test_config_settings(void)
*** 990,1001 ****
}
snprintf(cmd, sizeof(cmd),
! "\"%s\" --boot -x0 %s "
"-c max_connections=%d "
"-c shared_buffers=%d "
"-c dynamic_shared_memory_type=%s "
"< \"%s\" > \"%s\" 2>&1",
backend_exec, boot_options,
n_connections, test_buffs,
dynamic_shared_memory_type,
DEVNULL, DEVNULL);
--- 995,1007 ----
}
snprintf(cmd, sizeof(cmd),
! "\"%s\" --boot -x0 %s %s "
"-c max_connections=%d "
"-c shared_buffers=%d "
"-c dynamic_shared_memory_type=%s "
"< \"%s\" > \"%s\" 2>&1",
backend_exec, boot_options,
+ term_fd_opt ? term_fd_opt : "",
n_connections, test_buffs,
dynamic_shared_memory_type,
DEVNULL, DEVNULL);
*************** setup_config(void)
*** 1185,1190 ****
--- 1191,1203 ----
"password_encryption = md5");
}
+ if (cluster_passphrase_cmd)
+ {
+ snprintf(repltok, sizeof(repltok), "cluster_passphrase_command = '%s'",
+ escape_quotes(cluster_passphrase_cmd));
+ conflines = replace_token(conflines, "#cluster_passphrase_command = ''", repltok);
+ }
+
/*
* If group access has been enabled for the cluster then it makes sense to
* ensure that the log files also allow group access. Otherwise a backup
*************** bootstrap_template1(void)
*** 1395,1407 ****
unsetenv("PGCLIENTENCODING");
snprintf(cmd, sizeof(cmd),
! "\"%s\" --boot -x1 -X %u %s %s %s",
backend_exec,
wal_segment_size_mb * (1024 * 1024),
data_checksums ? "-k" : "",
boot_options,
! debug ? "-d 5" : "");
!
PG_CMD_OPEN;
--- 1408,1421 ----
unsetenv("PGCLIENTENCODING");
snprintf(cmd, sizeof(cmd),
! "\"%s\" --boot -x1 -X %u %s %s %s %s %s",
backend_exec,
wal_segment_size_mb * (1024 * 1024),
data_checksums ? "-k" : "",
+ cluster_passphrase_cmd ? "-e" : "",
boot_options,
! debug ? "-d 5" : "",
! term_fd_opt ? term_fd_opt : "");
PG_CMD_OPEN;
*************** usage(const char *progname)
*** 2290,2295 ****
--- 2304,2311 ----
printf(_(" --wal-segsize=SIZE size of WAL segments, in megabytes\n"));
printf(_("\nLess commonly used options:\n"));
printf(_(" -d, --debug generate lots of debugging output\n"));
+ printf(_(" -c --cluster-passphrase-command=COMMAND\n"
+ " set command to obtain passphrase for key management\n"));
printf(_(" -k, --data-checksums use data page checksums\n"));
printf(_(" -L DIRECTORY where to find the input files\n"));
printf(_(" -n, --no-clean do not clean up after errors\n"));
*************** void
*** 2816,2822 ****
initialize_data_directory(void)
{
PG_CMD_DECL;
! int i;
setup_signals();
--- 2832,2838 ----
initialize_data_directory(void)
{
PG_CMD_DECL;
! int i, terminal_fd;
setup_signals();
*************** initialize_data_directory(void)
*** 2860,2865 ****
--- 2876,2897 ----
/* Top level PG_VERSION is checked by bootstrapper, so make it first */
write_version_file(NULL);
+ if (pass_terminal_fd)
+ {
+ #ifndef WIN32
+ terminal_fd = open("/dev/tty", O_RDWR, 0);
+ #else
+ terminal_fd = open("CONOUT$", O_RDWR, 0);
+ #endif
+ if (terminal_fd < 0)
+ {
+ pg_log_error(_("%s: could not open terminal: %s\n"),
+ progname, strerror(errno));
+ exit(1);
+ }
+ term_fd_opt = psprintf(" -R %d", terminal_fd);
+ }
+
/* Select suitable configuration settings */
set_null_conf();
test_config_settings();
*************** initialize_data_directory(void)
*** 2883,2890 ****
fflush(stdout);
snprintf(cmd, sizeof(cmd),
! "\"%s\" %s template1 >%s",
backend_exec, backend_options,
DEVNULL);
PG_CMD_OPEN;
--- 2915,2923 ----
fflush(stdout);
snprintf(cmd, sizeof(cmd),
! "\"%s\" %s %s template1 >%s",
backend_exec, backend_options,
+ term_fd_opt ? term_fd_opt : "",
DEVNULL);
PG_CMD_OPEN;
*************** main(int argc, char *argv[])
*** 2953,2963 ****
--- 2986,2998 ----
{"no-clean", no_argument, NULL, 'n'},
{"nosync", no_argument, NULL, 'N'}, /* for backwards compatibility */
{"no-sync", no_argument, NULL, 'N'},
+ {"read-terminal", no_argument, NULL, 'R'},
{"sync-only", no_argument, NULL, 'S'},
{"waldir", required_argument, NULL, 'X'},
{"wal-segsize", required_argument, NULL, 12},
{"data-checksums", no_argument, NULL, 'k'},
{"allow-group-access", no_argument, NULL, 'g'},
+ {"cluster-passphrase-command", required_argument, NULL, 'c'},
{NULL, 0, NULL, 0}
};
*************** main(int argc, char *argv[])
*** 2999,3005 ****
/* process command-line options */
! while ((c = getopt_long(argc, argv, "dD:E:kL:nNU:WA:sST:X:g", long_options, &option_index)) != -1)
{
switch (c)
{
--- 3034,3040 ----
/* process command-line options */
! while ((c = getopt_long(argc, argv, "A:c:dD:E:kL:nNRsST:U:WX:g", long_options, &option_index)) != -1)
{
switch (c)
{
*************** main(int argc, char *argv[])
*** 3045,3050 ****
--- 3080,3088 ----
case 'N':
do_sync = false;
break;
+ case 'R':
+ pass_terminal_fd = true;
+ break;
case 'S':
sync_only = true;
break;
*************** main(int argc, char *argv[])
*** 3081,3086 ****
--- 3119,3127 ----
case 9:
pwfilename = pg_strdup(optarg);
break;
+ case 'c':
+ cluster_passphrase_cmd = pg_strdup(optarg);
+ break;
case 's':
show_setting = true;
break;
*************** main(int argc, char *argv[])
*** 3151,3156 ****
--- 3192,3205 ----
exit(1);
}
+ #ifndef USE_OPENSSL
+ if (cluster_passphrase_cmd)
+ {
+ pg_log_error("cluster encryption is not supported because OpenSSL is not supported by this build");
+ exit(1);
+ }
+ #endif
+
check_authmethod_unspecified(&authmethodlocal);
check_authmethod_unspecified(&authmethodhost);
*************** main(int argc, char *argv[])
*** 3218,3223 ****
--- 3267,3277 ----
else
printf(_("Data page checksums are disabled.\n"));
+ if (cluster_passphrase_cmd)
+ printf(_("Key management system is enabled.\n"));
+ else
+ printf(_("Key management system is disabled.\n"));
+
if (pwprompt || pwfilename)
get_su_pwd();
diff --git a/src/bin/pg_controldata/pg_controldata.c b/src/bin/pg_controldata/pg_controldata.c
new file mode 100644
index 3e00ac0..39ba7f0
*** a/src/bin/pg_controldata/pg_controldata.c
--- b/src/bin/pg_controldata/pg_controldata.c
***************
*** 25,30 ****
--- 25,31 ----
#include "access/xlog_internal.h"
#include "catalog/pg_control.h"
#include "common/controldata_utils.h"
+ #include "common/kmgr_utils.h"
#include "common/logging.h"
#include "getopt_long.h"
#include "pg_getopt.h"
*************** main(int argc, char *argv[])
*** 334,338 ****
--- 335,341 ----
ControlFile->data_checksum_version);
printf(_("Mock authentication nonce: %s\n"),
mock_auth_nonce_str);
+ printf(_("Key management version: %u\n"),
+ ControlFile->key_management_version);
return 0;
}
diff --git a/src/bin/pg_ctl/pg_ctl.c b/src/bin/pg_ctl/pg_ctl.c
new file mode 100644
index fc07f1a..a7a4e68
*** a/src/bin/pg_ctl/pg_ctl.c
--- b/src/bin/pg_ctl/pg_ctl.c
*************** typedef enum
*** 79,84 ****
--- 79,85 ----
static bool do_wait = true;
static int wait_seconds = DEFAULT_WAIT;
static bool wait_seconds_arg = false;
+ static bool pass_terminal_fd = false;
static bool silent_mode = false;
static ShutdownMode shutdown_mode = FAST_MODE;
static int sig = SIGINT; /* default */
*************** free_readfile(char **optlines)
*** 442,448 ****
static pgpid_t
start_postmaster(void)
{
! char cmd[MAXPGPATH];
#ifndef WIN32
pgpid_t pm_pid;
--- 443,450 ----
static pgpid_t
start_postmaster(void)
{
! char cmd[MAXPGPATH], *term_fd_opt = NULL;
! int terminal_fd;
#ifndef WIN32
pgpid_t pm_pid;
*************** start_postmaster(void)
*** 467,472 ****
--- 469,486 ----
/* fork succeeded, in child */
+ if (pass_terminal_fd)
+ {
+ terminal_fd = open("/dev/tty", O_RDWR, 0);
+ if (terminal_fd < 0)
+ {
+ write_stderr(_("%s: could not open terminal: %s\n"),
+ progname, strerror(errno));
+ exit(1);
+ }
+ term_fd_opt = psprintf(" -R %d", terminal_fd);
+ }
+
/*
* If possible, detach the postmaster process from the launching process
* group and make it a group leader, so that it doesn't get signaled along
*************** start_postmaster(void)
*** 487,498 ****
* has the same PID as the current child process.
*/
if (log_file != NULL)
! snprintf(cmd, MAXPGPATH, "exec \"%s\" %s%s < \"%s\" >> \"%s\" 2>&1",
exec_path, pgdata_opt, post_opts,
DEVNULL, log_file);
else
! snprintf(cmd, MAXPGPATH, "exec \"%s\" %s%s < \"%s\" 2>&1",
! exec_path, pgdata_opt, post_opts, DEVNULL);
(void) execl("/bin/sh", "/bin/sh", "-c", cmd, (char *) NULL);
--- 501,514 ----
* has the same PID as the current child process.
*/
if (log_file != NULL)
! snprintf(cmd, MAXPGPATH, "exec \"%s\" %s%s%s < \"%s\" >> \"%s\" 2>&1",
exec_path, pgdata_opt, post_opts,
+ term_fd_opt ? term_fd_opt : "",
DEVNULL, log_file);
else
! snprintf(cmd, MAXPGPATH, "exec \"%s\" %s%s%s < \"%s\" 2>&1",
! exec_path, pgdata_opt, post_opts,
! term_fd_opt ? term_fd_opt : "", DEVNULL);
(void) execl("/bin/sh", "/bin/sh", "-c", cmd, (char *) NULL);
*************** start_postmaster(void)
*** 513,518 ****
--- 529,548 ----
PROCESS_INFORMATION pi;
const char *comspec;
+ if (pass_terminal_fd)
+ {
+ /* Hopefully we can read and write CONOUT, see simple_prompt() */
+ /* Do CreateRestrictedProcess() children even inherit open file descriptors? */
+ terminal_fd = open("CONOUT$", O_RDWR, 0);
+ if (terminal_fd < 0)
+ {
+ write_stderr(_("%s: could not open terminal: %s\n"),
+ progname, strerror(errno));
+ exit(1);
+ }
+ term_fd_opt = psprintf(" -R %d", terminal_fd);
+ }
+
/* Find CMD.EXE location using COMSPEC, if it's set */
comspec = getenv("COMSPEC");
if (comspec == NULL)
*************** start_postmaster(void)
*** 553,564 ****
else
close(fd);
! snprintf(cmd, MAXPGPATH, "\"%s\" /C \"\"%s\" %s%s < \"%s\" >> \"%s\" 2>&1\"",
! comspec, exec_path, pgdata_opt, post_opts, DEVNULL, log_file);
}
else
! snprintf(cmd, MAXPGPATH, "\"%s\" /C \"\"%s\" %s%s < \"%s\" 2>&1\"",
! comspec, exec_path, pgdata_opt, post_opts, DEVNULL);
if (!CreateRestrictedProcess(cmd, &pi, false))
{
--- 583,596 ----
else
close(fd);
! snprintf(cmd, MAXPGPATH, "\"%s\" /C \"\"%s\" %s%s%s < \"%s\" >> \"%s\" 2>&1\"",
! comspec, exec_path, pgdata_opt, post_opts,
! term_fd_opt ? term_fd_opt : "", DEVNULL, log_file);
}
else
! snprintf(cmd, MAXPGPATH, "\"%s\" /C \"\"%s\" %s%s%s < \"%s\" 2>&1\"",
! comspec, exec_path, pgdata_opt, post_opts,
! term_fd_opt ? term_fd_opt : "", DEVNULL);
if (!CreateRestrictedProcess(cmd, &pi, false))
{
*************** wait_for_postmaster(pgpid_t pm_pid, bool
*** 689,695 ****
}
else
#endif
! print_msg(".");
}
pg_usleep(USEC_PER_SEC / WAITS_PER_SEC);
--- 721,728 ----
}
else
#endif
! if (!pass_terminal_fd)
! print_msg(".");
}
pg_usleep(USEC_PER_SEC / WAITS_PER_SEC);
*************** main(int argc, char **argv)
*** 2260,2265 ****
--- 2293,2299 ----
{"mode", required_argument, NULL, 'm'},
{"pgdata", required_argument, NULL, 'D'},
{"options", required_argument, NULL, 'o'},
+ {"read-terminal", no_argument, NULL, 'R'},
{"silent", no_argument, NULL, 's'},
{"timeout", required_argument, NULL, 't'},
{"core-files", no_argument, NULL, 'c'},
*************** main(int argc, char **argv)
*** 2332,2338 ****
/* process command-line options */
while (optind < argc)
{
! while ((c = getopt_long(argc, argv, "cD:e:l:m:N:o:p:P:sS:t:U:wW",
long_options, &option_index)) != -1)
{
switch (c)
--- 2366,2372 ----
/* process command-line options */
while (optind < argc)
{
! while ((c = getopt_long(argc, argv, "cD:e:l:m:N:o:p:P:RsS:t:U:wW",
long_options, &option_index)) != -1)
{
switch (c)
*************** main(int argc, char **argv)
*** 2385,2390 ****
--- 2419,2427 ----
case 'P':
register_password = pg_strdup(optarg);
break;
+ case 'R':
+ pass_terminal_fd = true;
+ break;
case 's':
silent_mode = true;
break;
diff --git a/src/bin/pg_resetwal/pg_resetwal.c b/src/bin/pg_resetwal/pg_resetwal.c
new file mode 100644
index cb6ef19..a9d200e
*** a/src/bin/pg_resetwal/pg_resetwal.c
--- b/src/bin/pg_resetwal/pg_resetwal.c
*************** PrintControlValues(bool guessed)
*** 804,809 ****
--- 804,811 ----
(ControlFile.float8ByVal ? _("by value") : _("by reference")));
printf(_("Data page checksum version: %u\n"),
ControlFile.data_checksum_version);
+ printf(_("Key management version: %u\n"),
+ ControlFile.key_management_version);
}
diff --git a/src/bin/pg_rewind/filemap.c b/src/bin/pg_rewind/filemap.c
new file mode 100644
index 1abc257..3084ae7
*** a/src/bin/pg_rewind/filemap.c
--- b/src/bin/pg_rewind/filemap.c
*************** static const char *excludeDirContents[]
*** 74,79 ****
--- 74,87 ----
"pg_notify",
/*
+ * Skip cryptographic keys. It's generally not good idea to copy the
+ * cryptographic keys from source database because these might use
+ * different cluster passphrase.
+ */
+ "pg_cryptokeys", /* defined as KMGR_DIR */
+ "pg_cryptokeys_tmp", /* defined as KMGR_TMP_DIR */
+
+ /*
* Old contents are loaded for possible debugging but are not required for
* normal operation, see SerialInit().
*/
diff --git a/src/bin/pg_upgrade/controldata.c b/src/bin/pg_upgrade/controldata.c
new file mode 100644
index 39bcaa8..e5794ff
*** a/src/bin/pg_upgrade/controldata.c
--- b/src/bin/pg_upgrade/controldata.c
***************
*** 9,18 ****
--- 9,24 ----
#include "postgres_fe.h"
+ #include <dirent.h>
#include <ctype.h>
#include "pg_upgrade.h"
+ #include "access/xlog_internal.h"
+ #include "common/controldata_utils.h"
+ #include "common/file_utils.h"
+ #include "common/kmgr_utils.h"
+
/*
* get_control_data()
*
*************** get_control_data(ClusterInfo *cluster, b
*** 59,64 ****
--- 65,71 ----
bool got_date_is_int = false;
bool got_data_checksum_version = false;
bool got_cluster_state = false;
+ bool got_key_management_enabled = false;
char *lc_collate = NULL;
char *lc_ctype = NULL;
char *lc_monetary = NULL;
*************** get_control_data(ClusterInfo *cluster, b
*** 202,207 ****
--- 209,221 ----
got_data_checksum_version = true;
}
+ /* Only in <= 14 */
+ if (GET_MAJOR_VERSION(cluster->major_version) <= 1400)
+ {
+ cluster->controldata.key_management_enabled = false;
+ got_key_management_enabled = true;
+ }
+
/* we have the result of cmd in "output". so parse it line by line now */
while (fgets(bufin, sizeof(bufin), output))
{
*************** get_control_data(ClusterInfo *cluster, b
*** 485,490 ****
--- 499,516 ----
cluster->controldata.data_checksum_version = str2uint(p);
got_data_checksum_version = true;
}
+ else if ((p = strstr(bufin, "Key management:")) != NULL)
+ {
+ p = strchr(p, ':');
+
+ if (p == NULL || strlen(p) <= 1)
+ pg_fatal("%d: controldata retrieval problem\n", __LINE__);
+
+ p++; /* remove ':' char */
+ /* used later for contrib check */
+ cluster->controldata.key_management_enabled = strstr(p, "on") != NULL;
+ got_key_management_enabled = true;
+ }
}
pclose(output);
*************** get_control_data(ClusterInfo *cluster, b
*** 539,545 ****
!got_index || !got_toast ||
(!got_large_object &&
cluster->controldata.ctrl_ver >= LARGE_OBJECT_SIZE_PG_CONTROL_VER) ||
! !got_date_is_int || !got_data_checksum_version)
{
if (cluster == &old_cluster)
pg_log(PG_REPORT,
--- 565,572 ----
!got_index || !got_toast ||
(!got_large_object &&
cluster->controldata.ctrl_ver >= LARGE_OBJECT_SIZE_PG_CONTROL_VER) ||
! !got_date_is_int || !got_data_checksum_version ||
! !got_key_management_enabled)
{
if (cluster == &old_cluster)
pg_log(PG_REPORT,
*************** get_control_data(ClusterInfo *cluster, b
*** 605,610 ****
--- 632,641 ----
if (!got_data_checksum_version)
pg_log(PG_REPORT, " data checksum version\n");
+ /* value added in Postgres 12 */
+ if (!got_key_management_enabled)
+ pg_log(PG_REPORT, " key management enabled\n");
+
pg_fatal("Cannot continue without required control information, terminating\n");
}
}
*************** check_control_data(ControlData *oldctrl,
*** 669,674 ****
--- 700,713 ----
pg_fatal("old cluster uses data checksums but the new one does not\n");
else if (oldctrl->data_checksum_version != newctrl->data_checksum_version)
pg_fatal("old and new cluster pg_controldata checksum versions do not match\n");
+
+ /*
+ * We cannot upgrade if the old cluster enables the key management but
+ * the new one doesn't support because the old one might already have
+ * data encrypted by the master encryption key.
+ */
+ if (oldctrl->key_management_enabled && !newctrl->key_management_enabled)
+ pg_fatal("old cluster uses key management but the new one does not\n");
}
diff --git a/src/bin/pg_upgrade/file.c b/src/bin/pg_upgrade/file.c
new file mode 100644
index cc8a675..282359f
*** a/src/bin/pg_upgrade/file.c
--- b/src/bin/pg_upgrade/file.c
***************
*** 11,16 ****
--- 11,17 ----
#include <sys/stat.h>
#include <fcntl.h>
+ #include <dirent.h>
#ifdef HAVE_COPYFILE_H
#include <copyfile.h>
#endif
***************
*** 21,26 ****
--- 22,28 ----
#include "access/visibilitymap.h"
#include "common/file_perm.h"
+ #include "common/file_utils.h"
#include "pg_upgrade.h"
#include "storage/bufpage.h"
#include "storage/checksum.h"
*************** check_hard_link(void)
*** 372,374 ****
--- 374,451 ----
unlink(new_link_file);
}
+
+ /*
+ * Copy cryptographic keys from the old cluster to the new cluster.
+ */
+ void
+ copy_master_encryption_key(ClusterInfo *old_cluster, ClusterInfo * new_cluster)
+ {
+ DIR *dir;
+ struct dirent *de;
+ char path[MAXPGPATH];
+
+ /* We copy the crypto keys only if both clusters enable the key management */
+ if (!old_cluster->controldata.key_management_enabled ||
+ !new_cluster->controldata.key_management_enabled)
+ return;
+
+ prep_status("Copying master encryption key");
+
+ snprintf(path, MAXPGPATH, "%s/%s", old_cluster->pgdata, KMGR_DIR);
+
+ if ((dir = opendir(path)) == NULL)
+ pg_fatal("could not open directory \"%s\": %m", path);
+
+ while ((de = readdir(dir)) != NULL)
+ {
+ if (strlen(de->d_name) == 4 &&
+ strspn(de->d_name, "0123456789ABCDEF") == 4)
+ {
+ CryptoKey key;
+ char src_path[MAXPGPATH];
+ char dst_path[MAXPGPATH];
+ uint32 id;
+ int src_fd;
+ int dst_fd;
+ int len;
+
+ id = strtoul(de->d_name, NULL, 16);
+
+ snprintf(src_path, MAXPGPATH, "%s/%s/%04X",
+ old_cluster->pgdata, KMGR_DIR, id);
+ snprintf(dst_path, MAXPGPATH, "%s/%s/%04X",
+ new_cluster->pgdata, KMGR_DIR, id);
+
+ if ((src_fd = open(src_path, O_RDONLY | PG_BINARY, 0)) < 0)
+ pg_fatal("could not open file \"%s\": %m", src_path);
+
+ if ((dst_fd = open(dst_path, O_RDWR | O_CREAT | O_TRUNC | PG_BINARY,
+ pg_file_create_mode)) < 0)
+ pg_fatal("could not open file \"%s\": %m", dst_path);
+
+ /* Read the source key */
+ len = read(src_fd, &key, sizeof(CryptoKey));
+ if (len != sizeof(CryptoKey))
+ {
+ if (len < 0)
+ pg_fatal("could not read file \"%s\": %m", src_path);
+ else
+ pg_fatal("could not read file \"%s\": read %d of %zu",
+ src_path, len, sizeof(CryptoKey));
+ }
+
+ /* Write to the dest key */
+ len = write(dst_fd, &key, sizeof(CryptoKey));
+ if (len != sizeof(CryptoKey))
+ pg_fatal("could not write fie \"%s\"", dst_path);
+
+ close(src_fd);
+ close(dst_fd);
+ }
+ }
+
+ closedir(dir);
+
+ check_ok();
+ }
diff --git a/src/bin/pg_upgrade/pg_upgrade.c b/src/bin/pg_upgrade/pg_upgrade.c
new file mode 100644
index 1bc86e4..d1bacdf
*** a/src/bin/pg_upgrade/pg_upgrade.c
--- b/src/bin/pg_upgrade/pg_upgrade.c
*************** main(int argc, char **argv)
*** 158,163 ****
--- 158,170 ----
old_cluster.pgdata, new_cluster.pgdata);
/*
+ * Copy the internal encryption keys from the old cluster to the new one.
+ * This is necessary because the data in the old cluster might be
+ * encrypted with the old master encryption key.
+ */
+ copy_master_encryption_key(&old_cluster, &new_cluster);
+
+ /*
* Assuming OIDs are only used in system tables, there is no need to
* restore the OID counter because we have not transferred any OIDs from
* the old system, but we do it anyway just in case. We do it late here
diff --git a/src/bin/pg_upgrade/pg_upgrade.h b/src/bin/pg_upgrade/pg_upgrade.h
new file mode 100644
index 8b90cef..32ab236
*** a/src/bin/pg_upgrade/pg_upgrade.h
--- b/src/bin/pg_upgrade/pg_upgrade.h
***************
*** 11,16 ****
--- 11,17 ----
#include <sys/time.h>
#include "libpq-fe.h"
+ #include "common/kmgr_utils.h"
/* Use port in the private/dynamic port number range */
#define DEF_PGUPORT 50432
*************** typedef struct
*** 219,224 ****
--- 220,226 ----
bool date_is_int;
bool float8_pass_by_value;
bool data_checksum_version;
+ bool key_management_enabled;
} ControlData;
/*
*************** void rewriteVisibilityMap(const char *f
*** 375,380 ****
--- 377,384 ----
const char *schemaName, const char *relName);
void check_file_clone(void);
void check_hard_link(void);
+ void copy_master_encryption_key(ClusterInfo *old_cluster,
+ ClusterInfo * new_cluster);
/* fopen_priv() is no longer different from fopen() */
#define fopen_priv(path, mode) fopen(path, mode)
diff --git a/src/common/Makefile b/src/common/Makefile
new file mode 100644
index 25c55bd..cfb34fd
*** a/src/common/Makefile
--- b/src/common/Makefile
*************** OBJS_COMMON = \
*** 49,54 ****
--- 49,55 ----
archive.o \
base64.o \
checksum_helper.o \
+ cipher.o \
config_info.o \
controldata_utils.o \
d2s.o \
*************** OBJS_COMMON = \
*** 61,66 ****
--- 62,68 ----
ip.o \
jsonapi.o \
keywords.o \
+ kmgr_utils.o \
kwlookup.o \
link-canary.o \
md5.o \
*************** OBJS_COMMON = \
*** 81,86 ****
--- 83,89 ----
ifeq ($(with_openssl),yes)
OBJS_COMMON += \
+ cipher_openssl.o \
protocol_openssl.o \
sha2_openssl.o
else
diff --git a/src/common/cipher.c b/src/common/cipher.c
new file mode 100644
index ...908afe8
*** a/src/common/cipher.c
--- b/src/common/cipher.c
***************
*** 0 ****
--- 1,87 ----
+ /*-------------------------------------------------------------------------
+ *
+ * cipher.c
+ * Shared frontend/backend for cryptographic functions
+ *
+ * Copyright (c) 2020, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ * src/common/cipher.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+ #ifndef FRONTEND
+ #include "postgres.h"
+ #else
+ #include "postgres_fe.h"
+ #endif
+
+ #include "common/cipher.h"
+ #ifdef USE_OPENSSL
+ #include "common/cipher_openssl.h"
+ #endif
+
+ /*
+ * Return a newly created cipher context. 'cipher' specifies cipher algorithm
+ * by identifer like PG_CIPHER_XXX.
+ */
+ PgCipherCtx *
+ pg_cipher_ctx_create(int cipher, uint8 *key, int klen)
+ {
+ PgCipherCtx *ctx = NULL;
+
+ if (cipher >= PG_MAX_CIPHER_ID)
+ return NULL;
+
+ #ifdef USE_OPENSSL
+ ctx = (PgCipherCtx *) palloc0(sizeof(PgCipherCtx));
+
+ ctx->encctx = ossl_cipher_ctx_create(cipher, key, klen, true);
+ ctx->decctx = ossl_cipher_ctx_create(cipher, key, klen, false);
+ #endif
+
+ return ctx;
+ }
+
+ void
+ pg_cipher_ctx_free(PgCipherCtx *ctx)
+ {
+ #ifdef USE_OPENSSL
+ ossl_cipher_ctx_free(ctx->encctx);
+ ossl_cipher_ctx_free(ctx->decctx);
+ #endif
+ }
+
+ bool
+ pg_cipher_encrypt(PgCipherCtx *ctx, const uint8 *in, int inlen,
+ uint8 *out, int *outlen, const uint8 *iv)
+ {
+ bool r = false;
+ #ifdef USE_OPENSSL
+ r = ossl_cipher_encrypt(ctx->encctx, in, inlen, out, outlen, iv);
+ #endif
+ return r;
+ }
+
+ bool
+ pg_cipher_decrypt(PgCipherCtx *ctx, const uint8 *in, int inlen,
+ uint8 *out, int *outlen, const uint8 *iv)
+ {
+ bool r = false;
+ #ifdef USE_OPENSSL
+ r = ossl_cipher_decrypt(ctx->decctx, in, inlen, out, outlen, iv);
+ #endif
+ return r;
+ }
+
+ bool
+ pg_HMAC_SHA512(const uint8 *key, const uint8 *in, int inlen,
+ uint8 *out)
+ {
+ bool r = false;
+ #ifdef USE_OPENSSL
+ r = ossl_HMAC_SHA512(key, in, inlen, out);
+ #endif
+ return r;
+ }
diff --git a/src/common/cipher_openssl.c b/src/common/cipher_openssl.c
new file mode 100644
index ...e8bc241
*** a/src/common/cipher_openssl.c
--- b/src/common/cipher_openssl.c
***************
*** 0 ****
--- 1,181 ----
+ /*-------------------------------------------------------------------------
+ * cipher_openssl.c
+ * Cryptographic function using OpenSSL
+ *
+ * This contains the common low-level functions needed in both frontend and
+ * backend, for implement the database encryption.
+ *
+ * Portions Copyright (c) 2020, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ * src/common/cipher_openssl.c
+ *
+ *-------------------------------------------------------------------------
+ */
+ #ifndef FRONTEND
+ #include "postgres.h"
+ #else
+ #include "postgres_fe.h"
+ #endif
+
+ #include "common/sha2.h"
+ #include "common/cipher_openssl.h"
+
+ #include <openssl/ssl.h>
+ #include <openssl/conf.h>
+ #include <openssl/evp.h>
+ #include <openssl/err.h>
+ #include <openssl/hmac.h>
+
+ /*
+ * prototype for the EVP functions that return an algorithm, e.g.
+ * EVP_aes_128_cbc().
+ */
+ typedef const EVP_CIPHER *(*ossl_EVP_cipher_func) (void);
+
+ static bool ossl_initialized = false;
+
+ static ossl_EVP_cipher_func get_evp_aes_cbc(int klen);
+
+ static ossl_EVP_cipher_func
+ get_evp_aes_cbc(int klen)
+ {
+ switch (klen)
+ {
+ case PG_AES128_KEY_LEN:
+ return EVP_aes_128_cbc;
+ case PG_AES192_KEY_LEN:
+ return EVP_aes_192_cbc;
+ case PG_AES256_KEY_LEN:
+ return EVP_aes_256_cbc;
+ default:
+ return NULL;
+ }
+ }
+
+ /*
+ * Initialize and return an EVP_CIPHER_CTX. Return NULL if the given
+ * cipher algorithm is not supported or on failure..
+ */
+ EVP_CIPHER_CTX *
+ ossl_cipher_ctx_create(int cipher, uint8 *key, int klen, bool enc)
+ {
+ EVP_CIPHER_CTX *ctx;
+ ossl_EVP_cipher_func func;
+ int ret;
+
+ if (!ossl_initialized)
+ {
+ #ifdef HAVE_OPENSSL_INIT_SSL
+ OPENSSL_init_ssl(OPENSSL_INIT_LOAD_CONFIG, NULL);
+ #else
+ OPENSSL_config(NULL);
+ SSL_library_init();
+ SSL_load_error_strings();
+ #endif
+ ossl_initialized = true;
+ }
+
+ ctx = EVP_CIPHER_CTX_new();
+
+ switch (cipher)
+ {
+ case PG_CIPHER_AES_CBC:
+ func = get_evp_aes_cbc(klen);
+ if (!func)
+ goto failed;
+ break;
+ default:
+ goto failed;
+ }
+
+
+ if (enc)
+ ret = EVP_EncryptInit_ex(ctx, (const EVP_CIPHER *) func(), NULL, key, NULL);
+ else
+ ret = EVP_DecryptInit_ex(ctx, (const EVP_CIPHER *) func(), NULL, key, NULL);
+
+ if (!ret)
+ goto failed;
+
+ if (!EVP_CIPHER_CTX_set_key_length(ctx, PG_AES256_KEY_LEN))
+ goto failed;
+
+ /*
+ * Always enable padding. We don't need to check the return value as
+ * EVP_CIPHER_CTX_set_padding always returns 1.
+ */
+ EVP_CIPHER_CTX_set_padding(ctx, 1);
+
+ return ctx;
+
+ failed:
+ EVP_CIPHER_CTX_free(ctx);
+ return NULL;
+ }
+
+ void
+ ossl_cipher_ctx_free(EVP_CIPHER_CTX *ctx)
+ {
+ return EVP_CIPHER_CTX_free(ctx);
+ }
+
+ bool
+ ossl_cipher_encrypt(EVP_CIPHER_CTX *ctx,
+ const uint8 *in, int inlen,
+ uint8 *out, int *outlen,
+ const uint8 *iv)
+ {
+ int len;
+ int enclen;
+
+ if (!EVP_EncryptInit_ex(ctx, NULL, NULL, NULL, iv))
+ return false;
+
+ if (!EVP_EncryptUpdate(ctx, out, &len, in, inlen))
+ return false;
+
+ enclen = len;
+
+ if (!EVP_EncryptFinal_ex(ctx, (uint8 *) ((char *) out + enclen),
+ &len))
+ return false;
+
+ *outlen = enclen + len;
+
+ return true;
+ }
+
+ bool
+ ossl_cipher_decrypt(EVP_CIPHER_CTX *ctx,
+ const uint8 *in, int inlen,
+ uint8 *out, int *outlen,
+ const uint8 *iv)
+ {
+ int declen;
+ int len;
+
+ if (!EVP_DecryptInit_ex(ctx, NULL, NULL, NULL, iv))
+ return false;
+
+ if (!EVP_DecryptUpdate(ctx, out, &len, in, inlen))
+ return false;
+
+ declen = len;
+
+ if (!EVP_DecryptFinal_ex(ctx, (uint8 *) ((char *) out + declen),
+ &len))
+ return false;
+
+ *outlen = declen + len;
+
+ return true;
+ }
+
+ bool
+ ossl_HMAC_SHA512(const uint8 *key, const uint8 *in, int inlen,
+ uint8 *out)
+ {
+ return HMAC(EVP_sha512(), key, PG_SHA512_DIGEST_LENGTH,
+ in, (uint32) inlen, out, NULL);
+ }
diff --git a/src/common/kmgr_utils.c b/src/common/kmgr_utils.c
new file mode 100644
index ...7bb4c5e
*** a/src/common/kmgr_utils.c
--- b/src/common/kmgr_utils.c
***************
*** 0 ****
--- 1,529 ----
+ /*-------------------------------------------------------------------------
+ *
+ * kmgr_utils.c
+ * Shared frontend/backend for cryptographic key management
+ *
+ * Copyright (c) 2020, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ * src/common/kmgr_utils.c
+ *
+ *-------------------------------------------------------------------------
+ */
+
+ #ifndef FRONTEND
+ #include "postgres.h"
+ #else
+ #include "postgres_fe.h"
+ #endif
+
+ #include <unistd.h>
+ #include <sys/stat.h>
+
+ #ifdef FRONTEND
+ #include "common/logging.h"
+ #endif
+ #include "common/file_perm.h"
+ #include "common/kmgr_utils.h"
+ #include "common/sha2.h"
+ #include "crypto/kmgr.h"
+ #include "postmaster/postmaster.h"
+ #include "utils/elog.h"
+ #include "storage/fd.h"
+
+ #ifndef FRONTEND
+ #include "pgstat.h"
+ #include "storage/fd.h"
+ #endif
+
+ #define KMGR_PROMPT_MSG "Enter database encryption pass phrase: "
+
+ #ifdef FRONTEND
+ static FILE *open_pipe_stream(const char *command);
+ static int close_pipe_stream(FILE *file);
+ #endif
+
+ static void read_one_keyfile(const char *dataDir, uint32 id, CryptoKey *key_p);
+
+ /* Return a key wrap context initialized with the given keys */
+ PgKeyWrapCtx *
+ pg_create_keywrap_ctx(uint8 key[KMGR_ENC_KEY_LEN], uint8 mackey[KMGR_MAC_KEY_LEN])
+ {
+ PgKeyWrapCtx *ctx;
+
+ ctx = (PgKeyWrapCtx *) palloc0(sizeof(PgKeyWrapCtx));
+
+ /* Create and initialize a cipher context */
+ ctx->cipherctx = pg_cipher_ctx_create(PG_CIPHER_AES_CBC, key, KMGR_ENC_KEY_LEN);
+ if (ctx->cipherctx == NULL)
+ return NULL;
+
+ /* Set encryption key and MAC key */
+ memcpy(ctx->key, key, KMGR_ENC_KEY_LEN);
+ memcpy(ctx->mackey, mackey, KMGR_MAC_KEY_LEN);
+
+ return ctx;
+ }
+
+ /* Free the key wrap context */
+ void
+ pg_free_keywrap_ctx(PgKeyWrapCtx *ctx)
+ {
+ if (!ctx)
+ return;
+
+ Assert(ctx->cipherctx);
+
+ pg_cipher_ctx_free(ctx->cipherctx);
+
+ #ifndef FRONTEND
+ pfree(ctx);
+ #else
+ pg_free(ctx);
+ #endif
+ }
+
+ /*
+ * Encrypt the given data. Return true and set encrypted data to 'out' if
+ * success. Otherwise return false. The caller must allocate sufficient space
+ * for cipher data calculated by using KmgrSizeOfCipherText(). Please note that
+ * this function modifies 'out' data even on failure case.
+ */
+ bool
+ kmgr_wrap_key(PgKeyWrapCtx *ctx, CryptoKey *in, CryptoKey *out)
+ {
+ uint8 *hmac;
+ uint8 *iv;
+ uint8 *enc;
+ int enclen;
+
+ Assert(ctx && in && out);
+
+ hmac = out->key;
+ iv = hmac + KMGR_HMAC_LEN;
+ enc = iv + PG_AES_IV_SIZE;
+
+ /* Generate IV */
+ if (!pg_strong_random(iv, PG_AES_IV_SIZE))
+ return false;
+
+ if (!pg_cipher_encrypt(ctx->cipherctx, in->key, in->klen, enc, &enclen, iv))
+ return false;
+
+ if (!pg_HMAC_SHA512(ctx->mackey, enc, enclen, hmac))
+ return false;
+
+ out->klen = KmgrSizeOfCipherText(in->klen);;
+ Assert(out->klen == KMGR_HMAC_LEN + PG_AES_IV_SIZE + enclen);
+
+ return true;
+ }
+
+ /*
+ * Decrypt the given Data. Return true and set plain text data to `out` if
+ * success. Otherwise return false. The caller must allocate sufficient space
+ * for cipher data calculated by using KmgrSizeOfPlainText(). Please note that
+ * this function modifies 'out' data even on failure case.
+ */
+ bool
+ kmgr_unwrap_key(PgKeyWrapCtx *ctx, CryptoKey *in, CryptoKey *out)
+ {
+ uint8 hmac[KMGR_HMAC_LEN];
+ uint8 *expected_hmac;
+ uint8 *iv;
+ uint8 *enc;
+ int enclen;
+
+ Assert(ctx && in && out);
+
+ expected_hmac = in->key;
+ iv = expected_hmac + KMGR_HMAC_LEN;
+ enc = iv + PG_AES_IV_SIZE;
+ enclen = in->klen - (enc - in->key);
+
+ /* Verify the correctness of HMAC */
+ if (!pg_HMAC_SHA512(ctx->mackey, enc, enclen, hmac))
+ return false;
+
+ if (memcmp(hmac, expected_hmac, KMGR_HMAC_LEN) != 0)
+ return false;
+
+ /* Decrypt encrypted data */
+ if (!pg_cipher_decrypt(ctx->cipherctx, enc, enclen, out->key, &(out->klen), iv))
+ return false;
+
+ return true;
+ }
+
+ /*
+ * Verify the correctness of the given passphrase by unwrapping the given keys.
+ * If the given passphrase is correct we set unwrapped keys to keys_out and return
+ * true. Otherwise return false. Please note that this function changes the
+ * contents of keys_out even on failure. Both keys_in and keys_out must be the
+ * same length, nkey.
+ */
+ bool
+ kmgr_verify_passphrase(char *passphrase, int passlen,
+ CryptoKey *keys_in, CryptoKey *keys_out, int nkeys)
+ {
+ PgKeyWrapCtx *tmpctx;
+ uint8 user_enckey[KMGR_ENC_KEY_LEN];
+ uint8 user_hmackey[KMGR_MAC_KEY_LEN];
+
+ /*
+ * Create temporary wrap context with encryption key and HMAC key extracted
+ * from the passphrase.
+ */
+ kmgr_derive_keys(passphrase, passlen, user_enckey, user_hmackey);
+ tmpctx = pg_create_keywrap_ctx(user_enckey, user_hmackey);
+
+ for (int i = 0; i < nkeys; i++)
+ {
+
+ if (!kmgr_unwrap_key(tmpctx, &(keys_in[i]), &(keys_out[i])))
+ {
+ /* The passphrase is not correct */
+ pg_free_keywrap_ctx(tmpctx);
+ return false;
+ }
+ }
+
+ /* The passphrase is correct, free the cipher context */
+ pg_free_keywrap_ctx(tmpctx);
+
+ return true;
+ }
+
+ /* Generate encryption key and mac key from given passphrase */
+ void
+ kmgr_derive_keys(char *passphrase, Size passlen,
+ uint8 enckey[KMGR_ENC_KEY_LEN],
+ uint8 mackey[KMGR_MAC_KEY_LEN])
+ {
+ pg_sha256_ctx ctx1;
+ pg_sha512_ctx ctx2;
+
+ StaticAssertStmt(KMGR_ENC_KEY_LEN == PG_AES256_KEY_LEN,
+ "derived encryption key size does not match AES256 key size");
+ StaticAssertStmt(KMGR_MAC_KEY_LEN == PG_HMAC_SHA512_KEY_LEN,
+ "derived mac key size does not match HMAC-SHA512 key size");
+
+ /* Generate encryption key from passphrase */
+ pg_sha256_init(&ctx1);
+ pg_sha256_update(&ctx1, (const uint8 *) passphrase, passlen);
+ pg_sha256_final(&ctx1, enckey);
+
+ /* Generate mac key from passphrase */
+ pg_sha512_init(&ctx2);
+ pg_sha512_update(&ctx2, (const uint8 *) passphrase, passlen);
+ pg_sha512_final(&ctx2, mackey);
+ }
+
+ /*
+ * Run cluster passphrase command.
+ *
+ * prompt will be substituted for %p, file descriptor for %R
+ *
+ * The result will be put in buffer buf, which is of size size.
+ * The return value is the length of the actual result.
+ */
+ int
+ kmgr_run_cluster_passphrase_command(char *passphrase_command, char *buf,
+ int size)
+ {
+ char command[MAXPGPATH];
+ char *p;
+ char *dp;
+ char *endp;
+ FILE *fh;
+ int pclose_rc;
+ size_t len = 0;
+
+ Assert(size > 0);
+ buf[0] = '\0';
+
+ dp = command;
+ endp = command + MAXPGPATH - 1;
+ *endp = '\0';
+
+ for (p = passphrase_command; *p; p++)
+ {
+ if (p[0] == '%')
+ {
+ switch (p[1])
+ {
+ case 'p':
+ strlcpy(dp, KMGR_PROMPT_MSG, strlen(KMGR_PROMPT_MSG)+1);
+ dp += strlen(KMGR_PROMPT_MSG);
+ p++;
+ break;
+ case 'R':
+ /* Safe from overflow? */
+ dp += sprintf(dp, "%d", terminal_fd);
+ p++;
+ break;
+ case '%':
+ p++;
+ if (dp < endp)
+ *dp++ = *p;
+ break;
+ default:
+ if (dp < endp)
+ *dp++ = *p;
+ break;
+ }
+ }
+ else
+ {
+ if (dp < endp)
+ *dp++ = *p;
+ }
+ }
+ *dp = '\0';
+
+ #ifdef FRONTEND
+ fh = open_pipe_stream(command);
+ if (fh == NULL)
+ {
+ pg_log_fatal("could not execute command \"%s\": %m",
+ command);
+ exit(EXIT_FAILURE);
+ }
+ #else
+ fh = OpenPipeStream(command, "r");
+ if (fh == NULL)
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not execute command \"%s\": %m",
+ command)));
+ #endif
+
+ if ((len = fread(buf, sizeof(char), size, fh)) < size)
+ {
+ if (ferror(fh))
+ {
+ #ifdef FRONTEND
+ pg_log_fatal("could not read from command \"%s\": %m",
+ command);
+ exit(EXIT_FAILURE);
+ #else
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not read from command \"%s\": %m",
+ command)));
+ #endif
+ }
+ }
+
+ #ifdef FRONTEND
+ pclose_rc = close_pipe_stream(fh);
+ #else
+ pclose_rc = ClosePipeStream(fh);
+ #endif
+
+ if (pclose_rc == -1)
+ {
+ #ifdef FRONTEND
+ pg_log_fatal("could not close pipe to external command: %m");
+ exit(EXIT_FAILURE);
+ #else
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not close pipe to external command: %m")));
+ #endif
+ }
+ else if (pclose_rc != 0)
+ {
+ #ifdef FRONTEND
+ pg_log_fatal("command \"%s\" failed", command);
+ exit(EXIT_FAILURE);
+ #else
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("command \"%s\" failed",
+ command),
+ errdetail_internal("%s", wait_result_to_str(pclose_rc))));
+ #endif
+ }
+
+ return len;
+ }
+
+ #ifdef FRONTEND
+ static FILE *
+ open_pipe_stream(const char *command)
+ {
+ FILE *res;
+
+ #ifdef WIN32
+ size_t cmdlen = strlen(command);
+ char *buf;
+ int save_errno;
+
+ buf = malloc(cmdlen + 2 + 1);
+ if (buf == NULL)
+ {
+ errno = ENOMEM;
+ return NULL;
+ }
+ buf[0] = '"';
+ mempcy(&buf[1], command, cmdlen);
+ buf[cmdlen + 1] = '"';
+ buf[cmdlen + 2] = '\0';
+
+ res = _popen(buf, "r");
+
+ save_errno = errno;
+ free(buf);
+ errno = save_errno;
+ #else
+ res = popen(command, "r");
+ #endif /* WIN32 */
+ return res;
+ }
+
+ static int
+ close_pipe_stream(FILE *file)
+ {
+ #ifdef WIN32
+ return _pclose(file);
+ #else
+ return pclose(file);
+ #endif /* WIN32 */
+ }
+ #endif /* FRONTEND */
+
+ CryptoKey *
+ kmgr_get_cryptokeys(const char *path, int *nkeys)
+ {
+ struct dirent *de;
+ DIR *dir;
+ CryptoKey *keys;
+
+ #ifndef FRONTEND
+ if ((dir = AllocateDir(path)) == NULL)
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not open directory \"%s\": %m",
+ path)));
+ #else
+ if ((dir = opendir(path)) == NULL)
+ pg_log_fatal("could not open directory \"%s\": %m", path);
+ #endif
+
+ keys = (CryptoKey *) palloc0(sizeof(CryptoKey) * KMGR_MAX_INTERNAL_KEYS);
+ *nkeys = 0;
+
+ #ifndef FRONTEND
+ while ((de = ReadDir(dir, KMGR_DIR)) != NULL)
+ #else
+ while ((de = readdir(dir)) != NULL)
+ #endif
+ {
+ if (strlen(de->d_name) == 4 &&
+ strspn(de->d_name, "0123456789ABCDEF") == 4)
+ {
+ uint32 id;
+
+ id = strtoul(de->d_name, NULL, 16);
+
+ if (id < 0 || id >= KMGR_MAX_INTERNAL_KEYS)
+ {
+ #ifndef FRONTEND
+ elog(ERROR, "invalid cryptographic key identifier %u", id);
+ #else
+ pg_log_fatal("invalid cryptographic key identifier %u", id);
+ #endif
+ }
+
+ if (*nkeys >= KMGR_MAX_INTERNAL_KEYS)
+ {
+ #ifndef FRONTEND
+ elog(ERROR, "too many cryptographic kes");
+ #else
+ pg_log_fatal("too many cryptographic keys");
+ #endif
+ }
+
+ read_one_keyfile(path, id, &(keys[id]));
+ (*nkeys)++;
+ }
+ }
+
+ #ifndef FRONTEND
+ FreeDir(dir);
+ #else
+ closedir(dir);
+ #endif
+
+ return keys;
+ }
+
+ static void
+ read_one_keyfile(const char *cryptoKeyDir, uint32 id, CryptoKey *key_p)
+ {
+ char path[MAXPGPATH];
+ int fd;
+ int r;
+
+ CryptoKeyFilePath(path, cryptoKeyDir, id);
+
+ #ifndef FRONTEND
+ if ((fd = OpenTransientFile(path, O_RDONLY | PG_BINARY)) == -1)
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not open file \"%s\" for reading: %m",
+ path)));
+ #else
+ if ((fd = open(path, O_RDONLY | PG_BINARY, 0)) == -1)
+ pg_log_fatal("could not open file \"%s\" for reading: %m",
+ path);
+ #endif
+
+ #ifndef FRONTEND
+ pgstat_report_wait_start(WAIT_EVENT_KEY_FILE_READ);
+ #endif
+
+ /* Get key bytes */
+ r = read(fd, key_p, sizeof(CryptoKey));
+ if (r != sizeof(CryptoKey))
+ {
+ if (r < 0)
+ {
+ #ifndef FRONTEND
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not read file \"%s\": %m", path)));
+ #else
+ pg_log_fatal("could not read file \"%s\": %m", path);
+ #endif
+ }
+ else
+ {
+ #ifndef FRONTEND
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg("could not read file \"%s\": read %d of %zu",
+ path, r, sizeof(CryptoKey))));
+ #else
+ pg_log_fatal("could not read file \"%s\": read %d of %zu",
+ path, r, sizeof(CryptoKey));
+ #endif
+ }
+ }
+
+ #ifndef FRONTEND
+ pgstat_report_wait_end();
+ #endif
+
+ #ifndef FRONTEND
+ if (CloseTransientFile(fd) != 0)
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not close file \"%s\": %m",
+ path)));
+ #else
+ if (close(fd) != 0)
+ pg_log_fatal("could not close file \"%s\": %m", path);
+ #endif
+ }
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
new file mode 100644
index 221af87..33f28c7
*** a/src/include/access/xlog.h
--- b/src/include/access/xlog.h
*************** extern void UpdateControlFile(void);
*** 319,324 ****
--- 319,325 ----
extern uint64 GetSystemIdentifier(void);
extern char *GetMockAuthenticationNonce(void);
extern bool DataChecksumsEnabled(void);
+ extern bool KeyManagementEnabled(void);
extern XLogRecPtr GetFakeLSNForUnloggedRel(void);
extern Size XLOGShmemSize(void);
extern void XLOGShmemInit(void);
diff --git a/src/include/catalog/pg_control.h b/src/include/catalog/pg_control.h
new file mode 100644
index 06bed90..c7df6d6
*** a/src/include/catalog/pg_control.h
--- b/src/include/catalog/pg_control.h
*************** typedef struct ControlFileData
*** 226,231 ****
--- 226,234 ----
*/
char mock_authentication_nonce[MOCK_AUTH_NONCE_LEN];
+ /* Key management cipher. Zero if no version */
+ uint32 key_management_version;
+
/* CRC of all above ... MUST BE LAST! */
pg_crc32c crc;
} ControlFileData;
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
new file mode 100644
index bbcac69..fbfe9de
*** a/src/include/catalog/pg_proc.dat
--- b/src/include/catalog/pg_proc.dat
***************
*** 10992,10995 ****
--- 10992,11001 ----
proname => 'is_normalized', prorettype => 'bool', proargtypes => 'text text',
prosrc => 'unicode_is_normalized' },
+ # function for key managements
+ { oid => '8200', descr => 'rotate cluter passphrase',
+ proname => 'pg_rotate_cluster_passphrase',
+ provolatile => 'v', prorettype => 'bool',
+ proargtypes => '', prosrc => 'pg_rotate_cluster_passphrase' },
+
]
diff --git a/src/include/common/cipher.h b/src/include/common/cipher.h
new file mode 100644
index ...f782791
*** a/src/include/common/cipher.h
--- b/src/include/common/cipher.h
***************
*** 0 ****
--- 1,78 ----
+ /*-------------------------------------------------------------------------
+ *
+ * cipher.h
+ * Declarations for cryptographic functions
+ *
+ * Portions Copyright (c) 2020, PostgreSQL Global Development Group
+ *
+ * src/include/common/cipher.h
+ *
+ *-------------------------------------------------------------------------
+ */
+ #ifndef CIPHER_H
+ #define CIPHER_H
+
+ #ifdef USE_OPENSSL
+ #include <openssl/evp.h>
+ #include <openssl/conf.h>
+ #include <openssl/err.h>
+ #endif
+
+ /*
+ * Supported symmetric encryption algorithm. These identifiers are passed
+ * to pg_cipher_ctx_create() function, and then actual encryption
+ * implementations need to initialize their context of the given encryption
+ * algorithm.
+ */
+ #define PG_CIPHER_AES_CBC 0
+ #define PG_MAX_CIPHER_ID 1
+
+ /* AES128/192/256 various length definitions */
+ #define PG_AES128_KEY_LEN (128 / 8)
+ #define PG_AES192_KEY_LEN (192 / 8)
+ #define PG_AES256_KEY_LEN (256 / 8)
+
+ /*
+ * The encrypted data is a series of blocks of size. Initialization
+ * vector(IV) is the same size of cipher block.
+ */
+ #define PG_AES_BLOCK_SIZE 16
+ #define PG_AES_IV_SIZE (PG_AES_BLOCK_SIZE)
+
+ /* HMAC key and HMAC length. We use HMAC-SHA256 */
+ #define PG_HMAC_SHA512_KEY_LEN 64
+ #define PG_HMAC_SHA512_LEN 64
+
+ #ifdef USE_OPENSSL
+ typedef EVP_CIPHER_CTX cipher_private_ctx;
+ #else
+ typedef void cipher_private_ctx;
+ #endif
+
+ /*
+ * This struct has two implementation-private context for
+ * encryption and decryption. The caller must create the encryption
+ * context using by pg_cipher_ctx_create() and pass the context to
+ * pg_cipher_encrypt() or pg_cipher_decrypt().
+ */
+ typedef struct PgCipherCtx
+ {
+ cipher_private_ctx *encctx;
+ cipher_private_ctx *decctx;
+ } PgCipherCtx;
+
+ extern PgCipherCtx *pg_cipher_ctx_create(int cipher, uint8 *key, int klen);
+ extern void pg_cipher_ctx_free(PgCipherCtx *ctx);
+ extern bool pg_cipher_encrypt(PgCipherCtx *ctx,
+ const uint8 *in, int inlen,
+ uint8 *out, int *outlen,
+ const uint8 *iv);
+ extern bool pg_cipher_decrypt(PgCipherCtx *ctx,
+ const uint8 *in, int inlen,
+ uint8 *out, int *outlen,
+ const uint8 *iv);
+ extern bool pg_HMAC_SHA512(const uint8 *key,
+ const uint8 *in, int inlen,
+ uint8 *out);
+
+ #endif /* CIPHER_H */
diff --git a/src/include/common/cipher_openssl.h b/src/include/common/cipher_openssl.h
new file mode 100644
index ...0fd1308
*** a/src/include/common/cipher_openssl.h
--- b/src/include/common/cipher_openssl.h
***************
*** 0 ****
--- 1,37 ----
+ /*-------------------------------------------------------------------------
+ *
+ * cipher_openssl.h
+ * Declarations for helper functions using OpenSSL
+ *
+ * Portions Copyright (c) 2020, PostgreSQL Global Development Group
+ *
+ * src/include/common/cipher_openssl.h
+ *
+ *-------------------------------------------------------------------------
+ */
+ #ifndef CIPHER_OPENSSL_H
+ #define CIPHER_OPENSSL_H
+
+ #ifndef FRONTEND
+ #include "postgres.h"
+ #else
+ #include "postgres_fe.h"
+ #endif
+
+ #include "common/cipher.h"
+
+ extern EVP_CIPHER_CTX *ossl_cipher_ctx_create(int cipher, uint8 *key, int klen,
+ bool enc);
+ extern void ossl_cipher_ctx_free(EVP_CIPHER_CTX *ctx);
+ extern bool ossl_cipher_encrypt(EVP_CIPHER_CTX *ctx,
+ const uint8 *in, int inlen,
+ uint8 *out, int *outlen,
+ const uint8 *iv);
+ extern bool ossl_cipher_decrypt(EVP_CIPHER_CTX *ctx,
+ const uint8 *in, int inlen,
+ uint8 *out, int *outlen,
+ const uint8 *iv);
+ extern bool ossl_HMAC_SHA512(const uint8 *key,
+ const uint8 *in, int inlen,
+ uint8 *out);
+ #endif
diff --git a/src/include/common/kmgr_utils.h b/src/include/common/kmgr_utils.h
new file mode 100644
index ...1dc8f43
*** a/src/include/common/kmgr_utils.h
--- b/src/include/common/kmgr_utils.h
***************
*** 0 ****
--- 1,105 ----
+ /*-------------------------------------------------------------------------
+ *
+ * kmgr_utils.h
+ * Declarations for utility function for key management
+ *
+ * Portions Copyright (c) 2020, PostgreSQL Global Development Group
+ *
+ * src/include/common/kmgr_utils.h
+ *
+ *-------------------------------------------------------------------------
+ */
+ #ifndef KMGR_UTILS_H
+ #define KMGR_UTILS_H
+
+ #include "common/cipher.h"
+
+ /* Current version number */
+ #define KMGR_VERSION 1
+
+ /*
+ * Directory where cryptographic keys reside within PGDATA. KMGR_DIR_TMP
+ * is used during cluster passphrase rotation.
+ */
+ #define KMGR_DIR "pg_cryptokeys"
+ #define KMGR_TMP_DIR "pg_cryptokeys_tmp"
+
+ /*
+ * Identifiers of internal keys. When adding a new internal key, we
+ * also need to add its key length to internalKeyLengths.
+ */
+ /* #define KMGR_XXX_KEY_ID 0 */
+ #define KMGR_MAX_INTERNAL_KEYS 0
+
+ /* Encryption key and MAC key used for key wrapping */
+ #define KMGR_ENC_KEY_LEN PG_AES256_KEY_LEN
+ #define KMGR_MAC_KEY_LEN PG_HMAC_SHA512_KEY_LEN
+ #define KMGR_HMAC_LEN PG_HMAC_SHA512_LEN
+
+ /* Key wrapping key consists of encryption key and mac key */
+ #define KMGR_KEY_LEN (PG_AEAD_ENC_KEY_LEN + PG_AEAD_MAC_KEY_LEN)
+
+ /* Allowed length of cluster passphrase */
+ #define KMGR_MIN_PASSPHRASE_LEN 64
+ #define KMGR_MAX_PASSPHRASE_LEN 1024
+
+ /* Maximum length of key the key manager can store */
+ #define KMGR_MAX_KEY_LEN 128
+ #define KMGR_MAX_WRAPPED_KEY_LEN KmgrSizeOfCipherText(KMGR_MAX_KEY_LEN)
+
+ /*
+ * Size of encrypted key size with padding. We use PKCS#7 padding,
+ * described in RFC 5652.
+ */
+ #define SizeOfDataWithPadding(klen) \
+ ((int)(klen) + (PG_AES_BLOCK_SIZE - ((int)(klen) % PG_AES_BLOCK_SIZE)))
+
+ /* Macros to compute the size of cipher text and plain text */
+ #define KmgrSizeOfCipherText(len) \
+ (KMGR_MAC_KEY_LEN + PG_AES_IV_SIZE + SizeOfDataWithPadding((int)(len)))
+ #define KmgrSizeOfPlainText(klen) \
+ ((int)(klen) - (KMGR_MAC_KEY_LEN + PG_AES_IV_SIZE))
+
+ /* CryptoKey file name is keys id */
+ #define CryptoKeyFilePath(path, dir, id) \
+ snprintf((path), MAXPGPATH, "%s/%04X", (dir), (id))
+
+ /*
+ * Cryptographic key data structure. This structure is used for
+ * both on-disk (raw key) and on-memory (wrapped key).
+ */
+ typedef struct CryptoKey
+ {
+ int klen;
+ uint8 key[KMGR_MAX_WRAPPED_KEY_LEN];
+ } CryptoKey;
+
+ /* Key wrapping cipher context */
+ typedef struct PgKeyWrapCtx
+ {
+ uint8 key[KMGR_ENC_KEY_LEN];
+ uint8 mackey[KMGR_MAC_KEY_LEN];
+ PgCipherCtx *cipherctx;
+ } PgKeyWrapCtx;
+
+ extern PgKeyWrapCtx *pg_create_keywrap_ctx(uint8 key[KMGR_ENC_KEY_LEN],
+ uint8 mackey[KMGR_MAC_KEY_LEN]);
+ extern void pg_free_keywrap_ctx(PgKeyWrapCtx *ctx);
+ extern bool kmgr_wrap_key(PgKeyWrapCtx *ctx, CryptoKey *in, CryptoKey *out);
+ extern bool kmgr_unwrap_key(PgKeyWrapCtx *ctx, CryptoKey *in, CryptoKey *out);
+
+
+
+ extern void kmgr_derive_keys(char *passphrase, Size passlen,
+ uint8 enckey[KMGR_ENC_KEY_LEN],
+ uint8 mackey[KMGR_MAC_KEY_LEN]);
+ extern bool kmgr_verify_passphrase(char *passphrase, int passlen,
+ CryptoKey *keys_in, CryptoKey *keys_out,
+ int nkey);
+ extern bool kmgr_wrap_key(PgKeyWrapCtx *ctx, CryptoKey *in, CryptoKey *out);
+ extern bool kmgr_unwrap_key(PgKeyWrapCtx *ctx, CryptoKey *in, CryptoKey *out);
+ extern int kmgr_run_cluster_passphrase_command(char *passphrase_command,
+ char *buf, int size);
+ extern CryptoKey *kmgr_get_cryptokeys(const char *path, int *nkeys);
+
+ #endif /* KMGR_UTILS_H */
diff --git a/src/include/crypto/kmgr.h b/src/include/crypto/kmgr.h
new file mode 100644
index ...783f06d
*** a/src/include/crypto/kmgr.h
--- b/src/include/crypto/kmgr.h
***************
*** 0 ****
--- 1,29 ----
+ /*-------------------------------------------------------------------------
+ *
+ * kmgr.h
+ *
+ * Portions Copyright (c) 2020, PostgreSQL Global Development Group
+ *
+ * src/include/crypto/kmgr.h
+ *
+ *-------------------------------------------------------------------------
+ */
+ #ifndef KMGR_H
+ #define KMGR_H
+
+ #include "common/cipher.h"
+ #include "common/kmgr_utils.h"
+ #include "storage/relfilenode.h"
+ #include "storage/bufpage.h"
+
+ /* GUC parameters */
+ extern bool key_management_enabled;
+ extern char *cluster_passphrase_command;
+
+ extern Size KmgrShmemSize(void);
+ extern void KmgrShmemInit(void);
+ extern void BootStrapKmgr(void);
+ extern void InitializeKmgr(void);
+ extern const CryptoKey *KmgrGetKey(int id);
+
+ #endif /* KMGR_H */
diff --git a/src/include/pg_config.h.in b/src/include/pg_config.h.in
new file mode 100644
index fb270df..d50a7c9
*** a/src/include/pg_config.h.in
--- b/src/include/pg_config.h.in
***************
*** 385,390 ****
--- 385,393 ----
/* Define to 1 if you have the `OPENSSL_init_ssl' function. */
#undef HAVE_OPENSSL_INIT_SSL
+ /* Define to 1 if you have the `OPENSSL_init_crypto' function. */
+ #undef HAVE_OPENSSL_INIT_CRYPTO
+
/* Define to 1 if you have the <ossp/uuid.h> header file. */
#undef HAVE_OSSP_UUID_H
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
new file mode 100644
index a821ff4..6549260
*** a/src/include/pgstat.h
--- b/src/include/pgstat.h
*************** typedef enum
*** 998,1003 ****
--- 998,1006 ----
WAIT_EVENT_DATA_FILE_TRUNCATE,
WAIT_EVENT_DATA_FILE_WRITE,
WAIT_EVENT_DSM_FILL_ZERO_WRITE,
+ WAIT_EVENT_KEY_FILE_READ,
+ WAIT_EVENT_KEY_FILE_WRITE,
+ WAIT_EVENT_KEY_FILE_SYNC,
WAIT_EVENT_LOCK_FILE_ADDTODATADIR_READ,
WAIT_EVENT_LOCK_FILE_ADDTODATADIR_SYNC,
WAIT_EVENT_LOCK_FILE_ADDTODATADIR_WRITE,
diff --git a/src/include/postmaster/postmaster.h b/src/include/postmaster/postmaster.h
new file mode 100644
index babc87d..b1f0721
*** a/src/include/postmaster/postmaster.h
--- b/src/include/postmaster/postmaster.h
*************** extern bool enable_bonjour;
*** 30,35 ****
--- 30,37 ----
extern char *bonjour_name;
extern bool restart_after_crash;
+ extern int terminal_fd;
+
#ifdef WIN32
extern HANDLE PostmasterHandle;
#else
diff --git a/src/include/utils/guc_tables.h b/src/include/utils/guc_tables.h
new file mode 100644
index 04431d0..a0e582a
*** a/src/include/utils/guc_tables.h
--- b/src/include/utils/guc_tables.h
*************** enum config_group
*** 89,94 ****
--- 89,95 ----
STATS,
STATS_MONITORING,
STATS_COLLECTOR,
+ ENCRYPTION,
AUTOVACUUM,
CLIENT_CONN,
CLIENT_CONN_STATEMENT,
diff --git a/src/test/Makefile b/src/test/Makefile
new file mode 100644
index 9774f53..6c998fc
*** a/src/test/Makefile
--- b/src/test/Makefile
*************** endif
*** 29,35 ****
endif
ifeq ($(with_openssl),yes)
ifneq (,$(filter ssl,$(PG_TEST_EXTRA)))
! SUBDIRS += ssl
endif
endif
--- 29,35 ----
endif
ifeq ($(with_openssl),yes)
ifneq (,$(filter ssl,$(PG_TEST_EXTRA)))
! SUBDIRS += ssl crypto
endif
endif
pass_fd.sh
Description: Bourne shell script
