Change Details

Pacemaker optionally supports encrypting communications with TLS certificates for two things: # Pacemaker Remote node connections # Remote CIB operations (`CIB_user=... CIB_encrypted=... CIB_port=... cibadmin -Q`) This is especially important for remote CIB operations which have no encryption at all otherwise. Testing this requires a fair bit of setup. We assume that users will use TLS certificates as follows: - They will set up their own Certificate Authority (CA). In the real world, you pay a real CA a lot of money to sign your credentials. This is because web browsers have lists of CAs they trust, and so if you use one of those then your credentials will be trusted. Otherwise the user will get a scary error about self-signed certificates and things like that. However, we expect that people will keep their clusters on private networks and connect via our own command line tools, so there's no real need for a trusted, real CA. - They will generate one set of credentials for each server. If they have ten Pacemaker Remote nodes (remember - remote nodes are the server, the cluster is the client) then each would have its own server cert and key. - They will generate one set of credentials for each client. If they have ten admin machines that need to run `cibadmin` to do remote CIB operations, then each would have its own client cert and key. This makes it possible to revoke a single client's credentials if necessary without a lot of annoying work. In addition, this document will assume there is not an intermediate CA involved. In the real world, you would create a root CA and only use that to sign credentials for intermediate CAs. The intermediate CA would then sign the client/server credentials. The reason for this is that if your CA gets compromised, everything it has ever signed is now compromised as well. The root CA credentials would be locked away and only taken out on the rare time when you need a new intermediate. Pacemaker should support this just fine as long as the root CA and intermediate CA credentials get bundled together. I'm not going to cover how to do that here. If you look around, you'll find tons of explanations for how to set this stuff up. These instructions worked for me. There very well may be much simpler ways to do all of this. To generate TLS credentials, you need to do all of the following... = Create your own CA = == Set up the directory structure == ``` $ mkdir -p ~/cert-stuff/ca/{certs,newcerts,private,csr} $ mkdir -p ~/cert-stuff/{clients,servers} $ cd ~/cert-stuff/ca $ echo 01 > serial $ echo 01 > crlnumber $ touch index.txt ``` == Create `openssl.cnf` == I'm not sure how much of this is necessary at all. I haven't experimented with simplification. At the least, check `dir`, `countryName`, `stateOrProvinceName`, `localityName`, and `organizationName` and set them to something that makes sense for you. ``` [ ca ] # The default CA section default_ca = CA_default # The default CA name [ CA_default ] # Default settings for the CA dir = /root/cert-stuff/ca # CA directory certs = $dir/certs # Certificates directory crl_dir = $dir/crl # CRL directory new_certs_dir = $dir/newcerts # New certificates directory database = $dir/index.txt # Certificate index file serial = $dir/serial # Serial number file RANDFILE = $dir/private/.rand # Random number file private_key = $dir/private/ca.key.pem # Root CA private key certificate = $dir/certs/ca.cert.pem # Root CA certificate crl = $dir/crl/ca.crl.pem # Root CA CRL crlnumber = $dir/crlnumber # Root CA CRL number crl_extensions = crl_ext # CRL extensions default_crl_days = 30 # Default CRL validity days default_md = sha256 # Default message digest preserve = no # Preserve existing extensions email_in_dn = no # Exclude email from the DN name_opt = ca_default # Formatting options for names cert_opt = ca_default # Certificate output options policy = policy_strict # Certificate policy unique_subject = no # Allow multiple certs with the same DN [ policy_strict ] # Policy for stricter validation countryName = match # Must match the issuer's country stateOrProvinceName = match # Must match the issuer's state organizationName = match # Must match the issuer's organization organizationalUnitName = optional # Organizational unit is optional commonName = optional # Must provide a common name emailAddress = optional # Email address is optional [ req ] # Request settings default_bits = 2048 # Default key size distinguished_name = req_distinguished_name # Default DN template string_mask = utf8only # UTF-8 encoding default_md = sha256 # Default message digest prompt = no # Non-interactive mode [ req_distinguished_name ] # Template for the DN in the CSR countryName = US stateOrProvinceName = NH localityName = Merrimack organizationName = Cluster [ v3_ca ] # Root CA certificate extensions subjectKeyIdentifier = hash # Subject key identifier authorityKeyIdentifier = keyid:always,issuer # Authority key identifier basicConstraints = critical, CA:true # Basic constraints for a CA keyUsage = critical, keyCertSign, cRLSign # Key usage for a CA [ crl_ext ] # CRL extensions authorityKeyIdentifier = keyid:always,issuer # Authority key identifier ``` == Generate the root CA certificate == Set `days` to something appropriate - this is how many days until the root CA expires. For testing, shorter is fine. For real world use, make it very long. When the root CA expires, everything it signed expires too. Years would be appropriate. Also set `subj` appropriately too. I'm not actually sure if this needs to be set at all given that it's already in the config file. ``` $ openssl genrsa -out private/ca.key.pem 4096 $ openssl req -config openssl.cnf -key private/ca.key.pem -new -x509 -days 100 -sha256 -extensions v3_ca -out certs/ca.cert.pem -subj "/C=US/ST=NH/L=Merrimack/O=Cluster/CN=Root CA" ``` == Installation == For real world use, keep the private `ca.key.pem` file private. Set ownership and permissions restrictively. Maybe store it on a USB key instead of permanently on the system. The `ca.cert.pem` file should be installed to every client and server system and readable by the cluster user (`hacluster:haclient` most likely). This file should also be referenced in `/etc/sysconfig/pacemaker` as `PCMK_ca_file`. = Generate server stuff = For each server (Pacemaker Remote node or hosts that you want to be able to access via `cibadmin`) do the following: == Generate private key and Certificate Signing Request (CSR) == Replace `<servername>` with the hostname (not FQDN) of the server. You can name it whatever you want, actually, but using the hostname makes keeping track of these things easier if there's a bunch of them. ``` $ openssl genrsa -out ../servers/<servename>.key.pem 4096 $ openssl req -new -key ../servers/<servername>.key.pem -out ../servers/<servername>.csr ``` Answer any questions openssl asks. If it asks for a passphrase, don't provide one. This has not been tested yet and it's likely the cluster will hang asking for the passphrase when it tries to read the key. == Create server_cert_ext.cnf == Some things can't be set on the command line so they go in this file instead. ``` basicConstraints = CA:FALSE nsCertType = server nsComment = "OpenSSL Generated Server Certificate" subjectKeyIdentifier = hash authorityKeyIdentifier = keyid,issuer:always keyUsage = critical, digitalSignature, keyEncipherment extendedKeyUsage = serverAuth ``` == Create and sign the server certificate == Again, change `<servername>` to the hostname and set `days` to something that makes sense. ``` $ openssl x509 -req -in ../servers/<servername>.csr -CA certs/ca.cert.pem -CAkey private/ca.key.pem -out ../servers/<servername>.cert.pem -CAcreateserial -days 100 -sha256 -extfile server_cert_ext.cnf $ openssl ca -extfile server_cert_ext.cnf -config openssl.cnf -days 100 -md sha256 -out ../servers/<servername>.cert.pem -infiles ../servers/<servername>.csr ``` == Installation == Copy `<servername>.cert.pem` and `<servername>.key.pem` to the server. Both should be owned by the cluster user (`hacluster:haclient` most likely). The key should only be readable by the cluster user. I think it's okay if the cert is world-readable. These files should also be referenced in `/etc/sysconfig/pacemaker` as `PCMK_cert_file` and `PCMK_key_file`. = Generate client stuff = Everything is exactly the same on clients, except the paths will be different and there's a different config file. I'm not going to cover any of this in depth, just list the commands. Refer to the server section for details. == Generate private key and CSR == ``` $ openssl genrsa -out ../clients/<clientname>.key.pem 4096 $ openssl req -new -key ../clients/<clientname>.key.pem -out ../clients/<clientname>.csr ``` == Create client_cert_ext.cnf == ``` basicConstraints = CA:FALSE nsCertType = client, email nsComment = "OpenSSL Generated Client Certificate" subjectKeyIdentifier = hash authorityKeyIdentifier = keyid,issuer keyUsage = critical, nonRepudiation, digitalSignature, keyEncipherment extendedKeyUsage = clientAuth, emailProtection ``` == Create and sign the client certificate == ``` $ openssl x509 -req -in ../clients/<clientname>.csr -CA certs/ca.cert.pem -CAkey private/ca.key.pem -out ../clients/<clientname>.cert.pem -CAcreateserial -days 100 -sha256 -extfile client_cert_ext.cnf $ openssl ca -extfile client_cert_ext.cnf -config openssl.cnf -days 100 -md sha256 -out ../clients/<clientname>.cert.pem -infiles ../clients/<clientname>.csr ``` == Installation == Installation of the client side is the same as the server side. == Revoking a certificate == ``` $ openssl ca -config openssl.cnf -revoke <client/server>.cert.pem $ openssl ca -config openssl.cnf -gencrl -out crl/rootca.crl ``` --- Having done all that, using it is a lot less complicated. Pacemaker Administration and Pacemaker Explained cover this in more depth, but the quick explanation is: - For remote CIB operations, the admin machine is the client and the cluster machine is the server. Make sure the server has the server credentials and they are listed in `/etc/sysconfig/pacemaker`. Command line tools (like `cibadmin`, most commonly used for remote CIB operations) do not read the sysconfig file so all the credentials must be passed by the environment. Note that in addition to the TLS parameters, you must still give a username and password. See Pacemaker Administration for more details. - For Pacemaker Remote nodes, the cluster machine running the remote resource is the client and the system running `pacemaker_remoted` is the server. Here, both sides are running a daemon so make sure the appropriate credentials are listed in `/etc/sysconfig/pacemaker` on both ends of the connection. There is no need to pass anything in the environment.

Pacemaker optionally supports encrypting communications with TLS certificates for two things: # Pacemaker Remote node connections # Remote CIB operations (`CIB_user=... CIB_encrypted=... CIB_port=... cibadmin -Q`) This is especially important for remote CIB operations which have no encryption at all otherwise. Testing this requires a fair bit of setup. We assume that users will use TLS certificates as follows: - They will set up their own Certificate Authority (CA). In the real world, you pay a real CA a lot of money to sign your credentials. This is because web browsers have lists of CAs they trust, and so if you use one of those then your credentials will be trusted. Otherwise the user will get a scary error about self-signed certificates and things like that. However, we expect that people will keep their clusters on private networks and connect via our own command line tools, so there's no real need for a trusted, real CA. - They will generate one set of credentials for each server. If they have ten Pacemaker Remote nodes (remember - remote nodes are the server, the cluster is the client) then each would have its own server cert and key. - They will generate one set of credentials for each client. If they have ten admin machines that need to run `cibadmin` to do remote CIB operations, then each would have its own client cert and key. This makes it possible to revoke a single client's credentials if necessary without a lot of annoying work. In addition, this document will assume there is not an intermediate CA involved. In the real world, you would create a root CA and only use that to sign credentials for intermediate CAs. The intermediate CA would then sign the client/server credentials. The reason for this is that if your CA gets compromised, everything it has ever signed is now compromised as well. The root CA credentials would be locked away and only taken out on the rare time when you need a new intermediate. Pacemaker should support this just fine as long as the root CA and intermediate CA credentials get bundled together. I'm not going to cover how to do that here. If you look around, you'll find tons of explanations for how to set this stuff up. These instructions worked for me. There very well may be much simpler ways to do all of this. To generate TLS credentials, you need to do all of the following... = Create your own CA = == Set up the directory structure == ``` $ mkdir -p ~/cert-stuff/ca/{certs,newcerts,private,csr} $ mkdir -p ~/cert-stuff/{clients,servers} $ cd ~/cert-stuff/ca $ echo 01 > serial $ echo 01 > crlnumber $ touch index.txt ``` == Create `openssl.cnf` == If you don't create your own config file and specify it on the openssl command line, it will default to using `/etc/pki/tls/openssl.cnf`. This may or may not be what you want. So, we'll create our own just to make sure everything is explicitly defined. Much of this is largely just boilerplate. However, pay attention to the `dir` setting and the default values in `req_distinguished_name`. ``` [ ca ] # Use the options from the CA_default section default_ca = CA_default [ CA_default ] # Directory and file locations. dir = /root/cert-stuff/ca certs = $dir/certs crl_dir = $dir/crl new_certs_dir = $dir/newcerts database = $dir/index.txt serial = $dir/serial RANDFILE = $dir/private/.rand # The root key and root certificate. private_key = $dir/private/ca.key.pem certificate = $dir/certs/ca.cert.pem # For certificate revocation lists. crlnumber = $dir/crlnumber crl = $dir/crl/ca.crl.pem crl_extensions = crl_ext default_crl_days = 30 # SHA-1 is deprecated, so use SHA-2 instead. default_md = sha256 # Formatting option for names name_opt = ca_default # Certificate output options cert_opt = ca_default # Default certificate validity default_days = 375 # Preserve existing extensions preserve = no # Certificate policy section policy = policy_strict # Policy section to use for validating root CA signatures [ policy_strict ] countryName = match stateOrProvinceName = match organizationName = match organizationalUnitName = optional commonName = supplied emailAddress = optional # Options to apply when creating certs or CSRs [ req ] default_bits = 2048 distinguished_name = req_distinguished_name string_mask = utf8only # SHA-1 is deprecated, so use SHA-2 instead. default_md = sha256 # Extension to add when the -x509 option is used. x509_extensions = v3_ca # Information required in a CSR, with optional defaults [ req_distinguished_name ] countryName = Country Name (2 letter code) stateOrProvinceName = State or Province Name localityName = Locality Name 0.organizationName = Organization Name organizationalUnitName = Organizational Unit Name commonName = Common Name emailAddress = Email Address # Optionally, specify some defaults. countryName_default = US stateOrProvinceName_default = NH localityName_default = Merrimack 0.organizationName_default = Cluster #organizationalUnitName_default = #emailAddress_default = # Options that are applied when `-extensions v3_ca` is given when creating # the root certificate [ v3_ca ] subjectKeyIdentifier = hash authorityKeyIdentifier = keyid:always,issuer basicConstraints = critical, CA:true keyUsage = critical, digitalSignature, cRLSign, keyCertSign # Options that are applied when `-extensions server_cert` is given when signing # server certificates [ server_cert ] basicConstraints = CA:FALSE nsCertType = server nsComment = "OpenSSL Generated Server Certificate" subjectKeyIdentifier = hash authorityKeyIdentifier = keyid,issuer:always keyUsage = critical, digitalSignature, keyEncipherment extendedKeyUsage = serverAuth # Options that are applied when `-extensions client_cert` is given when signing # client certificates [ client_cert ] basicConstraints = CA:FALSE nsCertType = client, email nsComment = "OpenSSL Generated Client Certificate" subjectKeyIdentifier = hash authorityKeyIdentifier = keyid,issuer keyUsage = critical, nonRepudiation, digitalSignature, keyEncipherment extendedKeyUsage = clientAuth, emailProtection # Options that are applied when creating CRLs [ crl_ext ] authorityKeyIdentifier = keyid:always ``` == Generate the root CA certificate == Set `days` to something appropriate - this is how many days until the root CA expires. For testing, shorter is fine. For real world use, make it very long. When the root CA expires, everything it signed expires too. Years would be appropriate. You can optionally provide `-subj` on the command line, but if you don't, openssl will prompt you for this information. ``` $ openssl genrsa -out private/ca.key.pem 4096 $ chmod 400 private/ca.key.pem $ openssl req -config openssl.cnf -key private/ca.key.pem -new -x509 -days 100 -sha256 -extensions v3_ca -out certs/ca.cert.pem $ chmod 444 certs/ca.cert.pem ``` == Installation == For real world use, keep the private `ca.key.pem` file private. Set ownership and permissions restrictively. Maybe store it on a USB key instead of permanently on the system. The `ca.cert.pem` file should be installed to every client and server system and readable by the cluster user (`hacluster:haclient` most likely). This file should also be referenced in `/etc/sysconfig/pacemaker` as `PCMK_ca_file`. = Generate server stuff = For each server (Pacemaker Remote node or hosts that you want to be able to access via `cibadmin`) do the following: == Generate private key and Certificate Signing Request (CSR) == Replace `<servername>` with the hostname (not FQDN) of the server. You can name it whatever you want, actually, but using the hostname makes keeping track of these things easier if there's a bunch of them. ``` $ openssl genrsa -out ../servers/<servename>.key.pem 4096 $ chmod 400 ../servers/<servername>.key.pem $ openssl req -config openssl.cnf -new -key ../servers/<servername>.key.pem -out ../servers/<servername>.csr ``` Answer any questions openssl asks. If it asks for a passphrase, don't provide one. This has not been tested yet and it's likely the cluster will hang asking for the passphrase when it tries to read the key. When it asks for a Common Name, use the FQDN of the server. Note that the server's Common Name cannot be the same as the root CA's Common Name. == Create and sign the server certificate == Again, change `<servername>` to the hostname and set `days` to something that makes sense. ``` $ openssl ca -config openssl.cnf -extensions server_cert -days 100 -notext -md sha256 -in ../servers/<servername>.csr -out ../servers/<servername>.cert.pem $ chmod 444 ../servers/<servername>.cert.pem ``` == Installation == Copy `<servername>.cert.pem` and `<servername>.key.pem` to the server. Both should be owned by the cluster user (`hacluster:haclient` most likely). The key should only be readable by the cluster user. I think it's okay if the cert is world-readable. These files should also be referenced in `/etc/sysconfig/pacemaker` as `PCMK_cert_file` and `PCMK_key_file`. = Generate client stuff = Everything is exactly the same on clients, except the paths will be different and there's a different extension to use. I'm not going to cover any of this in depth, just list the commands. Refer to the server section for details. == Generate private key and CSR == ``` $ openssl genrsa -out ../clients/<clientname>.key.pem 4096 $ chmod 400 ../clients/<clientname>.key.pem $ openssl req -config openssl.cnf -new -key ../clients/<clientname>.key.pem -out ../clients/<clientname>.csr ``` For clients, the Common Name should be something like an email address. == Create and sign the client certificate == ``` $ openssl ca -config openssl.cnf -extensions client_cert -days 100 -notext -md sha256 -in ../clients/<clientname>.csr -out ../clients/<clientname>.cert.pem $ chmod 444 ../clients/<clientname>.cert.pem ``` == Installation == Installation of the client side is the same as the server side. == Revoking a certificate == If one of the admin machines used to access the CIB remotely is compromised, stolen, or can otherwise not be trusted, a Certificate Revocation List can be used to tell cluster machines to no longer allow use of that certificate. It's also possible to revoke the certificate of a server to prevent clients from accessing a potentially compromised server. In both cases, the commands are the same. ``` $ openssl ca -config openssl.cnf -gencrl -out crl/rootca.crl $ openssl ca -config openssl.cnf -revoke ../<clients/servers>/<name>.cert.pem ``` CRLs should be regenerated frequently (according to the config file's `default_crl_days` setting, they expire in 30 days) and distributed to all clients and servers. --- Having done all that, using it is a lot less complicated. Pacemaker Administration and Pacemaker Explained cover this in more depth, but the quick explanation is: - For remote CIB operations, the admin machine is the client and the cluster machine is the server. Make sure the server has the server credentials and they are listed in `/etc/sysconfig/pacemaker`. Command line tools (like `cibadmin`, most commonly used for remote CIB operations) do not read the sysconfig file so all the credentials must be passed by the environment. Note that in addition to the TLS parameters, you must still give a username and password. See Pacemaker Administration for more details. - For Pacemaker Remote nodes, the cluster machine running the remote resource is the client and the system running `pacemaker_remoted` is the server. Here, both sides are running a daemon so make sure the appropriate credentials are listed in `/etc/sysconfig/pacemaker` on both ends of the connection. There is no need to pass anything in the environment.

Pacemaker optionally supports encrypting communications with TLS certificates for two things: # Pacemaker Remote node connections # Remote CIB operations (`CIB_user=... CIB_encrypted=... CIB_port=... cibadmin -Q`) This is especially important for remote CIB operations which have no encryption at all otherwise. Testing this requires a fair bit of setup. We assume that users will use TLS certificates as follows: - They will set up their own Certificate Authority (CA). In the real world, you pay a real CA a lot of money to sign your credentials. This is because web browsers have lists of CAs they trust, and so if you use one of those then your credentials will be trusted. Otherwise the user will get a scary error about self-signed certificates and things like that. However, we expect that people will keep their clusters on private networks and connect via our own command line tools, so there's no real need for a trusted, real CA. - They will generate one set of credentials for each server. If they have ten Pacemaker Remote nodes (remember - remote nodes are the server, the cluster is the client) then each would have its own server cert and key. - They will generate one set of credentials for each client. If they have ten admin machines that need to run `cibadmin` to do remote CIB operations, then each would have its own client cert and key. This makes it possible to revoke a single client's credentials if necessary without a lot of annoying work. In addition, this document will assume there is not an intermediate CA involved. In the real world, you would create a root CA and only use that to sign credentials for intermediate CAs. The intermediate CA would then sign the client/server credentials. The reason for this is that if your CA gets compromised, everything it has ever signed is now compromised as well. The root CA credentials would be locked away and only taken out on the rare time when you need a new intermediate. Pacemaker should support this just fine as long as the root CA and intermediate CA credentials get bundled together. I'm not going to cover how to do that here. If you look around, you'll find tons of explanations for how to set this stuff up. These instructions worked for me. There very well may be much simpler ways to do all of this. To generate TLS credentials, you need to do all of the following... = Create your own CA = == Set up the directory structure == ``` $ mkdir -p ~/cert-stuff/ca/{certs,newcerts,private,csr} $ mkdir -p ~/cert-stuff/{clients,servers} $ cd ~/cert-stuff/ca $ echo 01 > serial $ echo 01 > crlnumber $ touch index.txt ``` == Create `openssl.cnf` == I'm not sure how much of this is necessary at all. I haven't experimented with simplificationf you don't create your own config file and specify it on the openssl command line, it will default to using `/etc/pki/tls/openssl.cnf`. At the least, check `dir`This may or may not be what you want. So, `countryName`, `stateOrProvinceName`, `localityName`we'll create our own just to make sure everything is explicitly defined. Much of this is largely just boilerplate. However, and `organizationName` and set them to something that makes sense for youpay attention to the `dir` setting and the default values in `req_distinguished_name`. ``` [ ca ] # The default CA section default_ca = CA_default # The default CA name# Use the options from the CA_default section default_ca = CA_default [ CA_default ] # Directory and file locations. dir = /root/cert-stuff/ca certs = $dir/certs crl_dir = $dir/crl new_certs_dir = $dir/newcerts database = $dir/index.txt serial = $dir/serial RANDFILE = $dir/private/.rand # The root key and root certificate. private_key = $dir/private/ca.key.pem certificate = $dir/certs/ca.cert.pem # For certificate revocation lists. crlnumber = $dir/crlnumber crl = $dir/crl/ca.crl.pem crl_extensions = crl_ext default_crl_days = 30 [ CA_default ] # Default settings for the CA# SHA-1 is deprecated, so use SHA-2 instead. dir default_md = /root/cert-stuff/ca # CA directorysha256 # Formatting option for names certs name_opt = $dir/certs # Certificates directoryca_default crl_dir = $dir/crl # CRL directory# Certificate output options new_certs_dir = $dir/newcerts _opt # New certificates directory = ca_default database = $dir/index.txt # C# Default certificate index filevalidity serial = $dir/serial default_days # Serial number file = 375 RANDFILE = $dir/private/.rand # Random number file# Preserve existing extensions private_key = $dir/private/ca.key.pem preserve # Root CA private key= no c# Certificate = $dir/certs/ca.cert.pem # Root CA certificatepolicy section crl policy = $dir/crl/ca.crl.pem # Rpolicy_strict # Policy section to use for validating root CA CRLsignatures [ policy_strict ] crlnumber = $dir/crlnumber countryName # Root CA CRL number= match crl_extensions = crl_ext stateOrProvinceName # CRL extensions = match default_crl_days = 30 organizationName # Default CRL validity days = match default_md = sha256 # Default message digestorganizationalUnitName = optional preserve = no commonName # Preserve existing extensions= supplied email_in_dn Address = no # Exclude email from the DN= optional # Options to apply when creating certs or CSRs [ req ] name_opt = ca_default _bits # Formatting options for names = 2048 cert_opt = ca_default # Certificate output optionsdistinguished_name = req_distinguished_name policy = policy_strict # Certificate policystring_mask = utf8only # SHA-1 is deprecated, so use SHA-2 instead. default_md = sha256 # Extension to add when the -x509 option is used. unique_subject = no # Allow multiple certs with the same DNx509_extensions = v3_ca # Information required in a CSR, with optional defaults [ policy_strict ] # Policy for stricter validationreq_distinguished_name ] countryName = match # Must match the issuer's country= Country Name (2 letter code) stateOrProvinceName = match # Must match the issuer's state= State or Province Name organizationlocalityName = match # Must match the issuer's organization = Locality Name 0.organizationalUnitnName = optional #= Organizational unit is optionaln Name commonName = oporganizational UnitName # Must provide a common name= Organizational Unit Name emailAddresscommonName = optional Common Name emailAddress #= Email aAddress is optionalss [ req ] # Optionally, specify some defaults. countryName_default # Request settings= US stateOrProvinceName_default = NH default_bits = 2048 localityName_default # Default key size= Merrimack distinguished_name = req_distinguished_name 0.organizationName_default # Default DN template= Cluster string_mask = utf8only # UTF-8 encoding#organizationalUnitName_default = #emailAddress_default_md = sha256 # Default message digest # Options that are applied when `-extensions v3_ca` is given when creating prompt = no # Non-interactive mode# the root certificate [ req_distinguished_name ] # Template for the DN in the CSRv3_ca ] countryName = USsubjectKeyIdentifier = hash authorityKeyIdentifier = keyid:always,issuer stateOrProvinceName = NHbasicConstraints = critical, CA:true localityName = MerrimackkeyUsage = critical, digitalSignature, cRLSign, keyCertSign # Options that are applied when `-extensions server_cert` is given when signing organizationName = Cluster# server certificates [ v3_ca ] # Root CA certificate extensionsserver_cert ] subjectKeyIdentifier = hash # Subject key identifierbasicConstraints = CA:FALSE authorityKeyIdentifier = keyid:always,issuer # Authority key identifiernsCertType = server basicConstraints = criticalnsComment = "OpenSSL Generated Server Certificate" subjectKeyIdentifier = hash authorityKeyIdentifier = keyid,issuer:always keyUsage = critical, digitalSignature, CA:true # Basic constraints for a CAkeyEncipherment extendedKeyUsage = serverAuth # Options that are applied when `-extensions client_cert` is given when signing keyUsage = critical, keyCertSign, cRLSign # Key usage for a CA# client certificates [ crl_ext ] # CRL extensionsclient_cert ] authorityKeyIdentifier = keyid:always,issuer # Authority key identifibasicConstraints = CA:FALSE nsCertType = client, email nsComment = "OpenSSL Generated Client Certificate" subjectKeyIdentifier = hash authorityKeyIdentifier = keyid,issuer keyUsage = critical, nonRepudiation, digitalSignature, keyEncipherment extendedKeyUsage = clientAuth, emailProtection # Options that are applied when creating CRLs [ crl_ext ] authorityKeyIdentifier = keyid:always ``` == Generate the root CA certificate == Set `days` to something appropriate - this is how many days until the root CA expires. For testing, shorter is fine. For real world use, make it very long. When the root CA expires, everything it signed expires too. Years would be appropriate. Also set `subj` appropriately too. I'm not actually sure if this needs to be set at all given that it's already in the config file You can optionally provide `-subj` on the command line, but if you don't, openssl will prompt you for this information. ``` $ openssl genrsa -out private/ca.key.pem 4096 $ chmod 400 private/ca.key.pem $ openssl req -config openssl.cnf -key private/ca.key.pem -new -x509 -days 100 -sha256 -extensions v3_ca -out certs/ca.cert.pem $ chmod 444 certs/ca.cert.pem -subj "/C=US/ST=NH/L=Merrimack/O=Cluster/CN=Root CA" ``` == Installation == For real world use, keep the private `ca.key.pem` file private. Set ownership and permissions restrictively. Maybe store it on a USB key instead of permanently on the system. The `ca.cert.pem` file should be installed to every client and server system and readable by the cluster user (`hacluster:haclient` most likely). This file should also be referenced in `/etc/sysconfig/pacemaker` as `PCMK_ca_file`. = Generate server stuff = For each server (Pacemaker Remote node or hosts that you want to be able to access via `cibadmin`) do the following: == Generate private key and Certificate Signing Request (CSR) == Replace `<servername>` with the hostname (not FQDN) of the server. You can name it whatever you want, actually, but using the hostname makes keeping track of these things easier if there's a bunch of them. ``` $ openssl genrsa -out ../servers/<servename>.key.pem 4096 $ openssl req -new -key$ chmod 400 ../servers/<servername>.key.pem -out ../servers/<servername>.csr ``` Answer any questions openssl asks. If it asks for a passphrase, don't provide one. This has not been tested yet and it's likely the cluster will hang asking for the passphrase when it tries to read the key. == Create server_cert_ext.cnf == Some things can't be set on the command line so they go in this file instead. ``` basicConstraints = CA:FALSE nsCertType = server nsComment = "OpenSSL Generated Server Certificate" subjectKeyIdentifier = hash authorityKeyIdentifier = keyid,issuer:always keyUsage = critical, digitalSignature, keyEncipherment extendedKeyUsage = $ openssl req -config openssl.cnf -new -key ../servers/<servername>.key.pem -out ../servers/<serverAuthname>.csr ``` Answer any questions openssl asks. If it asks for a passphrase, don't provide one. This has not been tested yet and it's likely the cluster will hang asking for the passphrase when it tries to read the key. When it asks for a Common Name, use the FQDN of the server. Note that the server's Common Name cannot be the same as the root CA's Common Name. == Create and sign the server certificate == Again, change `<servername>` to the hostname and set `days` to something that makes sense. ``` $ openssl x509 -req -in ../servers/<ca -config openssl.cnf -extensions servername>.csr -CA certs/ca.cert.pem -CAkey private/ca.key.pem_cert -days 100 -notext -md sha256 -in ../servers/<servername>.csr -out ../servers/<servername>.cert.pem -CAcreateserial -days 100 -sha256 -extfile server_cert_ext.cnf $ openssl ca -extfile server_cert_ext.cnf -config openssl.cnf -days 100 -md sha256 -out$ chmod 444 ../servers/<servername>.cert.pem -infiles ../servers/<servername>.csr ``` == Installation == Copy `<servername>.cert.pem` and `<servername>.key.pem` to the server. Both should be owned by the cluster user (`hacluster:haclient` most likely). The key should only be readable by the cluster user. I think it's okay if the cert is world-readable. These files should also be referenced in `/etc/sysconfig/pacemaker` as `PCMK_cert_file` and `PCMK_key_file`. = Generate client stuff = Everything is exactly the same on clients, except the paths will be different and there's a different config fileextension to use. I'm not going to cover any of this in depth, just list the commands. Refer to the server section for details. == Generate private key and CSR == ``` $ openssl genrsa -out ../clients/<clientname>.key.pem 4096 $ openssl req -new -key$ chmod 400 ../clients/<clientname>.key.pem -out ../clients/<clientname>.csr ``` == Create client_cert_ext.cnf == ``` basicConstraints = CA:FALSE nsCertType = client, email nsComment = "OpenSSL Generated Client Certificate" subjectKeyIdentifier = hash authorityKeyIdentifier = keyid,issuer keyUsage = critical, nonRepudiation, digitalSignature, keyEncipherment extendedKeyUsage = clientAuth, emailProtection$ openssl req -config openssl.cnf -new -key ../clients/<clientname>.key.pem -out ../clients/<clientname>.csr ``` For clients, the Common Name should be something like an email address. == Create and sign the client certificate == ``` $ openssl x509 -req -in ../clients/<ca -config openssl.cnf -extensions clientname>.csr -CA certs/ca.cert.pem -CAkey private/ca.key.pem_cert -days 100 -notext -md sha256 -in ../clients/<clientname>.csr -out ../clients/<clientname>.cert.pem -CAcreateserial -days 100 -sha256 -extfile client_cert_ext.cnf $ openssl ca -extfile client_cert_ext.cnf -config openssl.cnf -days 100 -md sha256 -out$ chmod 444 ../clients/<clientname>.cert.pem -infiles ../clients/<clientname>.csr ``` == Installation == Installation of the client side is the same as the server side. == Revoking a certificate == If one of the admin machines used to access the CIB remotely is compromised, stolen, or can otherwise not be trusted, a Certificate Revocation List can be used to tell cluster machines to no longer allow use of that certificate. It's also possible to revoke the certificate of a server to prevent clients from accessing a potentially compromised server. In both cases, the commands are the same. ``` $ openssl ca -config openssl.cnf -revoke <client/server>.cert.pem $ openssl ca -config openssl.cnf -gencrl -out crl/rootca.crl $ openssl ca -config openssl.cnf -revoke ../<clients/servers>/<name>.cert.pem ``` CRLs should be regenerated frequently (according to the config file's `default_crl_days` setting, they expire in 30 days) and distributed to all clients and servers. --- Having done all that, using it is a lot less complicated. Pacemaker Administration and Pacemaker Explained cover this in more depth, but the quick explanation is: - For remote CIB operations, the admin machine is the client and the cluster machine is the server. Make sure the server has the server credentials and they are listed in `/etc/sysconfig/pacemaker`. Command line tools (like `cibadmin`, most commonly used for remote CIB operations) do not read the sysconfig file so all the credentials must be passed by the environment. Note that in addition to the TLS parameters, you must still give a username and password. See Pacemaker Administration for more details. - For Pacemaker Remote nodes, the cluster machine running the remote resource is the client and the system running `pacemaker_remoted` is the server. Here, both sides are running a daemon so make sure the appropriate credentials are listed in `/etc/sysconfig/pacemaker` on both ends of the connection. There is no need to pass anything in the environment.