Fluxtail
Log Management Guides

What Is Public Key Cryptography? Keys, Signatures, TLS

Learn how public and private keys support encryption, signatures, and key agreement, and how TLS, SSH, and certificates establish trust.

By Fluxtail Engineering Updated

Public key cryptography—also called asymmetric cryptography—uses a related pair of keys: a public key that can be shared and a private key that must be protected. Depending on the algorithm and protocol, the pair can help encrypt data, verify a digital signature, or establish a shared secret. It does not make an unknown public key trustworthy by itself. Before relying on a key, you must know whose key it is. NIST's glossary defines the key-pair relationship and notes that deriving a private key from its public key is computationally infeasible with an appropriate scheme.

The public and private keys are related, but they are not two copies of the same secret. A public key may be published in a certificate or supplied to a signature verifier. A private key stays with its owner or in a controlled signing device.

What can a public/private key pair do?

These are three distinct jobs. A particular key type and protocol may support one of them, not all three.

Job What the public and private material do What it provides
Public-key encryption A sender encrypts to a recipient's public key; the matching private key is needed to decrypt Confidentiality for the intended recipient, if the key really belongs to them
Digital signature A signer uses a private key; others verify with the public key Evidence of who held the signing key and that the signed data was not changed
Key agreement Each side contributes secret and public material to derive a shared secret A basis for symmetric session keys; peer identity still needs authentication

For encryption, imagine sending a document to Mira. If you encrypt it using Mira's verified encryption public key, her private key can decrypt it. If an attacker substituted their own public key while claiming it was Mira's, the encryption could be mathematically sound but protect the document for the attacker instead. Key authenticity is therefore part of the security design, not an optional finishing touch.

For a digital signature, Mira signs a specific message using a signing private key. A verifier checks the signature with Mira's public key. Successful verification means the message matches the signature and the corresponding private key was used. It does not establish who Mira is unless the verifier already has a trusted way to associate that public key with Mira. Nor does a signature hide the message: signing is not encryption.

In key agreement, such as an ephemeral Diffie–Hellman exchange, the parties exchange public contributions and derive matching secret material without sending the resulting secret directly. An unauthenticated exchange can still be intercepted by an active attacker who substitutes their own contributions. Protocols such as TLS and SSH add authentication to make the exchange meaningful. NIST's key-management guidance treats encryption, signatures, key establishment, and key protection as separate functions.

This is why “encrypt with the public key, decrypt with the private key” is an incomplete definition. It describes one possible asymmetric operation, not every public-key algorithm and not the whole of a modern secure connection. For example, a signature-only key is not automatically a key that can encrypt, and an agreement key is not automatically a signing key. Choose keys according to the operation and protocol rather than expecting one pair to do everything.

How TLS 1.3 uses public keys

When a browser connects to a typical HTTPS server using certificate-based TLS 1.3, the handshake has two related tasks: establish fresh shared key material and authenticate the server. The client and server exchange ephemeral key shares for key establishment. The server sends a certificate containing an authentication public key and proves possession of the corresponding private key by signing the handshake transcript in CertificateVerify. The client validates that certificate and signature before treating the server as authenticated. The resulting traffic keys protect application data with symmetric cryptography, not by public-key-encrypting each web response. This is the flow specified in RFC 8446.

The certificate key and the ephemeral agreement keys have different jobs. A server can have an RSA certificate for signing while the connection uses ephemeral Diffie–Hellman for key establishment. TLS 1.3 does not use the old static RSA key-transport mechanism. Also, TLS 1.3 has pre-shared-key and resumption modes, so not every connection repeats the full certificate-based flow; the description above is the ordinary full certificate-authenticated case.

What if the certificate is expired, issued by an untrusted authority, or valid for another hostname? A client that correctly checks the server's identity should reject the connection even if the cryptographic key exchange itself could produce encrypted traffic. Encryption alone would protect a conversation with someone; validation helps establish that the someone is the server requested. The difference matters when diagnosing an SSL/TLS handshake failure: a trust-chain or name error is not fixed by merely enabling more encryption.

How SSH uses public keys

SSH also separates the identity of the server from the identity of the user. During SSH transport setup, the server presents a host key and proves possession of its private key as part of the key exchange. The client must decide whether that host key belongs to the intended server. It may compare the key against a previously trusted record or another verified source. Accepting an unfamiliar key without checking it provides a secure channel to an unknown machine, not reliable server authentication. RFC 4253 explicitly warns about this distinction.

After the transport is established, SSH can use public-key user authentication. The client signs authentication data with the user's private key; the server checks that signature against a public key it accepts for that user. This is separate from checking the server's host key. A correct host key does not automatically grant the user access, and a valid user key should not make an unverified server safe to connect to. RFC 4252 defines the public-key authentication method and permits a server to require additional authentication.

SSH does not require an X.509 web certificate for every host. Many deployments use locally managed trusted host keys. The trust question remains the same: where did the expected public key come from, and what should happen if it changes? A changed host key may be legitimate after a rebuild, but it should be checked through an independent, trusted channel before it is accepted.

What a certificate adds to a public key

A raw public key is just key material. An X.509 certificate adds identity information, validity dates, and an issuer's signature; extensions can also specify permitted uses. In a public web PKI, a certificate authority (CA) signs a server certificate; a client attempts to build a valid path to a trust anchor it accepts. It then checks that the certificate is suitable for the connection, including the requested name. RFC 5280 specifies the certificate profile and path-validation model; RFC 9525 specifies how TLS clients verify the service name.

A CA signature is not a magic guarantee that the endpoint is safe. The client must trust the right CA, perform the required name and usage checks, and protect its own trust store. A certificate can be validly signed yet be wrong for the hostname the user requested. Conversely, a self-signed certificate is not inherently useless: it can be trusted when its key is intentionally distributed and pinned through a secure process. The question is whether the relying party has a trustworthy binding from identity to public key.

Certificates also have a lifecycle. They expire, keys may be replaced, and a compromised private key requires response. Revocation information can help, but do not assume every client checks it the same way or receives fresh information every time. RFC 5280's security considerations note that stale or unavailable revocation information reduces assurance. Keep certificate inventory and renewal procedures, but validate what your actual clients enforce before claiming a revocation control works end to end.

Inspect a certificate without exposing a private key

To inspect a local PEM certificate without changing it, run:

openssl x509 -in certificate.pem -noout -subject -issuer -dates -ext subjectAltName

Replace certificate.pem with the path to a certificate file, not a private-key file. The command prints the subject, issuer, validity dates, and subject alternative names if present. The OpenSSL x509 manual documents these inspection flags. This is a read-only inspection of one file, not a certificate-path, hostname, or revocation validation. It also says nothing about which certificate a live server currently presents unless the inspected file is known to be that server's active certificate.

An unexpected issuer may indicate that the wrong certificate was deployed. An expired notAfter date is a renewal problem. A missing expected DNS name is an identity mismatch. Do not “fix” these by turning off client verification; correct the certificate, chain, trust configuration, or target name.

Protect the private key and the trust decision

Treat a private key as a credential. Give access only to the service or person that needs it; do not paste it into tickets, commands, logs, or a public repository. Where a key can be kept in hardware-backed storage and used without export, that can reduce exposure, but the access policy and recovery process still matter. NIST SP 800-57 Part 1 covers protection, usage periods, compromise, and other key-management duties.

Plan how to replace keys and certificates before an emergency. If a private key may have leaked, stop using it, issue replacement material, update the systems that trust it, and investigate what it protected. The exact response depends on whether the key signed software, authenticated an SSH host, or served TLS traffic; one blanket “rotate the certificate” instruction is not enough for every case. Keep public keys verifiable through a trusted channel, because a perfectly protected private key does not help if clients accept an attacker's public key instead.

If application, proxy, or SSH logs are sent to Fluxtail, its log search can help correlate handshake and authentication errors with a deployment or certificate change. It does not validate certificates or manage keys for you. For the broader logging path, see log management; for a connection that fails before HTTP begins, start with the TLS handshake checks.