kes

package module
v0.23.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 10, 2023 License: AGPL-3.0 Imports: 35 Imported by: 20

README


KES is a cloud-native distributed key management and encryption server designed to secure modern applications at scale.

What is KES?

KES is a distributed key management server that scales horizontally. It can either be run as edge server close to the applications reducing latency to and load on a central key management system (KMS) or as central key management service. KES nodes are self-contained stateless instances that can be scaled up and down automatically.

Install

The KES server and CLI is available as a single binary, container image or can be build from source.

Homebrew
brew install minio/stable/kes
Docker

Pull the latest release via:

docker pull minio/kes
Binary Releases
OS ARCH Binary
linux amd64 linux-amd64
linux arm64 linux-arm64
darwin arm64 darwin-arm64
windows amd64 windows-amd64

Download the binary via curl but replace <OS> and <ARCH> with your operating system and CPU architecture.

curl -sSL --tlsv1.2 'https://github.com/minio/kes/releases/latest/download/kes-<OS>-<ARCH>' -o ./kes
chmod +x ./kes

You can also verify the binary with minisign by downloading the corresponding .minisig signature file. Run:

curl -sSL --tlsv1.2 'https://github.com/minio/kes/releases/latest/download/kes-<OS>-<ARCH>.minisig' -o ./kes.minisig
minisign -Vm ./kes -P RWTx5Zr1tiHQLwG9keckT0c45M3AGeHD6IvimQHpyRywVWGbP1aVSGav
Build from source

Download and install the binary via your Go toolchain:

go install github.com/minio/kes/cmd/kes@latest

Quick Start

We run a public KES instance at https://play.min.io:7373 as playground. You can interact with our play instance either via the KES CLI or cURL. Alternatively, you can get started by setting up your own KES server in less than five minutes.

First steps
1. Configure CLI

Point the KES CLI to the KES server at https://play.min.io:7373 and use the following API key:

export KES_SERVER=https://play.min.io:7373
export KES_API_KEY=kes:v1:AD9E7FSYWrMD+VjhI6q545cYT9YOyFxZb7UnjEepYDRc
3. Create a Key

Create a new root encryption key - e.g. my-key.

kes key create my-key

Note that creating a new key will fail with key already exist if it already exist.

4. Generate a DEK

Derive a new data encryption keys (DEK).

kes key dek my-key

The plaintext part of the DEK would be used by an application to encrypt some data. The ciphertext part of the DEK would be stored alongside the encrypted data for future decryption.

Docs

If you want to learn more about KES checkout our documentation.

FAQs

I have received an insufficient permissions error

This means that you are using a KES identity that is not allowed to perform a specific operation, like creating or listing keys.

The KES admin identity can perform any general purpose API operation. You should never experience a not authorized: insufficient permissions error when performing general purpose API operations using the admin identity.

In addition to the admin identity, KES supports a policy-based access control model. You will receive a not authorized: insufficient permissions error in the following two cases:

  1. You are using a KES identity that is not assigned to any policy. KES rejects requests issued by unknown identities.

    This can be fixed by assigning a policy to the identity. Checkout the examples.

  2. You are using a KES identity that is assigned to a policy but the policy either not allows or even denies the API call.

    In this case, you have to grant the API permission in the policy assigned to the identity. Checkout the list of APIs. For example, when you want to create a key you should allow the /v1/key/create/<key-name>. The <key-name> can either be a specific key name, like my-key-1 or a pattern allowing arbitrary key names, like my-key*.

    Also note that deny rules take precedence over allow rules. Hence, you have to make sure that any deny pattern does not accidentally matches your API request.


License

Use of KES is governed by the AGPLv3 license that can be found in the LICENSE file.

Documentation

Index

Examples

Constants

View Source
const ServerShutdownTimeout = 1 * time.Second

ServerShutdownTimeout is the default time period the server waits while trying to shutdown gracefully before forcefully closing connections.

Variables

This section is empty.

Functions

This section is empty.

Types

type AuditHandler added in v0.23.0

type AuditHandler interface {
	// Enabled reports whether the handler handles records at
	// the given level. The handler ignores records whose level
	// is lower. It is called early, before an audit record is
	// created, to safe effort if the audit event should be
	// discarded.
	//
	// The Server will pass the request context as the first
	// argument, or context.Background() if no context is
	// available. Enabled may use the context to make a
	// decision.
	Enabled(context.Context, slog.Level) bool

	// Handle handles the AuditRecord. It will only be called when
	// Enabled returns true.
	//
	// The context is present for providing AuditHandlers access
	// to the context's values and to potentially pass it to an
	// underlying slog.Handler. Canceling the context should not
	// affect record processing.
	Handle(context.Context, AuditRecord) error
}

An AuditHandler handles audit records produced by a Server.

A typical handler may print audit records to standard error, or write them to a file or database.

Any of the AuditHandler's methods may be called concurrently with itself or with other methods. It is the responsibility of the Handler to manage this concurrency.

type AuditLogHandler added in v0.23.0

type AuditLogHandler struct {
	Handler slog.Handler
}

AuditLogHandler is an AuditHandler adapter that wraps an slog.Handler. It converts AuditRecords to slog.Records and passes them to the slog.Handler. An AuditLogHandler acts as a bridge between AuditHandlers and slog.Handlers.

Example

This example shows how to connect an AuditHandler to any slog.Handler, here an TextHandler writing to stdout.

package main

import (
	"context"
	"log/slog"
	"net/http"
	"net/netip"
	"os"
	"time"

	"github.com/minio/kes"
)

func main() {
	audit := &kes.AuditLogHandler{
		Handler: slog.NewTextHandler(os.Stdout, nil),
	}
	conf := &kes.Config{
		AuditLog: audit,
	}
	_ = conf

	// Handle will be called by the KES server internally
	audit.Handle(context.Background(), kes.AuditRecord{
		Time:         time.Date(2023, time.October, 19, 8, 44, 0, 0, time.UTC),
		Method:       http.MethodPut,
		Path:         "/v1/key/create/my-key",
		Identity:     "2ecb8804e7702a6b768e89b7bba5933044c9d071e4f4035235269b919e56e691",
		RemoteIP:     netip.MustParseAddr("10.1.2.3"),
		StatusCode:   http.StatusOK,
		ResponseTime: 200 * time.Millisecond,
		Level:        slog.LevelInfo,
		Message:      "secret key 'my-key' created",
	})
}
Output:

time=2023-10-19T08:44:00.000Z level=INFO msg="secret key 'my-key' created" req.method=PUT req.path=/v1/key/create/my-key req.ip=10.1.2.3 req.identity=2ecb8804e7702a6b768e89b7bba5933044c9d071e4f4035235269b919e56e691 res.code=200 res.time=200ms

func (*AuditLogHandler) Enabled added in v0.23.0

func (a *AuditLogHandler) Enabled(ctx context.Context, level slog.Level) bool

Enabled reports whether the AuditLogHandler handles records at the given level. It returns true if the underlying handler returns true.

func (*AuditLogHandler) Handle added in v0.23.0

func (a *AuditLogHandler) Handle(ctx context.Context, r AuditRecord) error

Handle converts the AuditRecord to an slog.Record and passes it to the underlying handler.

type AuditRecord added in v0.23.0

type AuditRecord struct {
	// Point in time when the audit event happened.
	Time time.Time

	// The request HTTP method. (GET, PUT, ...)
	Method string

	// Request URL path. Always starts with a '/'.
	Path string

	// Identity that send the request.
	Identity kes.Identity

	// IP address of the client that sent the request.
	RemoteIP netip.Addr

	// Status code the KES server responded with.
	StatusCode int

	// Amount of time the server took to process the
	// request and generate a response.
	ResponseTime time.Duration

	// The log level of this event.
	Level slog.Level

	// The log message describing the event.
	Message string
}

AuditRecord describes an audit event logged by a KES server.

type CacheConfig added in v0.23.0

type CacheConfig struct {
	// Expiry controls how long a particular key resides
	// in the cache. If zero or negative, keys remain in
	// the cache as long as the KES server has sufficient
	// memory.
	Expiry time.Duration

	// ExpiryUnused is the interval in which a particular
	// key must be accessed to remain in the cache. Keys
	// that haven't been accessed get evicted from the
	// cache. The general cache expiry still applies.
	//
	// ExpiryUnused does nothing if <= 0 or greater than
	// Expiry.
	ExpiryUnused time.Duration

	// ExpiryOffline controls how long a particular key
	// resides in the cache once the key store becomes
	// unavailable. It overwrites Expiry and ExpiryUnused
	// if the key store is not available. Once the key
	// store is available again, Expiry and ExpiryUnused,
	// if set, apply.
	//
	// A common use of ExpiryOffline is reducing the impact
	// of a key store outage, and therefore, improving
	// availability.
	//
	// Offline caching is disabled if ExpiryOffline <= 0.
	ExpiryOffline time.Duration
}

CacheConfig is a structure containing the KES server key store cache configuration.

type Config added in v0.23.0

type Config struct {
	// Admin is the KES server admin identity. It must not be empty.
	// To disable admin access set it to a non-hex value. For example,
	// "disabled".
	Admin kes.Identity

	// TLS contains the KES server's TLS configuration.
	//
	// A KES server requires a TLS certificate. Therefore, either
	// Config.Certificates, Config.GetCertificate or
	// Config.GetConfigForClient must be set.
	//
	// Further, the KES server has to request client certificates
	// for mTLS authentication. Hence, Config.ClientAuth must be
	// at least tls.RequestClientCert.
	TLS *tls.Config

	// Cache specifies how long the KES server caches keys from the
	// KeyStore. If nil, caching is disabled.
	Cache *CacheConfig

	// Policies is a set of policies and identities. Each identity
	// must be assigned to a policy only once.
	Policies map[string]Policy

	// Keys is the KeyStore the KES server fetches keys from.
	Keys KeyStore

	// Routes allows customization of the KES server API routes. It
	// contains a set of API route paths, for example "/v1/status",
	// and the corresponding route configuration.
	//
	// The KES server uses sane defaults for all its API routes.
	Routes map[string]RouteConfig

	// ErrorLog is an optional handler for handling the server's
	// error log events. If nil, defaults to a slog.TextHandler
	// writing to os.Stderr. The server's error log level is
	// controlled by Server.ErrLevel.
	ErrorLog slog.Handler

	// AuditLog is an optional handler for handling the server's
	// audit log events. If nil, defaults to a slog.TextHandler
	// writing to os.Stdout. The server's audit log level is
	// controlled by Server.AuditLevel.
	AuditLog AuditHandler
}

Config is a structure that holds configuration for a KES server.

type Identity

type Identity = kes.Identity

An Identity should uniquely identify a client and is computed from the X.509 certificate presented by the client during the TLS handshake.

type KeyStore added in v0.23.0

type KeyStore interface {
	// Closes the key store and releases associated resources,
	// like background go routines, if any.
	io.Closer

	// Status returns the current state of the KeyStore.
	Status(context.Context) (KeyStoreState, error)

	// Create creates a new entry with the given name if and only
	// if no such entry exists.
	// Otherwise, Create returns kes.ErrKeyExists.
	Create(ctx context.Context, name string, value []byte) error

	// Delete removes the entry. It may return either no error or
	// kes.ErrKeyNotFound if no such entry exists.
	Delete(ctx context.Context, name string) error

	// Get returns the value for the given name. It returns
	// kes.ErrKeyNotFound if no such entry exits.
	Get(ctx context.Context, name string) ([]byte, error)

	// List returns the first n key names, that start with the given
	// prefix, and the next prefix from which the listing should
	// continue.
	//
	// It returns all keys with the prefix if n < 0 and less than n
	// names if n is greater than the number of keys with the prefix.
	//
	// An empty prefix matches any key name. At the end of the listing
	// or when there are no (more) keys starting with the prefix, the
	// returned prefix is empty.
	List(ctx context.Context, prefix string, n int) ([]string, string, error)
}

A KeyStore stores key-value pairs. It provides durable storage for a KES server to persist and access keys. A KeyStore may be modified concurrently by different go routines.

type KeyStoreState added in v0.23.0

type KeyStoreState struct {
	Latency time.Duration
}

KeyStoreState is a structure containing information about the current state of a KeyStore.

type MemKeyStore added in v0.23.0

type MemKeyStore struct {
	// contains filtered or unexported fields
}

MemKeyStore is a volatile KeyStore that stores key-value pairs in memory. Its zero value is ready and safe to be used concurrently from different go routines. It is optimized for reads but not well-suited for many writes/deletes.

func (*MemKeyStore) Close added in v0.23.0

func (ks *MemKeyStore) Close() error

Close does nothing and returns no error.

It is implemented to satisfy the KeyStore interface.

func (*MemKeyStore) Create added in v0.23.0

func (ks *MemKeyStore) Create(_ context.Context, name string, value []byte) error

Create creates a new entry with the given name if and only if no such entry exists. Otherwise, Create returns kes.ErrKeyExists.

func (*MemKeyStore) Delete added in v0.23.0

func (ks *MemKeyStore) Delete(_ context.Context, name string) error

Delete removes the entry. It may return either no error or kes.ErrKeyNotFound if no such entry exists.

func (*MemKeyStore) Get added in v0.23.0

func (ks *MemKeyStore) Get(_ context.Context, name string) ([]byte, error)

Get returns the value for the given name. It returns kes.ErrKeyNotFound if no such entry exits.

func (*MemKeyStore) List added in v0.23.0

func (ks *MemKeyStore) List(_ context.Context, prefix string, n int) ([]string, string, error)

List returns the first n key names that start with the given prefix and the next prefix from which to continue the listing.

It returns all keys with the prefix if n < 0 and less than n names if n is grater than the number of keys with the prefix.

An empty prefix matches any key name. At the end of the listing or when there are no (more) keys starting with the prefix, the returned prefix is empty.

List never returns an error.

func (*MemKeyStore) Status added in v0.23.0

Status returns the current state of the MemKeyStore. It never returns an error.

func (*MemKeyStore) String added in v0.23.0

func (ks *MemKeyStore) String() string

type Policy

type Policy struct {
	Allow map[string]kes.Rule // Set of allow rules

	Deny map[string]kes.Rule // Set of deny rules

	Identities []kes.Identity
}

Policy is a KES policy with associated identities.

A policy contains a set of allow and deny rules.

type RouteConfig added in v0.23.0

type RouteConfig struct {
	// Timeout specifies when the API handler times out.
	//
	// A handler times out when it fails to send the
	// *entire* response body to the client within the
	// given time period.
	//
	// If <= 0, timeouts are disabled for the API route.
	//
	// Disabling timeouts may leave client/server connections
	// hung or allow certain types of denial-of-service (DOS)
	// attacks.
	Timeout time.Duration

	// InsecureSkipAuth, if set, disables authentication for the
	// API route. It allows anyone that can send HTTPS requests
	// to the KES server to invoke the API.
	//
	// For example the KES readiness API authentication may be
	// disabled when the probing clients do not support mTLS
	// client authentication
	//
	// If setting InsecureSkipAuth for any API then clients that
	// do not send a client certificate during the TLS handshake
	// no longer encounter a TLS handshake error but receive a
	// HTTP error instead. In particular, when the server's TLS
	// client auth type has been set  tls.RequireAnyClientCert
	// or tls.RequireAndVerifyClientCert.
	InsecureSkipAuth bool
}

RouteConfig is a structure holding API route configuration.

type Server added in v0.23.0

type Server struct {
	// ShutdownTimeout controls how long Server.Close
	// tries to shutdown the server gracefully without
	// interrupting any active connections.
	//
	// If 0, defaults to ServerShutdownTimeout. If
	// negative, Server.Close waits indefinitely for
	// connections to return to idle and then shut down.
	ShutdownTimeout time.Duration

	// ErrLevel controls which errors are logged by the server.
	// It may be adjusted after the server has been started to
	// change its logging behavior.
	//
	// Log records are passed to the Config.ErrorLog handler
	// if and only if their log level is equal or greater than
	// ErrLevel. A custom Config.ErrorLog may handle records
	// independently from this ErrLevel.
	//
	// Defaults to slog.LevelInfo which includes TLS and HTTP
	// errors when handling requests.
	ErrLevel slog.LevelVar

	// AuditLevel controls which audit events are logged by
	// the server. It may be adjusted after the server has
	// been started to change its logging behavior.
	//
	// Log records are passed to the Config.AuditLog handler
	// if and only if their log level is equal or greater than
	// AuditLevel. A custom Config.AuditLog may handle records
	// independently from this AuditLevel.
	//
	// Defaults to slog.LevelInfo.
	AuditLevel slog.LevelVar
	// contains filtered or unexported fields
}

Server is a KES server.

func (*Server) Addr added in v0.23.0

func (s *Server) Addr() string

Addr returns the server's listener address, or the empty string if the server hasn't been started.

func (*Server) Close added in v0.23.0

func (s *Server) Close() error

Close closes the server and underlying listener. It first tries to shutdown the server gracefully by waiting for requests to finish before closing the server forcefully.

func (*Server) ListenAndStart added in v0.23.0

func (s *Server) ListenAndStart(ctx context.Context, addr string, conf *Config) error

ListenAndStart listens on the TCP network address addr and then calls Start to start the server using the given config. Accepted connections are configured to enable TCP keep-alives.

HTTP/2 support is only enabled if conf.TLS is configured with "h2" in the TLS Config.NextProtos.

ListenAndStart returns once the server is closed or ctx.Done returns, whatever happens first. It returns the first error encountered while shutting down the HTTPS server and closing the listener, if any. It attempts to shutdown the server gracefully by waiting for requests to finish before closing the server forcefully.

func (*Server) Start added in v0.23.0

func (s *Server) Start(ctx context.Context, ln net.Listener, conf *Config) error

Start starts the server using the given config and accepts incoming HTTPS connections on the listener ln, creating a new service goroutine for each.

HTTP/2 support is only enabled if conf.TLS is configured with "h2" in the TLS Config.NextProtos.

Start returns once the server is closed or ctx.Done returns, whatever happens first. It returns the first error encountered while shutting down the HTTPS server and closing the listener, if any. It attempts to shutdown the server gracefully by waiting for requests to finish before closing the server forcefully.

func (*Server) Update added in v0.23.0

func (s *Server) Update(conf *Config) (io.Closer, error)

Update changes the server's configuration. Callers should close the returned io.Closer once they want to releases any resources allocated by the previous configuration, like open file handles or background go routines.

For only changing the server's admin identity, TLS configuration or policies use Server.UpdateAdmin, Server.UpdateTLS or Server.UpdatePolicies. These more specific methods are usually simpler to use and more efficient.

func (*Server) UpdateAdmin added in v0.23.0

func (s *Server) UpdateAdmin(admin kes.Identity) error

UpdateAdmin updates the server's admin identity. All other server configuration options remain unchanged. It returns an error if the server has not been started or has been closed.

func (*Server) UpdatePolicies added in v0.23.0

func (s *Server) UpdatePolicies(policies map[string]Policy) error

UpdatePolicies updates the server policies. All other server configuration options remain unchanged. It returns an error if the server has not been started or has been closed.

func (*Server) UpdateTLS added in v0.23.0

func (s *Server) UpdateTLS(conf *tls.Config) error

UpdateTLS updates the server's TLS configuration. All other server configuration options remain unchanged. It returns an error if the server has not been started or has been closed.

Directories

Path Synopsis
cmd
kes
internal
api
cli
cpu
headers
Package headers defines common HTTP headers.
Package headers defines common HTTP headers.
key
keystore/fs
Package fs implements a key-value store that stores keys as file names and values as file content.
Package fs implements a key-value store that stores keys as file names and values as file content.
keystore/gemalto
Package gemalto implements a key store that fetches/stores cryptographic keys on a Gemalto KeySecure instance.
Package gemalto implements a key store that fetches/stores cryptographic keys on a Gemalto KeySecure instance.
keystore/vault
Package vault implements a secret key store that stores secret keys as key-value entries on the Hashicorp Vault K/V secret backend.
Package vault implements a secret key store that stores secret keys as key-value entries on the Hashicorp Vault K/V secret backend.
sys
yml

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL