Documentation
¶
Index ¶
Constants ¶
const DefaultDialKeepAlive = 30 * time.Second
DefaultDialKeepAlive is used when no Addr2Getter function is provided to control the keep-alive duration for an active connection.
const DefaultDialTimeout = 5 * time.Second
DefaultDialTimeout is used when no Addr2Getter function is provided to control the timeout for establishing a new connection.
const DefaultMaxIdleConnsPerHost = 1
DefaultMaxIdleConnsPerHost is used when no Addr2Getter function is provided to control how many idle connections to keep alive per host.
const DefaultQueryTimeout = 30 * time.Second
DefaultQueryTimeout is used when no Addr2Getter function is provided to control the duration a query will remain in flight prior to automatic cancellation.
Variables ¶
This section is empty.
Functions ¶
func Proxy ¶ added in v2.9.0
func Proxy(config ProxyConfig) error
Proxy creates a proxy http server on the port that proxies range queries to the specified range servers.
Types ¶
type CachingClient ¶
type CachingClient struct {
// contains filtered or unexported fields
}
CachingClient memoizes responses from a Querier.
func (*CachingClient) Close ¶ added in v2.7.1
func (c *CachingClient) Close() error
Close releases all memory and go-routines used by the Simple swarm. If during instantiation, checkVersionPeriodicty was greater than the zero-value for time.Duration, this method may block while completing any in progress updates due to `%version` changes.
func (*CachingClient) Expand ¶ added in v2.10.0
func (c *CachingClient) Expand(query string) (string, error)
Expand returns the response of the query, first checking in the TTL cache, then by actually invoking the Expand method on the underlying Querier.
func (*CachingClient) List ¶ added in v2.10.0
func (c *CachingClient) List(query string) ([]string, error)
List returns the response of the query, first checking in the TTL cache, then by actually invoking the List method on the underlying Querier.
func (*CachingClient) Query ¶
func (c *CachingClient) Query(query string) ([]string, error)
Query returns the response of the query, first checking in the TTL cache, then by actually invoking the Query method on the underlying Querier.
func (*CachingClient) Raw ¶ added in v2.15.0
func (c *CachingClient) Raw(query string) (io.ReadCloser, error)
Raw sends the range request and checks for invalid responses from downstream. If the response is valid, this returns the response body as an io.ReadCloser for the client to use. It is the client's responsibility to invoke the Close method on the returned io.ReadCloser.
type Client ¶
Client attempts to resolve range queries to a list of strings or an error.
func (*Client) Expand ¶ added in v2.10.0
Expand sends the specified query string to the Client's Getter, and converts a non-error result into a slice of bytes.
If the response includes a RangeException header, it returns ErrRangeException. If the status code is not okay, it returns ErrStatusNotOK. Finally, if it cannot parse the lines in the response body, it returns ErrParseException.
// use the range querier result, err := querier.Expand("%someQuery") if err != nil { fmt.Fprintf(os.Stderr, "%s", err) os.Exit(1) } fmt.Printf("%s\n", result)
func (*Client) List ¶ added in v2.10.0
List sends the specified query string to the Client's Getter, and converts a non-error result into a list of strings.
If the response includes a RangeException header, it returns ErrRangeException. If the status code is not okay, it returns ErrStatusNotOK. Finally, if it cannot parse the lines in the response body, it returns ErrParseException.
// use the range querier list, err := querier.List("%someQuery") if err != nil { fmt.Fprintf(os.Stderr, "%s", err) os.Exit(1) } for _, line := range list { fmt.Println(line) }
func (*Client) Query ¶
Query sends the specified query string to the Client's Getter, and converts a non-error result into a list of strings.
If the response includes a RangeException header, it returns ErrRangeException. If the status code is not okay, it returns ErrStatusNotOK. Finally, if it cannot parse the lines in the response body, it returns ErrParseException.
// use the range querier lines, err := querier.Query("%someQuery") if err != nil { fmt.Fprintf(os.Stderr, "%s", err) os.Exit(1) } for _, line := range lines { fmt.Println(line) }
func (*Client) Raw ¶ added in v2.15.0
func (c *Client) Raw(query string) (io.ReadCloser, error)
Raw sends the range request and checks for invalid responses from downstream. If the response is valid, this returns the response body as an io.ReadCloser for the client to use. It is the client's responsibility to invoke the Close method on the returned io.ReadCloser.
type Configurator ¶
type Configurator struct { // Addr2Getter converts a range server address to a Getter, ideally a customized http.Client // object with a Timeout set. Leave nil to create default gogetter.Getter with // DefaultQueryTimeout. Addr2Getter func(string) gogetter.Getter // RetryCallback is predicate function that tests whether query should be retried for a // given error. Leave nil to retry all errors. RetryCallback func(error) bool // RetryCount is number of query retries to be issued if query returns error. Leave 0 to // never retry query errors. RetryCount int // RetryPause is the amount of time to wait before retrying the query with the underlying // Getter. RetryPause time.Duration // Servers is slice of range server address strings. Must contain at least one string. Servers []string // TTL is duration of time to cache query responses. Leave 0 to not cache responses. When a // value is older than its TTL, it becomes stale. When a key is queried for a value that is // stale, an asynchronous routine attempts to lookup the new value, while the existing value // is immediately returned to the user. TTL, TTE, and CheckVersionPeriodicity work together // to prevent frequently needlessly asking servers for information that is still current // while preventing heap build-up on clients. TTL time.Duration // TTE is duration of time before cached response is no longer able to be served, even if // attempts to fetch new value repeatedly fail. This value should be large if your // application needs to still operate even when range servers are down. A zero-value for // this implies that values never expire and can continue to be served. TTL, TTE, and // CheckVersionPeriodicity work together to prevent frequently needlessly asking servers for // information that is still current while preventing heap build-up on clients. TTE time.Duration // CheckVersionPeriodicity is the amount of time between checking the range `%version` // key. If your range server returns the epoch seconds of the time the data set became // active when given the `%version` query, using this option is much better than using just // TTL and TTE. After the specified period of time the CachingClient will query the range // server's `%version` key, and if greater than the value discovered during the previous // check, schedules an asynchronous refresh of all keys last requested by the client less // than the amount of time specified by the TTL from the new version epoch. In other words, // say key A was last requested at time 300, and key B was last requested at time 360. If // the version comes back as 400, and the TTL is 60, then key A will be deleted and B will // be refreshed. It makes no sense for CheckVersionPeriodicity to be a non-zero value when // TTL and TTE are both zero-values. CheckVersionPeriodicity time.Duration }
Configurator provides a way to list the range server addresses, and a way to override defaults when creating new http.Client instances.
type ErrParseException ¶
type ErrParseException struct {
Err error
}
ErrParseException is returned by Client.Query method when an error occurs while parsing the Get response.
func (ErrParseException) Error ¶
func (err ErrParseException) Error() string
type ErrRangeException ¶
type ErrRangeException struct {
Message string
}
ErrRangeException is returned when the response headers includes 'RangeException'.
func (ErrRangeException) Error ¶
func (err ErrRangeException) Error() string
type ErrStatusNotOK ¶
ErrStatusNotOK is returned when the response status code is not Ok.
func (ErrStatusNotOK) Error ¶
func (err ErrStatusNotOK) Error() string
type ProxyConfig ¶ added in v2.9.0
type ProxyConfig struct { // CheckVersionPeriodicity directs the range proxy to periodically send the '%version' query // the proxied range servers, and if the value is greater than the previous value, to // asynchronously update all the values for recently used range keys. CheckVersionPeriodicity time.Duration // Log directs the proxy to emit common log formatted log lines to the specified io.Writer. Log io.Writer // LogFormat specifies the log format to use when Log is not nil. See `gohm` package for // LogFormat specifications. If left blank, uses `gohm.DefaultLogFormat`. LogFormat string // Port specifies which network port the proxy should bind to. Port uint // Servers specifies which addresses ought to be consulted as the source of truth for range // queries. Servers []string // Timeout specifies how long to wait for the source of truth to respond. If the zero-value, // no timeout will be used. Not having a timeout value may cause resource exhaustion where // any of the proxied servers take too long to return a response. Timeout time.Duration // TTE is duration of time before cached response is no longer able to be served, even if // attempts to fetch new value repeatedly fail. This value should be large if your application // needs to still operate even when range servers are down. A zero-value for this implies that // values never expire and can continue to be served. TTE and CheckVersionPeriodicity work // together to prevent frequently needlessly asking servers for information that is still // current while preventing heap build-up on clients. TTE time.Duration }
ProxyConfig specifies the configuration for a gorange proxy HTTP server.
type Querier ¶
type Querier interface { Close() error Expand(string) (string, error) List(string) ([]string, error) Query(string) ([]string, error) Raw(string) (io.ReadCloser, error) }
Querier is the interface implemented by an object that allows key-value lookups, where keys are strings and values are slices of strings.
func NewQuerier ¶
func NewQuerier(config *Configurator) (Querier, error)
NewQuerier returns a new instance that sends queries to one or more range servers. The provided Configurator not only provides a way of listing one or more range servers, but also allows specification of optional retry-on-failure feature and optional TTL cache that memoizes range query responses.
func main() { servers := []string{"range1.example.com", "range2.example.com", "range3.example.com"} config := &gorange.Configurator{ RetryCount: len(servers), RetryPause: 5 * time.Second, Servers: servers, CheckVersionPeriodicity: 15 * time.Second, TTL: 30 * time.Second, TTE: 15 * time.Minute, } // create a range querier; could list additional servers or include other options as well querier, err := gorange.NewQuerier(config) if err != nil { fmt.Fprintf(os.Stderr, "%s", err) os.Exit(1) } // must invoke Close method when finished using to prevent resource leakage defer func() { _ = querier.Close() }() }