Documentation ¶
Overview ¶
Package overseer ; cmd runs external commands with concurrent access to output and status. It wraps the Go standard library os/exec.Command to correctly handle reading output (STDOUT and STDERR) while a command is running and killing a command. All operations are safe to call from multiple goroutines.
Credit: https://github.com/go-cmd/cmd Copyright (c) 2017 go-cmd & contribuitors The architecture is quite heavily modified from the original version
A basic example that runs env and prints its output:
import ( "fmt" cmd "https://github.com/ShinyTrinkets/overseer" ) func main() { // Create a Cmd envCmd := cmd.NewCmd("env") // Run and wait for Cmd to return Status status := <-envCmd.Start() // Print each line of STDOUT from Cmd for _, line := range status.Stdout { fmt.Println(line) } }
Commands can be ran synchronously (blocking) or asynchronously (non-blocking):
envCmd := cmd.NewCmd("env") // create status := <-envCmd.Start() // run blocking statusChan := envCmd.Start() // run non-blocking // Do other work while Cmd is running... status <- statusChan // blocking
Start returns a channel to which the final Status is sent when the command finishes for any reason. The first example blocks receiving on the channel. The second example is non-blocking because it saves the channel and receives on it later. Only one final status is sent to the channel; use Done for multiple goroutines to wait for the command to finish, then call Status to get the final status.
Package overseer ;
Index ¶
- Constants
- func SetupLogBuilder(b ml.LogBuilderType)
- type Attrs
- type Backoff
- type Cmd
- func (c *Cmd) Clone() *Cmd
- func (c *Cmd) Done() <-chan struct{}
- func (c *Cmd) IsFinalState() bool
- func (c *Cmd) IsInitialState() bool
- func (c *Cmd) IsRunningState() bool
- func (c *Cmd) Signal(sig syscall.Signal) error
- func (c *Cmd) Start() <-chan Status
- func (c *Cmd) Status() Status
- func (c *Cmd) Stop() error
- type CmdState
- type ErrLineBufferOverflow
- type LogMsg
- type Logger
- type Options
- type OutputBuffer
- type OutputStream
- type Overseer
- func (ovr *Overseer) Add(id string, exec string, args ...interface{}) *Cmd
- func (ovr *Overseer) HasProc(id string) bool
- func (ovr *Overseer) IsRunning() bool
- func (ovr *Overseer) IsStopping() bool
- func (ovr *Overseer) ListAll() []string
- func (ovr *Overseer) ListGroup(group string) []string
- func (ovr *Overseer) Remove(id string) bool
- func (ovr *Overseer) Signal(id string, sig syscall.Signal) error
- func (ovr *Overseer) Status(id string) *ProcessJSON
- func (ovr *Overseer) Stop(id string) error
- func (ovr *Overseer) StopAll(kill bool)
- func (ovr *Overseer) Supervise(id string) int
- func (ovr *Overseer) SuperviseAll()
- func (ovr *Overseer) UnWatchLogs(logChan chan *LogMsg)
- func (ovr *Overseer) UnWatchState(outputChan chan *ProcessJSON)
- func (ovr *Overseer) WatchLogs(logChan chan *LogMsg)
- func (ovr *Overseer) WatchState(outputChan chan *ProcessJSON)
- type OvrState
- type ProcessJSON
- type Status
Constants ¶
const ( // DEFAULT_LINE_BUFFER_SIZE is the default size of the OutputStream line buffer. // The default value is usually sufficient, but if ErrLineBufferOverflow errors // occur, try increasing the size by calling OutputBuffer.SetLineBufferSize. DEFAULT_LINE_BUFFER_SIZE = 16384 // DEFAULT_STREAM_CHAN_SIZE is the default string channel size for a Cmd when // Options.Streaming is true. The string channel size can have a minor // performance impact if too small by causing OutputStream.Write to block // excessively. DEFAULT_STREAM_CHAN_SIZE = 1000 )
const ( STDOUT uint8 = 1 STDERR uint8 = 2 )
const ( INITIAL = iota IDLE STARTING RUNNING STOPPING INTERRUPT // final state (used when stopped or signaled) FINISHED // final state (used then was a natural exit) FATAL // final state (used when was an error while starting) )
Variables ¶
This section is empty.
Functions ¶
func SetupLogBuilder ¶ added in v0.3.0
func SetupLogBuilder(b ml.LogBuilderType)
SetupLogBuilder is called to add user's provided log builder. It's just a wrapper around meta-logger SetupLogBuilder.
Types ¶
type Backoff ¶
type Backoff struct { // Factor is the multiplying factor for each increment step. // // Defaults to 2. Factor float64 // Jitter eases contention by randomizing backoff steps. // // Defaults to false. Jitter bool // Minimum value of the counter. // // Defaults to 100 milliseconds. Min time.Duration // Maximum value of the counter. // // Defaults to 10 seconds. Max time.Duration // contains filtered or unexported fields }
Backoff is a time.Duration counter, starting at Min. After every call to the Duration method the current timing is multiplied by Factor, but it never exceeds Max.
Backoff is not generally concurrent-safe, but the ForAttempt method can be used concurrently.
func (*Backoff) Duration ¶
Duration returns the duration for the current attempt before incrementing the attempt counter. See ForAttempt.
func (*Backoff) ForAttempt ¶
ForAttempt returns the duration for a specific attempt. This is useful if you have a large number of independent Backoffs, but don't want use unnecessary memory storing the Backoff parameters per Backoff. The first attempt should be 0.
ForAttempt is concurrent-safe.
type Cmd ¶
type Cmd struct { *sync.Mutex Name string // Name of binary (command) to run; required! Group string // Used by the manager (optional) Args []string // Commands line arguments passed to the command (optional) Env []string // Environment variables set before running the command (optional) Dir string // Current working directory from which to run the command (optional) DelayStart uint // Nr of milli-seconds to delay the start (used by the manager) RetryTimes uint // Nr of times to restart on failure (used by the manager) Stdout chan string // streaming STDOUT if enabled, else nil (see Options) Stderr chan string // streaming STDERR if enabled, else nil (see Options) State CmdState // The state of the cmd (stopped, started, etc) // contains filtered or unexported fields }
Cmd represents an external command, similar to the Go built-in os/exec.Cmd. A Cmd cannot be reused after calling Start, but it can be cloned with Clone. To create a new Cmd, call NewCmd.
func NewCmd ¶
NewCmd creates a new Cmd for the given command name and arguments. The command is not started until Start is called.
func (*Cmd) Clone ¶ added in v0.3.0
Clone clones a Cmd. All the options are transferred, but the state of the original object is lost. Cmd is one-use only, so if you need to re-start a Cmd, you need to Clone it.
func (*Cmd) Done ¶
func (c *Cmd) Done() <-chan struct{}
Done returns a channel that's closed when the command stops running. This method is useful for multiple goroutines to wait for the command to finish. Call Status after the command finishes to get its final status.
func (*Cmd) IsFinalState ¶
IsFinalState returns true if the Cmd is in a final state. Final states are definitive and cannot be exited from.
func (*Cmd) IsInitialState ¶ added in v0.3.0
IsInitialState returns true if the Cmd is in the initial state.
func (*Cmd) IsRunningState ¶ added in v0.3.4
IsRunningState returns true if the Cmd is starting, or running.
func (*Cmd) Start ¶
Start starts the command and immediately returns a channel that the caller can use to receive the final Status of the command when it ends. The caller can start the command and wait like,
status := <-myCmd.Start() // blocking
or start the command asynchronously and be notified later when it ends,
statusChan := myCmd.Start() // non-blocking // Do other work while Cmd is running... status := <-statusChan // blocking
Exactly one Status is sent on the channel when the command ends. The channel is not closed. Any Go error is set to Status.Error. Start is idempotent; it always returns the same channel.
func (*Cmd) Status ¶
Status returns the Status of the command at any time. It is safe to call concurrently by multiple goroutines.
With buffered output, Status.Stdout and Status.Stderr contain the full output as of the Status call time. For example, if the command counts to 3 and three calls are made between counts, Status.Stdout contains:
"1" "1 2" "1 2 3"
The caller is responsible for tailing the buffered output if needed. Else, consider using streaming output. When the command finishes, buffered output is complete and final.
func (*Cmd) Stop ¶
Stop stops the command by sending its process group a SIGTERM signal. Stop makes sure the command doesn't restart, by resetting the retry times. Stop is idempotent. An error should only be returned in the rare case that Stop is called immediately after the command ends but before Start can update its internal state.
type ErrLineBufferOverflow ¶
type ErrLineBufferOverflow struct { Line string // Unterminated line that caused the error BufferSize int // Internal line buffer size BufferFree int // Free bytes in line buffer }
ErrLineBufferOverflow is returned by OutputStream.Write when the internal line buffer is filled before a newline character is written to terminate a line. Increasing the line buffer size by calling OutputStream.SetLineBufferSize can help prevent this error.
func (ErrLineBufferOverflow) Error ¶
func (e ErrLineBufferOverflow) Error() string
type Options ¶
type Options struct { Group string Dir string Env []string DelayStart uint RetryTimes uint // If Buffered is true, STDOUT and STDERR are written to Status.Stdout and // Status.Stderr. The caller can call Cmd.Status to read output at intervals. // See Cmd.Status for more info. Buffered bool // If Streaming is true, Cmd.Stdout and Cmd.Stderr channels are created and // STDOUT and STDERR output lines are written them in real time. This is // faster and more efficient than polling Cmd.Status. The caller must read both // streaming channels, else lines are dropped silently. Streaming bool // LineBufferSize sets the size of the OutputStream line buffer. The default // value DEFAULT_LINE_BUFFER_SIZE is usually sufficient, but if // ErrLineBufferOverflow errors occur, try increasing the size with this // field. LineBufferSize uint }
Options represents customizations for NewCmd.
type OutputBuffer ¶
OutputBuffer represents command output that is saved, line by line, in an unbounded buffer. It is safe for multiple goroutines to read while the command is running and after it has finished. If output is small (a few megabytes) and not read frequently, an output buffer is a good solution.
A Cmd in this package uses an OutputBuffer for both STDOUT and STDERR by default when created by calling NewCmd. To use OutputBuffer directly with a Go standard library os/exec.Command:
import "os/exec" import cmd "github.com/ShinyTrinkets/overseer" runnableCmd := exec.Command(...) stdout := cmd.NewOutputBuffer() runnableCmd.Stdout = stdout
While runnableCmd is running, call stdout.Lines() to read all output currently written.
func NewOutputBuffer ¶
func NewOutputBuffer() *OutputBuffer
NewOutputBuffer creates a new output buffer. The buffer is unbounded and safe for multiple goroutines to read while the command is running by calling Lines.
func (*OutputBuffer) Lines ¶
func (rw *OutputBuffer) Lines() []string
Lines returns lines of output written by the Cmd. It is safe to call while the Cmd is running and after it has finished. Subsequent calls returns more lines, if more lines were written. "\r\n" are stripped from the lines.
type OutputStream ¶
type OutputStream struct {
// contains filtered or unexported fields
}
OutputStream represents real time, line by line output from a running Cmd. Lines are terminated by a single newline preceded by an optional carriage return. Both newline and carriage return are stripped from the line when sent to a caller-provided channel.
The caller must begin receiving before starting the Cmd. Write blocks on the channel; the caller must always read the channel. The channel is closed when the Cmd exits and all output has been sent.
A Cmd in this package uses an OutputStream for both STDOUT and STDERR when created by calling NewCmdOptions and Options.Streaming is true. To use OutputStream directly with a Go standard library os/exec.Command:
import "os/exec" import cmd "github.com/ShinyTrinkets/overseer" stdoutChan := make(chan string, 100) go func() { for line := range stdoutChan { // Do something with the line } }() runnableCmd := exec.Command(...) stdout := cmd.NewOutputStream(stdoutChan) runnableCmd.Stdout = stdout
While runnableCmd is running, lines are sent to the channel as soon as they are written and newline-terminated by the command.
func NewOutputStream ¶
func NewOutputStream(streamChan chan string) *OutputStream
NewOutputStream creates a new streaming output on the given channel. The caller must begin receiving on the channel before the command is started. The OutputStream never closes the channel.
func (*OutputStream) Flush ¶ added in v0.5.0
func (rw *OutputStream) Flush()
Flush empties the buffer of its last line.
func (*OutputStream) Lines ¶
func (rw *OutputStream) Lines() <-chan string
Lines returns the channel to which lines are sent. This is the same channel passed to NewOutputStream.
func (*OutputStream) SetLineBufferSize ¶
func (rw *OutputStream) SetLineBufferSize(n int)
SetLineBufferSize sets the internal line buffer size. The default is DEFAULT_LINE_BUFFER_SIZE. This function must be called immediately after NewOutputStream, and it is not safe to call by multiple goroutines.
Increasing the line buffer size can help reduce ErrLineBufferOverflow errors.
type Overseer ¶
type Overseer struct {
// contains filtered or unexported fields
}
Overseer structure. For instantiating, it's best to use the NewOverseer() function.
func NewOverseer ¶
func NewOverseer() *Overseer
NewOverseer creates a new process manager. After creating it, Add the procs and call SuperviseAll.
func (*Overseer) IsStopping ¶ added in v0.3.3
IsStopping returns True if StopAll was called and the procs are preparing to close
func (*Overseer) ListGroup ¶ added in v0.3.0
ListGroup returns the names of all the procs from a specific group.
func (*Overseer) Status ¶
func (ovr *Overseer) Status(id string) *ProcessJSON
Status returns a child process status, ready to be converted to JSON. Use this after calling ListAll() or ListGroup() (PID, Exit code, Error, Runtime seconds, Stdout, Stderr)
func (*Overseer) Stop ¶
Stop the process by sending its process group a SIGTERM signal. The process can be started again, if needed.
func (*Overseer) SuperviseAll ¶
func (ovr *Overseer) SuperviseAll()
SuperviseAll is the *main* function. It supervises all registered processes and waits for them to finish.
func (*Overseer) UnWatchLogs ¶ added in v0.5.0
UnWatchLogs allows un-subscribing from log messages.
func (*Overseer) UnWatchState ¶ added in v0.5.0
func (ovr *Overseer) UnWatchState(outputChan chan *ProcessJSON)
UnWatchState allows un-subscribing from state changes.
func (*Overseer) WatchLogs ¶ added in v0.5.0
WatchLogs allows subscribing to log messages via provided output channel.
func (*Overseer) WatchState ¶ added in v0.5.0
func (ovr *Overseer) WatchState(outputChan chan *ProcessJSON)
WatchState allows subscribing to state changes via provided output channel.
type ProcessJSON ¶ added in v0.3.0
type ProcessJSON struct { ID string `json:"id"` Group string `json:"group"` Cmd string `json:"cmd"` Dir string `json:"dir"` PID int `json:"PID"` State string `json:"state"` ExitCode int `json:"exitCode"` // exit code of process Error error `json:"error"` // Go error StartTime time.Time `json:"startTime"` DelayStart uint `json:"delayStart"` RetryTimes uint `json:"retryTimes"` }
ProcessJSON public structure
type Status ¶
type Status struct { Cmd string PID int Exit int // exit code of process Error error // Go error StartTs int64 // Unix ts (nanoseconds), zero if Cmd not started StopTs int64 // Unix ts (nanoseconds), zero if Cmd not started or running Runtime float64 // seconds, zero if Cmd not started Stdout []string // buffered STDOUT; see Cmd.Status for more info Stderr []string // buffered STDERR; see Cmd.Status for more info }
Status represents the running status and consolidated return of a Cmd. It can be obtained any time by calling Cmd.Status. If StartTs > 0, the command has started. If StopTs > 0, the command has stopped. After the command finishes for any reason, this combination of values indicates success (presuming the command only exits zero on success):
Exit = 0 Error = nil
Error is a Go error from the underlying os/exec.Cmd.Start or os/exec.Cmd.Wait. If not nil, the command either failed to start (it never ran) or it started but was terminated unexpectedly (probably signaled). In either case, the command failed. Callers should check Error first. If nil, then check Exit and Status.