Documentation
¶
Index ¶
- Constants
- Variables
- func AssertTrue(b bool)
- func AssertTruef(b bool, format string, args ...interface{})
- func Check(err error)
- func Check2(_ interface{}, err error)
- func CompareKeys(key1, key2 []byte) int
- func Copy(a []byte) []byte
- func CreateSyncedFile(filename string, sync bool) (*os.File, error)
- func FileSync(f *os.File) error
- func FixedDuration(d time.Duration) string
- func KeyWithTs(key []byte, ts uint64) []byte
- func Madvise(b []byte, readahead bool) error
- func Mmap(fd *os.File, writable bool, size int64) ([]byte, error)
- func Munmap(b []byte) error
- func OpenExistingFile(filename string, flags uint32) (*os.File, error)
- func OpenSyncedFile(filename string, sync bool) (*os.File, error)
- func OpenTruncFile(filename string, sync bool) (*os.File, error)
- func ParseKey(key []byte) []byte
- func ParseTs(key []byte) uint64
- func SafeCopy(a, src []byte) []byte
- func SameKey(src, dst []byte) bool
- func Wrap(err error) error
- func Wrapf(err error, format string, args ...interface{}) error
- type Closer
- type Iterator
- type MergeIterator
- type Slice
- type Throttle
- type ValueStruct
- type WaterMark
- func (w *WaterMark) Begin(index uint64)
- func (w *WaterMark) BeginMany(indices []uint64)
- func (w *WaterMark) Done(index uint64)
- func (w *WaterMark) DoneMany(indices []uint64)
- func (w *WaterMark) DoneUntil() uint64
- func (w *WaterMark) Init(closer *Closer)
- func (w *WaterMark) LastIndex() uint64
- func (w *WaterMark) SetDoneUntil(val uint64)
- func (w *WaterMark) WaitForMark(ctx context.Context, index uint64) error
Constants ¶
const ( // Sync indicates that O_DSYNC should be set on the underlying file, // ensuring that data writes do not return until the data is flushed // to disk. Sync = 1 << iota // ReadOnly opens the underlying file on a read-only basis. ReadOnly )
Variables ¶
var ( // LSMSize has size of the LSM in bytes LSMSize *expvar.Map // VlogSize has size of the value log in bytes VlogSize *expvar.Map // PendingWrites tracks the number of pending writes. PendingWrites *expvar.Map // NumReads has cumulative number of reads NumReads *expvar.Int // NumWrites has cumulative number of writes NumWrites *expvar.Int // NumBytesRead has cumulative number of bytes read NumBytesRead *expvar.Int // NumBytesWritten has cumulative number of bytes written NumBytesWritten *expvar.Int // NumLSMGets is number of LMS gets NumLSMGets *expvar.Map // NumLSMBloomHits is number of LMS bloom hits NumLSMBloomHits *expvar.Map // NumGets is number of gets NumGets *expvar.Int // NumPuts is number of puts NumPuts *expvar.Int // NumBlockedPuts is number of blocked puts NumBlockedPuts *expvar.Int // NumMemtableGets is number of memtable gets NumMemtableGets *expvar.Int )
var ( // CastagnoliCrcTable is a CRC32 polynomial table CastagnoliCrcTable = crc32.MakeTable(crc32.Castagnoli) )
var ErrEOF = errors.New("End of mapped region")
ErrEOF indicates an end of file when trying to read from a memory mapped file and encountering the end of slice.
Functions ¶
func AssertTrue ¶
func AssertTrue(b bool)
AssertTrue asserts that b is true. Otherwise, it would log fatal.
func AssertTruef ¶
AssertTruef is AssertTrue with extra info.
func Check2 ¶
func Check2(_ interface{}, err error)
Check2 acts as convenience wrapper around Check, using the 2nd argument as error.
func CompareKeys ¶
CompareKeys checks the key without timestamp and checks the timestamp if keyNoTs is same. a<timestamp> would be sorted higher than aa<timestamp> if we use bytes.compare All keys should have timestamp.
func CreateSyncedFile ¶
CreateSyncedFile creates a new file (using O_EXCL), errors if it already existed.
func FileSync ¶
FileSync calls os.File.Sync with the right parameters. This function can be removed once we stop supporting Go 1.11 on MacOS.
More info: https://golang.org/issue/26650.
func FixedDuration ¶
FixedDuration returns a string representation of the given duration with the hours, minutes, and seconds.
func Madvise ¶
Madvise uses the madvise system call to give advise about the use of memory when using a slice that is memory-mapped to a file. Set the readahead flag to false if page references are expected in random order.
func Mmap ¶
Mmap uses the mmap system call to memory-map a file. If writable is true, memory protection of the pages is set so that they may be written to as well.
func OpenExistingFile ¶
OpenExistingFile opens an existing file, errors if it doesn't exist.
func OpenSyncedFile ¶
OpenSyncedFile creates the file if one doesn't exist.
func OpenTruncFile ¶
OpenTruncFile opens the file with O_RDWR | O_CREATE | O_TRUNC
Types ¶
type Closer ¶
type Closer struct {
// contains filtered or unexported fields
}
Closer holds the two things we need to close a goroutine and wait for it to finish: a chan to tell the goroutine to shut down, and a WaitGroup with which to wait for it to finish shutting down.
func (*Closer) AddRunning ¶
AddRunning Add()'s delta to the WaitGroup.
func (*Closer) HasBeenClosed ¶
func (lc *Closer) HasBeenClosed() <-chan struct{}
HasBeenClosed gets signaled when Signal() is called.
func (*Closer) SignalAndWait ¶
func (lc *Closer) SignalAndWait()
SignalAndWait calls Signal(), then Wait().
type Iterator ¶
type Iterator interface { Next() Rewind() Seek(key []byte) Key() []byte Value() ValueStruct Valid() bool // All iterators should be closed so that file garbage collection works. Close() error }
Iterator is an interface for a basic iterator.
type MergeIterator ¶
type MergeIterator struct {
// contains filtered or unexported fields
}
MergeIterator merges multiple iterators. NOTE: MergeIterator owns the array of iterators and is responsible for closing them.
func NewMergeIterator ¶
func NewMergeIterator(iters []Iterator, reversed bool) *MergeIterator
NewMergeIterator returns a new MergeIterator from a list of Iterators.
func (*MergeIterator) Key ¶
func (s *MergeIterator) Key() []byte
Key returns the key associated with the current iterator
func (*MergeIterator) Next ¶
func (s *MergeIterator) Next()
Next returns the next element. If it is the same as the current key, ignore it.
func (*MergeIterator) Rewind ¶
func (s *MergeIterator) Rewind()
Rewind seeks to first element (or last element for reverse iterator).
func (*MergeIterator) Seek ¶
func (s *MergeIterator) Seek(key []byte)
Seek brings us to element with key >= given key.
func (*MergeIterator) Valid ¶
func (s *MergeIterator) Valid() bool
Valid returns whether the MergeIterator is at a valid element.
func (*MergeIterator) Value ¶
func (s *MergeIterator) Value() ValueStruct
Value returns the value associated with the iterator.
type Slice ¶
type Slice struct {
// contains filtered or unexported fields
}
Slice holds a reusable buf, will reallocate if you request a larger size than ever before. One problem is with n distinct sizes in random order it'll reallocate log(n) times.
type Throttle ¶
type Throttle struct {
// contains filtered or unexported fields
}
Throttle allows a limited number of workers to run at a time. It also provides a mechanism to check for errors encountered by workers and wait for them to finish.
func NewThrottle ¶
NewThrottle creates a new throttle with a max number of workers.
func (*Throttle) Do ¶
Do should be called by workers before they start working. It blocks if there are already maximum number of workers working. If it detects an error from previously Done workers, it would return it.
func (*Throttle) Done ¶
Done should be called by workers when they finish working. They can also pass the error status of work done.
type ValueStruct ¶
type ValueStruct struct { Meta byte UserMeta byte ExpiresAt uint64 Value []byte Version uint64 // This field is not serialized. Only for internal usage. }
ValueStruct represents the value info that can be associated with a key, but also the internal Meta field.
func (*ValueStruct) Decode ¶
func (v *ValueStruct) Decode(b []byte)
Decode uses the length of the slice to infer the length of the Value field.
func (*ValueStruct) Encode ¶
func (v *ValueStruct) Encode(b []byte)
Encode expects a slice of length at least v.EncodedSize().
func (*ValueStruct) EncodeTo ¶
func (v *ValueStruct) EncodeTo(buf *bytes.Buffer)
EncodeTo should be kept in sync with the Encode function above. The reason this function exists is to avoid creating byte arrays per key-value pair in table/builder.go.
func (*ValueStruct) EncodedSize ¶
func (v *ValueStruct) EncodedSize() uint16
EncodedSize is the size of the ValueStruct when encoded
type WaterMark ¶
type WaterMark struct { Name string // contains filtered or unexported fields }
WaterMark is used to keep track of the minimum un-finished index. Typically, an index k becomes finished or "done" according to a WaterMark once Done(k) has been called
- as many times as Begin(k) has, AND
- a positive number of times.
An index may also become "done" by calling SetDoneUntil at a time such that it is not inter-mingled with Begin/Done calls.
Since doneUntil and lastIndex addresses are passed to sync/atomic packages, we ensure that they are 64-bit aligned by putting them at the beginning of the structure.
func (*WaterMark) DoneUntil ¶
DoneUntil returns the maximum index that has the property that all indices less than or equal to it are done.
func (*WaterMark) SetDoneUntil ¶
SetDoneUntil sets the maximum index that has the property that all indices less than or equal to it are done.