README
¶
CDB64
This is a native Go implementation of cdb, a constant key/value database with some very nice properties, but without the 4GB size limit.
Adapted from the original design doc:
cdb is a fast, reliable, simple package for creating and reading constant databases. Its database structure provides several features:
- Fast lookups: A successful lookup in a large database normally takes just two disk accesses. An unsuccessful lookup takes only one.
- Low overhead: A database uses 4096 bytes, plus 32 bytes per record, plus the space for keys and data.
- No random limits: cdb can handle any database up to 16 exabytes. There are no other restrictions; records don't even have to fit into memory. Databases are stored in a machine-independent format.
This repo is based on https://github.com/colinmarc/cdb
Usage
writer, err := cdb64.Create("/tmp/example.cdb")
if err != nil {
log.Fatal(err)
}
// Write some key/value pairs to the database.
writer.Put([]byte("Alice"), []byte("Practice"))
writer.Put([]byte("Bob"), []byte("Hope"))
writer.Put([]byte("Charlie"), []byte("Horse"))
// Freeze the database, and open it for reads.
db, err := writer.Freeze()
if err != nil {
log.Fatal(err)
}
// Fetch a value.
v, err := db.Get([]byte("Alice"))
if err != nil {
log.Fatal(err)
}
log.Println(string(v))
// => Practice
// Iterate over the database
iter := db.Iter()
for iter.Next() {
log.Printf("The key %s has a value of length %d\n", string(iter.Key()), len(iter.Value()))
}
if err := iter.Err(); err != nil {
log.Fatal(err)
}
Documentation
¶
Overview ¶
Package cdb64 provides a native implementation of cdb, a fast constant key/value database, but without the 4GB size limitation.
For more information on cdb, see the original design doc at http://cr.yp.to/cdb.html.
This is based on the code from https://github.com/colinmarc/cdb
Example ¶
writer, err := Create("/tmp/example.cdb")
if err != nil {
log.Fatal(err)
}
// Write some key/value pairs to the database.
writer.Put([]byte("Alice"), []byte("Practice"))
writer.Put([]byte("Bob"), []byte("Hope"))
writer.Put([]byte("Charlie"), []byte("Horse"))
// Freeze the database, and open it for reads.
db, err := writer.Freeze()
if err != nil {
log.Fatal(err)
}
// Fetch a value.
v, err := db.Get([]byte("Alice"))
if err != nil {
log.Fatal(err)
}
fmt.Println(string(v))
Output: Practice
Index ¶
Examples ¶
Constants ¶
This section is empty.
Variables ¶
var ErrTooMuchData = errors.New("CDB files are limited to 4GB of data")
Functions ¶
This section is empty.
Types ¶
type CDB ¶
type CDB struct {
// contains filtered or unexported fields
}
CDB represents an open CDB database. It can only be used for reads; to create a database, use Writer.
Example ¶
db, err := Open("./test/test.cdb")
if err != nil {
log.Fatal(err)
}
// Fetch a value.
v, err := db.Get([]byte("foo"))
if err != nil {
log.Fatal(err)
}
fmt.Println(string(v))
Output: bar
func New ¶
New opens a new CDB instance for the given io.ReaderAt. It can only be used for reads; to create a database, use Writer.
If hasher is nil, it will default to the CDB hash function. If a database was created with a particular hash function, that same hash function must be passed to New, or the database will return incorrect results.
type Iterator ¶
type Iterator struct {
// contains filtered or unexported fields
}
Iterator represents a sequential iterator over a CDB database.
Example ¶
db, err := Open("./test/test.cdb")
if err != nil {
log.Fatal(err)
}
// Create an iterator for the database.
iter := db.Iter()
for iter.Next() {
// Do something with iter.Key()/iter.Value()
}
if err := iter.Err(); err != nil {
log.Fatal(err)
}
Output:
type Writer ¶
type Writer struct {
// contains filtered or unexported fields
}
Writer provides an API for creating a CDB database record by record.
Close or Freeze must be called to finalize the database, or the resulting file will be invalid.
Example ¶
writer, err := Create("/tmp/example.cdb")
if err != nil {
log.Fatal(err)
}
// Write some key/value pairs to the database.
writer.Put([]byte("Alice"), []byte("Practice"))
writer.Put([]byte("Bob"), []byte("Hope"))
writer.Put([]byte("Charlie"), []byte("Horse"))
// It's important to call Close or Freeze when you're finished writing
// records.
writer.Close()
Output:
func Create ¶
Create opens a CDB database at the given path. If the file exists, it will be overwritten.
func NewWriter ¶
func NewWriter(writer io.WriteSeeker, hasher HashFunc) (*Writer, error)
NewWriter opens a CDB database for the given io.WriteSeeker.
If hasher is nil, it will default to the CDB hash function.
func (*Writer) Close ¶
Close finalizes the database, then closes it to further writes.
Close or Freeze must be called to finalize the database, or the resulting file will be invalid.
Source Files
Directories