bohm

package module
v0.0.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Apr 26, 2024 License: Apache-2.0 Imports: 13 Imported by: 0

README

WIP

Documentation

Index

Examples

Constants

This section is empty.

Variables

This section is empty.

Functions

func Logger

func Logger(ctx context.Context) *slog.Logger

Logger can be used with the contexts provided to Handlers to retrieve a slog instance that has values populated by the Bohm internals, or the default logger in other contexts.

Essentially this gets you the rowID, popCount, and queueName columns.

Types

type Cluster

type Cluster struct {
	Logger *slog.Logger
	// contains filtered or unexported fields
}

Cluster represents the state shared between multiple instances of this process i.e. the database.

func New

func New(db *pgxpool.Pool) *Cluster

func (*Cluster) NewPollingLoop

func (c *Cluster) NewPollingLoop(ctx context.Context, opts ...Option) error

NewPollingLoop is similar to NewQueue, but instead of creating a queue that watches a table, it creates a queue with a single row used to poll at a dynamic, configurable interval. Returning an empty RequeueAfter will break the loop!

func (*Cluster) NewQueue

func (c *Cluster) NewQueue(ctx context.Context, table string, opts ...Option) error

NewQueue migrates the schema for the given queue and spawns the corresponding listener if a handler is given.

The schema template is defined in bohm/schema.sql.

Example
package main

import (
	"context"
	"fmt"
	"os"
	"time"

	"github.com/jackc/pgx/v5/pgxpool"
	"github.com/jveski/bohm"
)

func main() {
	ctx := context.Background()
	db, err := pgxpool.New(ctx, "postgresql://localhost:5432/postgres")
	if err != nil {
		panic(err)
	}
	defer db.Close()

	migrate(db)

	// Set up a handler to process change events
	cluster := bohm.New(db)
	handler := func(ctx context.Context, i int64) (bohm.Result, error) {
		logger := bohm.Logger(ctx) // this logger has some relevant fields already set by Bohm!

		foo, bar := queryRow(db, i)
		logger.Info("current values", "foo", foo, "bar", bar)

		// Calculate the correct value of the 'bar' column
		expectedBar := fmt.Sprintf("%s-but-in-bar", foo)

		if bar != nil && *bar == expectedBar {
			os.Stdout.Write([]byte("in sync\n"))
			return bohm.Result{}, nil // nothing to do!
		}

		writeRow(db, i, expectedBar)
		logger.Info("updated value of bar")

		// - Returning an error causes this work item to be retried with exponential backoff (maybe on another queue listener)
		// - Setting RequeueAfter in the result schedules the next sync of this row
		return bohm.Result{}, nil
	}
	cluster.NewQueue(ctx, "test_table", bohm.WithHandler(handler))

	// Insert a row
	_, err = db.Exec(ctx, "INSERT INTO test_table (string_foo) VALUES ('hello world')")
	if err != nil {
		panic(err)
	}

	// Wait to clean up until things have sync'd
	time.Sleep(time.Millisecond * 500)
	cleanup(db)

}

func migrate(db *pgxpool.Pool) {
	_, err := db.Exec(context.Background(), `
		CREATE TABLE IF NOT EXISTS test_table (
			pkey SERIAL PRIMARY KEY,
			string_foo TEXT NOT NULL,
			string_bar TEXT
		);
	`)
	if err != nil {
		panic(err)
	}
}

func cleanup(db *pgxpool.Pool) {
	_, err := db.Exec(context.Background(), "DROP SCHEMA public CASCADE; CREATE SCHEMA public;")
	if err != nil {
		panic(err)
	}
}

func queryRow(db *pgxpool.Pool, row int64) (string, *string) {
	var foo string
	var bar *string
	err := db.QueryRow(context.Background(), "SELECT string_foo, string_bar FROM test_table WHERE pkey = $1", row).Scan(&foo, &bar)
	if err != nil {
		panic(err)
	}
	return foo, bar
}

func writeRow(db *pgxpool.Pool, row int64, bar any) {
	_, err := db.Exec(context.Background(), "UPDATE test_table SET string_bar = $1 WHERE pkey = $2", bar, row)
	if err != nil {
		panic(err)
	}
}
Output:

in sync

func (*Cluster) WaitForHandlers

func (c *Cluster) WaitForHandlers()

WaitForHandlers blocks until all currently in-flight workers return.

type Filter

type Filter struct {
	// contains filtered or unexported fields
}

Filter describes some logic that can be executed by Postgres.

func Changed

func Changed(col string) *Filter

Changed is true when the column has changed during an upgrade.

type Handler

type Handler func(context.Context, int64) (Result, error)

Handler takes the primary key of a row and does something with it.

- Errors are retried - Handlers can be invoked concurrently - For a given row, only one handler can be active (per queue, across all listeners)

type Option

type Option func(*watchQueue)

func WithConcurrencyLimit

func WithConcurrencyLimit(limit int) Option

WithConcurrencyLimit limits the concurrency of the queue's handlers within this particular process.

func WithFilter

func WithFilter(f *Filter) Option

WithFilter applies logic to be evaluated by the db when processing changes in order to determine if a work item should be enqueued for a given database write.

This is useful for avoiding calling Handlers when there cannot possibly be anything for them to do.

func WithHandler

func WithHandler(fn Handler) Option

WithHandler sets the queue's Handler. If not given, the queue schema will still be migrated.

func WithKeepalives

func WithKeepalives(percent float64) Option

WithKeepalives sets at what percentage of the lock TTL it should be renewed. e.g. 0.8 means the lock TTL will be reset to the configured value when it has 20% of its lifespan remaining.

func WithLockTTL

func WithLockTTL(ttl time.Duration) Option

WithLockTTL determines how long a work item is locked when taken off of the queue. When the lock expires another process can start processing the same item if it hasn't already been completed.

Larger values means longer waits when workers crash or are partitioned from the db. Smaller values may result in unexpected concurrency for slow Handlers. This should be tuned for the particular handler given WithKeepalives.

func WithQueueName

func WithQueueName(name string) Option

WithQueueName sets the queue name. Defaults to ${tablename}_default.

This allows multiple queues to be defined for the same table.

type Result

type Result struct {
	RequeueAfter time.Duration
}

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL