Introducing goqueuelite: Golang + SQLite queue

Posted on July 26, 2023 [development] [squeuelite] [opensource]

It finally happened! I am about to introduce my first proper open source project, it is called squeuelite and it is a Golang package that tries to fix the queue issue using SQLite only.

The package can be found out github.com/risico/goqueuelite, check it out. The package is not production ready yet, although I’ve been using it already in maile.io (from where I’ve extracted it) and is able to handle quite a bit of load (I’ll post benchmarks soon).

Getting started

Import the library into your project as usual:

go
package main

import (
	"github.com/risico/goequeuelite"
)

func main() {
	s, err : squeuelite.New(squeuelite.Params{
		DatabasePath: "queue.db",
	})

	err := s.Put("default", "somedata", 0) // put "somedata" into the "default" namespace, execute as soon as possible
  // handle err
	
	v, err := s.Pop("default")
	// handle err

	// do something with v 

	s.Done(v.MessageID)
	// or 
  s.Fail(v.MessageID)
  // or 
  s.Retry(v.MessageID)
}

The library is safe to run concurrently, so here the main usecase would be to run workers in Go routines that pick jobs to work on. Jobs can be marked as done, failed or be sent back for retry.

The API will def change until v1 is stable so please be prepared for that.

Have fun!