Skip to main content

Intro to Garak

We purpose-built Garak at Hiyllo to replace MongoDB. We wanted to build a database that would intentionally make our lives harder and force us to write efficient database queries. Essentially, the issue was in MongoDB (and really, most databases), nothing stops you from doing something like this:

const SomeCollection = new mongo.Collection("...");
SomeCollection.find({ status: "red", foo: { $gte: 200 }, bleh: { $ne: "bar" }});

These kinds of queries are extremely inefficient and will almost always result in full collection scans. Not good!

With Garak we took a fairly drastic approach. Absolutely no full collection scans. We didn't even build a way to do it.

Instead, we took these principles:

  • The only way to write data is to either insert a new record or to perform an update using it's node ID (u128).
  • The only way to read data is to either get it by it's node ID or retrieve it from an index directly

With indexes, we've taken a more deliberate route. Rather than indexes being sometihng abstracted away that help your queries run faster, indexes are now elements you directly query from.

Here's an example:

export const SignaturesStore = new Garak.Store<RegalCloudSignature>(
  db,
  "signatures"
);
export const signatureByRequestSKI = new Garak.SecondaryKeyIndex(
  SignaturesStore,
  "signatureRequestUUID"
);

Here we define a data store, and we define a secondary key index (SKI). SKIs are cool cause they hold 1:1 relationships of a given key to a document. No two documents can have the same value for a SKI'd field (nullish values are okay though). We enforce this when you write to the database as well, rejecting writes that would cause this rule to be broken.

If we want to retrieve a document based on it's SKI value, we call the index:

const tx = db.tx("...");
const mySignature = await signatureRequestUUID.fetch(tx, "some-foo-value-bar");

Again, this forces us to be deliberate about how we design indexes. Rather than using them to optimize our db scans, we build indexes to specifically solve our use cases.