Meilisearch: From a Hand-Built Index to a Fast Production Engine

From Tukuk-OS Wiki, the free knowledge base.

Why I chose Meilisearch for Tukuk-OS, how I tuned it, and lessons from running an index live in production.

After building an inverted index by hand I knew exactly what I needed: fast search, typo tolerance, faceted filtering, and minimal administration. Meilisearch met that list in a way nothing else I tested could, and the decision to use it turned Tukuk-OS from a personal project into a system that runs 24 hours a day.

Why Meilisearch, Not the Alternatives?

I tested several options. Heavy options required a JVM or multi-node configuration beyond my small server's ability. Lightweight options did not support faceted search or autocomplete.

Meilisearch gave me three things I needed immediately: responses in milliseconds, filtering during search (language, site, type), and autocomplete. It also runs as a single process with reasonable memory usage — perfect for a server sharing resources with a web application.

How I Tuned the Index

Tukuk-OS's main index, halaman_web, has searchableAttributes ordered by priority: title, headings, description, content, keywords, site name, and path. That order is not random — a match in the title outranks a match in body text.

filterableAttributes define what can be filtered in the UI: host, lang, type, section, siteName, and more. When I added a new siteName field the "Publisher" filter link appeared with no extra work. I learned that index design is really UI design — decisions I make in the backend immediately shape what users can do in the frontend.

Measuring Search Quality with Reference Queries

Tuning index settings without measurement is guessing in the dark. I set a list of about twenty reference queries — a mix of popular searches, long-tail queries, and deliberately challenging ones — and run it every time I change configuration.

Each query is rated on three simple criteria: is the top result genuinely relevant, are the relevant filters available, and does response time stay below the threshold. When I add a new filterableAttributes field I expect filters to appear without slowing search down. If it slows search down I have to weigh whether the filter is worth it.

This approach prevents me from getting stuck optimising numbers that do not matter. A higher average score means nothing if your most popular query gets worse. With a reference set, a bad change shows itself within seconds and I can roll it back before it reaches users.

One extra lesson: keep the list in a portable text file, not in your memory. When I need to remember why a setting was chosen six months ago, the reference list and the notes beside it save hours of guessing.

Managing a Live Index

The most important thing I learned about Meilisearch in production is how to handle updates. If I update the index live, users see half-finished results during the swap. My solution is simple: batch new documents, keep the old index until the new one is ready, then swap.

I also set displayedAttributes so internal fields are never sent to clients. This not only saves bandwidth, it prevents internal fields like internal scores from leaking into API responses.

Backups: The Part Nobody Likes to Do

A Meilisearch index can be rebuilt from source, but rebuilding takes time and can fail if the source has changed. I wrote an export script that saves the index to a gzip file every few hours to Cloudflare R2. The script is about twenty lines, but it buys me immeasurable peace of mind.

The most important test I ran was not a successful backup — it was a successful restore. I deliberately deleted an index in a testing environment and restored it. After that I knew the procedure actually worked, not just that it looked like it worked.

When Meilisearch Is Not Enough

Meilisearch solves text search, but Tukuk-OS needs more: image and video search requires external sources, autocomplete for new queries requires custom logic, and different scoring for blog posts versus link pages requires adjustment.

I decided not to force Meilisearch to do everything. Instead I built a layer around it: result scoring, media source aggregation, and filter management. The separation makes my system easy to maintain — each component does one thing well.

Advice for Users

If you are starting a search project, begin with Meilisearch and do not build your own system too early. Spend your time on index design: what can be filtered, what can be searched, and what should be displayed. Those decisions will outlive any framework choice, and they determine whether your product feels smart or ordinary.

See also

• Building an Inverted Index by Hand: My First Search Engine Code
• My First Crawler: The Spider That Collected the World
• What Is Tukuk-OS? What This Search Engine Actually Does