Caching Strategy and Circuit Breakers: Keeping Tukuk-OS Fast and Stable

From Tukuk-OS Wiki, the free knowledge base.

Caching Strategy and Circuit Breakers: Keeping Tukuk-OS Fast and Stable
Type	Encyclopedia article
Subject	Caching
Author	Fevian Donald
Published	28 October 2026
Revised	28 October 2026
License	Free to read & cite

Two techniques that turned Tukuk-OS from a frequently slow site into a fast one even when external services fail: query-keyed caching and circuit breakers.

Contents

1. Cache: Why Same, How Fresh
2. How I Know the Cache Works
3. The Circuit Breaker: Three States
4. What Users See
5. What I Do Not Cache
6. The Lesson

Every search engine depends on external services, and every external service will fail at an unexpected moment. Tukuk-OS handles this fact with two simple but powerful techniques: carefully planned caching, and a circuit breaker that stops failures from spreading.

Cache: Why Same, How Fresh

The Tukuk-OS cache is organised by query and parameters, not time alone. The cache key contains the query, the page, and the media type — so "nasa" page 1 and page 2 never mix.

But the most important part is the rule about what is not stored. Weak results (too few hits) and failures never enter the cache. If I stored failed results, a temporary problem would become the "official answer" for that query for the whole cache duration — a mistake I learned about the hard way when my image grid showed a single result for several minutes.

Time-to-live matters too: text search results can live longer because the index only changes when I update it, while daily NASA data has a short life because it changes every day.

How I Know the Cache Works

Like any optimisation, a cache is only worth it if the effect can be measured. I installed simple response-time logging: every request records whether it was served from cache or from source, and how long it took.

After a few weeks of data two things became clear. First, most search queries repeat within a short window — when a topic trends, hundreds of people type variations of the same query within hours, and all of them are served in milliseconds from cache. Second, a well-designed cache reduces not only user latency but also load on external services — which in turn reduces the risk of being blocked for excessive requests.

I also track the hit rate: the share of requests served from cache. When that rate drops sharply it usually signals a change in query shape (perhaps a new feature) rather than a problem — and it tells me to review my key generation strategy.

The metric that matters most, however, remains the simplest: does the page feel fast to the reader? All other numbers are proxies for that answer. When there is a conflict between numbers and user perception I always trust user perception — because ultimately the cache exists for them, not for a dashboard that only I see each morning.

The Circuit Breaker: Three States

A circuit breaker has three states:

CLOSED — all requests pass through to the external service as normal.

OPEN — the external service has failed repeatedly; requests are rejected immediately and the fallback answer is used without waiting.

HALF-OPEN — after a cool-down period, a few trial requests are allowed to see whether the service has recovered.

In Tukuk-OS this is applied to the NASA APOD call. When NASA has an outage (which happens more often than you would think), the first request might take ten seconds before failing; the second records the failure; the third opens the circuit. From then on every client gets the fallback instantly — no more timeouts for every visitor.

What Users See

The UX key: when data comes from cache or a fallback, users are told. The APOD display shows a "data may be delayed" warning with a retry button instead of pretending everything is normal.

This honesty beats silence. Users are more understanding and more patient when they know what is happening, and the retry button gives them control to try again when they want.

What I Do Not Cache

I do not cache contact form submissions (rare and critical), do not cache private responses (there are no private users to store), and do not cache static pages at the application layer because Cloudflare already does that more efficiently at the edge.

The main point: caching is not "cache everything". Every cache decision must weigh speed against the risk of serving stale data.

The Lesson

If you build anything that calls external services, start with two rules: never store failures, and never let a third party's failure slow you down. Both look small in code, but they determine whether your service feels solid or fragile to anyone who uses it.