Saving 100 terabytes of memory by optimizing 1.1.1.1's DNS cache

Produce working product first, validate the idea, stabilize the business, start generating profit, and then you can start optimizing your costs.

In fact optimization is by far the easiest part of the process because there are many system programming experts on this HN thread who consider these optimizations to be trivial.

Looks like they're missing the obvious optimisation of putting the record data right after the CacheEntry members instead of allocating memory separately though. But that might just be me as a C-programmer talking and not be all that easy in Rust.

When I was using one malloc() per entry, a large blacklist took up 237 megabytes of memory. The same blacklist, once optimized to be loaded with a single malloc() call, only took up 9.5 megabytes of memory.

https://samboy.github.io/blog/entries/MaraDNS.html#BlogEntry...

  type Wasteful struct {
    a int16
    b int
    c byte
  }

  type Aligned struct {
    b int
    a int16
    c byte
  }

If you previous had three distinct Vec objects, then Rust would guarantee that you can't index out of bounds. If you now put all those objects into a single Vec and rely on offsets, then you now open the door to indexing out of range of these sub-slices without any panics.

It's a minor point, and it doesn't really invalidate the optimization, but I'm surprised the article didn't mention it.

Even a BTree with compressed prefix keys can save space in the qname.

The 100 TiB number almost gives me vertigo. Though in this context it was "just" 50%

It is almost like nobody even thought during the design phase about what might happen down the road.

This is why so much software is bloated and often buggy. Just gets something that half-way works out the door ASAP and worry about the rest later (too often, never).

> Big Pineapple uses jemalloc, an allocator designed for multithreaded, allocation-heavy workloads.

jemalloc multithreaded performance is actually poor(ish) compared to other modern allocators, which makes it a weird choice. But even weirder is why they're even using an allocator in the first place compared to a va MAP_ANON | MAP_NORESERVE arena carveout approach? You can also do punning that way too, which I'm not even certain if Rust supports?

And they say nobody uses IPV6.

Interestingly this is exactly how netlink works-ish: https://manpages.ubuntu.com/manpages/focal/man3/netlink.3.ht...

You start, get the type & length, and then that is how many bytes you read.

Some issues with that when you deserialize, from a raw stream in to `[u8; 4096]` buffer, the alignment is only guaranteed to be on 1 byte, not 4 bytes.

In practice it is 4 bytes, but if you run those tests with Miri, you'll get yelled at. So the fix there is to declare the buffer with a type that mandates the alignment of the largest type that you're going to be deserializing.

So then you start your buffer as follows: `[u32; 1024]`, and with `slice::from_raw_parts` you get to turn that into `[u8; 4096]` with the expected alignment.

As an exercise I wrote a streaming parser for netlink, the current existing package serializes everything, all at once.

leaves

Were there no design discussions/reviews when the system was setup to catch trivial things like this?

What's the speed of service/response time relative to the data source?

At that point it might be enough to replace your multiple caches with fewer in-RAM databases?

It's an interesting problem.