How Our Rust-to-Zig Rewrite Is Going

I think this is a fine post. But one comment:

> remember that for compilers which emit machine code, like roc and rustc, doing memory-unsafe things is a big part of the job

I don't really think that this is true, in the way that it's written.

I think that for the hot binary patching / code reloading features, yes, that is going to need unsafe. But for regular old "producing an executable" compilation? Emitting machine code isn't the part that requires unsafe. The language's runtime is a more likely site to find unsafe.

>ReleaseSafe catches use-after-free errors through runtime checks which panic if the program tries to use freed memory.

I don't know Zig so maybe they know something I don't, but I have seen no evidence that it catches any type of use-after-free including double-free?

While writing a blog post (below) I went through the documentation to figure out the possible runtime memory safety checks Zig can insert. The term "use-after-free" or "UaF" never occurs on that documentation page. Searching for "safety-checked" doesn't yield any related hits either.

Unless maybe they're using the DebugAllocator in release builds? Even that does not reliably surface UaF.

https://landaire.net/memory-safety-by-default-is-non-negotia...

Interesting that OCaml was flexible and expressive enough to be used as a prototype testbed but not chosen as the implementation language, especially given the maturity of both. I would be surprised if Zigs incremental builds could be meaningfully faster than dune's.

Cross compilation is great, but not mentioned in the "why Zig" section. Is memory control that crucial for a compiler?

Rust itself was originally written in OCaml, same with WASM. I'm curious about what milestone gets reached where the maintainers collectively decide to transition away.

Zig's incremental builds are DEFINITELY a killer feature. In the short term, I could see why you'd make a switch to get it. But, in the medium term, can we really not expect to see this in Rust in the somewhat near future?

I want to go fast, but I don't want to go fast just to shoot my foot off.

If only somehow we could get Rust's safety with all of Zig's features and Go's runtime without GC...

That's what I'm working on building [=

Compile times are a really underrated thing. My #1 gripe with C++ is waiting 10 minutes on a build, it absolutely kills flow.

This piece would have been a lot more compelling if they had actually done science on selecting a language for compiler development. From what I can tell, they had an untested hypothesis that a low level systems language is necessary for a high performance compiler https://www.roc-lang.org/faq#self-hosted-compiler and from that concluded that their only choice besides rust was zig.

I know from experience that this initial assumption is wrong. Compiler performance is dominated by algorithms. The fastes managed languages tend to be at worst within a factor of two for wall time on any given algorithm. Algorithmic differences can be unbounded in their performance gaps. Zig itself is a perfect counterexample to the theory that writing a compiler in a low level systems language will lead to a fast compiler. Roc seems to compile at around 15k lines per second. That is not fast. There were evidently compilers written in ml that did 3k likes per second in 1998 https://flint.cs.yale.edu/cs421/case-for-ml.html

The zig rewrite of roc looks like the author's second compiler. Compiler and language design is a skill like any other and from my vantage point, they appear to have overcommitted to an initial design at the expense of developing their higher level design skills. In my opinion, the best thing they could do for the future of roc is stop working on their current compiler and use it to write a self hosting compiler for a much smaller subset of roc. They should be able to do that in less than 10k lines of code. They might even find that their self hosting compiler is faster than their zig based bootstrap compiler for the self hosted subset of roc. If the self hosting compiler is inadequate. Now they at least have identified a smaller useful subset of roc and can experiment with different compiler implementations in 10k likes of code rather than 300k lines of code. Then they could actually test the theory of whether or not a low level language is necessary to meet whatever arbitrary compiler performance goals they have.

By self hosting, they would also discover what roc features actually matter and they would spend much more time actually writing roc code. The features that are needed to write a self hosted compiler are all features that are generally useful. By improving the self hosted compiler, they also improve downstream programs.

Tangentially relates, but if any Roc devs are around I'm curious about the use cases for Roc.

It's supposed to be a scripting language right you embed into your C ABI right?

Do you see it competing with WASM for the plugin use case (i.e. a really large Roc platform)? Why would an app author prefer to expose a Roc layer to their app rather than a WASM layer? With a WASM layer, plugin devs can write in any language.

Another use case I've heard from it is as a more app-level language (i.e. a really small Roc platform). Do you see it competing with Gleam for server side http code? Do you see it competing with Elm for client side code?

Quite interesting the hand waving of security issues with Zig, oh well.

If I want to use allocator debuggers I already have the production ready tools that exist for C and C++ for at least 30 years.

One thing I wish Rust would improve over time is the builds. Its one of the biggest sources of wasted storage space on all my computers, builds a ton of libraries can take tens of gigs, it adds up very quickly. Not sure what the best solution is, one I found is to set the global build folder so dependencies get reused across projects, but imho it should be an OOTB default behavior whatever the real solution should be.

Zig is a pre-1.0 language while Rust is post-1.0. This alone is settles which one to pick for may developers. The library support is probably favours Rust too. Rust build times are much slower than Zig, I get that, but I rarely optimize software for build times.

Roc seems interesting. But for some reason I find it very grating to have the type definition on a separate line. Very much prefer F# syntax for that.

Nitpicking ahead:

I am not sure, but there might be a bug in their pattern matching example.

What happens if 'verb' is "GET" and 'path' is "/users/1234/posts/1234/extra_path/and/more/"? Will 'post_id' become "extra_path/and/more/"?

I tried running it in the sandbox, and it does indeed seem to buggily result in:

"Post ID: 1234/extra_path/and/more"

I suspect that the reason it is behaving like it is, is due to how it handles characters in the string literal. The example program exploits that only the slashes present in the string literal pattern are matched, to enable matching on 'page' having slashes. But then in the nested 'match', it forgot to account for any possible extra slashes.

Nitpicking end.

I have not read the whole post yet, but the pattern matching not requiring any allocations, seems very nice. The string literal patterns also seem interesting, though I am not completely sold on them, also as per the above possible bug. It seems really clean in some ways, but the specific semantics, I am not fully sure about. Maybe it is excellent, and is so clean and concise that it is overall less bug-prone than alternatives in other programming languages. I do not know.

The 35ms incremental rebuild is the part that sold me. I'd be curious to see the same benchmark on ARM once -fincremental gets there.

Does this mean every time you find yourself using lots of “unsafe” Rust blocks, it’s not the right tool for the job? I suspect it’s not that simple, but what are people’s experience?

Didn’t know Roc was still being worked on. I think it’s an interesting concept for a language that I personally haven’t seen elsewhere

is this uno reverse for bun post of zig to rust port ?

Irrespective to the technical merits of both language, moving from a stable language to a pre-1.0 one that just lost his most popular open source project is a wild move.

why not rewrite in ROC?? Would be much more cooler.

I think precious cognitive time should be spent more on the language itself rather than wasting it on rewrites.

Can anybody explain to me why anthropic bought bun in the first place ?

I think there will be soon a wave of rewriting rust to language X coming up.

While I'm a rust enthusiast, I do agree that certain languages lend themselves well to particular domains. So a rewrite from Rust to something better suited is fine by me. In fact, while I do work on a rust project, I would not have and still would not recommend it as the choice for that particular project.

That being said, I had to do some double takes while reading this.

> https://rtfeldman.com/rust-to-zig#memory-safety-post-rewrite

I feel that it's a bit weird to compare a rather well tested 7 (?) year old rust implementation with a brand new not yet released less than a year old Zig implementation. Without that context, this looks like a bad comparison for rust, when it is in fact the complete opposite.

> https://rtfeldman.com/rust-to-zig#build-times

The swiftness of the Zig compilere here is insane, and would would very much shift my recommendation of Rust if it got to similar speeds.

That being said, I do find it funny that currently, the compilation speed is actually worse on Zig than Rust, despite Zig (anonymous commenters at least tbf) claiming the opposite for years.

How did you eventually discover the 35 ms figure for Roc? Did you have to temporarily update the codebase to 0.17?

> https://rtfeldman.com/rust-to-zig#memory-control-zero-parse-...

Nothing negative here. I did play around with implementing a scripting language in this DOD-ish, index-based paradigm and yeah, it is neat.

I was thinking that it might be possible to do resumable computation across the network like this (in the context of frontend frameworks "resuming" UIs), but ultimately I have no use for this so just the experience itself was enough.

One note here is that it does tend to break completely if non-pointer-free data is introduced. It seems like it's either all or nothing.

> https://rtfeldman.com/rust-to-zig#ecosystem-relevance

This is more of an LLVM thing, which is fair, but I find it funny that "LLVM unstable bad" while "Zig unstable whatever".

Overall though, this was an interesting read. And if the folks contributing to roc like zig then more power to them.

Last thing, the link here is broken (points to a TODO):

> Zig's compiler itself is another

[dead]

[dead]

[dead]

[dead]

[dead]

I don't even know what Zig is but I've seen this topic come up so many times on this site that I'm starting to think the people who are actually doing this are unsure themselves whether it's a good idea or not.