A couple million lines of Haskell: Production engineering at Mercury

> Haskell gives you tools to encode these incantations in types so they cannot be forgotten. This is, for my money, the single most valuable thing the language offers a production engineering organization.

Haskell is admittedly, probably the most powerful widely (or even somewhat widely) used language for doing this, but this general pattern works really well in Rust and TypeScript too and is one of my very favorite tools for writing better code.

I also really like doing things like User -> LoggedInUser -> AccessControlledLoggedInUser to prevent the kind of really obvious AuthZ bugs people make in web applications time and time again.

I've found this pattern to be massively underutilized in industry.

I loved working in Haskell for a few years. I wasn't actively looking it, but the opportunity just sort of landed in my lap. It was exciting and mentally stimulating. But the unfortunate fact is, I am easily twice as productive in Rust as I am Haskell, even after 3 years of nothing but Haskell. There are more pitfalls in Haskell that you have to just know how to avoid. It can be very difficult to digest as the language can be borderline write-only at times, depending on the author of the code. The tooling is often married to Nix, which is it's own complex beast. And it feels like language extensions are all over the place. Cabal files are not my favorite. And the compiler errors take some time to get get used to.

I think perhaps contrary to popular belief, Mercury choosing Haskell and their early leadership having such a storied experience in it probably played some non-insignificant role in their success.

As a customer of Mercury, it's truly one of the critical companies my toolkit, and I just can't help but feel that their choosing of Haskell made their progress, development and overall journey that much better. I realize that you can make this argument with most languages, and it's not to say that a FP lang like Haskell is a recipe for success, but this intentional decision particularly pre "vibe coding" and the LLM era seems particularly prescient, of course combined with their engineering culture that was detailed in the post.

A similar Haskell success story (from Bellroy) is the subject of an upcoming Melbourne Compose meeting: https://luma.com/uhdgct1v

I don't believe I'm the target market (I'm plenty happy with my small CU), and seeing their billboards makes me want to never use them BUT: seeing this post and their culture and that they use Haskell is kinda changing my mind.

It’s hard to imagine what two millions lines of Haskell could possibly be doing. I mean that’s a lot of code and I have the impression that Haskell is “tight” meaning a little code can do a lot. Maybe they have a lot of libraries to do things like json serializing/deserializing, rest api frameworks, logging etc?

My bestie works at this company and looking from the outside they have a good engineering culture. I do think Haskell is the right tool for the job, and they are playing to it's strengths, but part of me wonders if a lot of their success is attributable to the place just being well run in general.

> [To lib authors] Nobody is obviously in charge in the way a fast-moving production team would mean "in charge," and that creates understandable hesitation around making breaking changes, even when experience has taught us better ways to design these systems.

> This is not a complaint about volunteer maintainers. It is simply one of the ambient risks of building serious systems on a smaller ecosystem.

And so instead of paying the lib authors who already have domain expertise and know their codebase, they chose to rewrite it from scratch/fork without contributing back. So classic.

I think you have to get a Haskell job early in career and stick to Haskell jobs. Breaking in is really hard as you come without experience there will be plenty of others with Haskell experience to compete. And because the jobs are rare if it doesnt work out (company becomes bad to work for or layoff) you can be unstuck (or I guess you would switch to Rust, Scala or F#)

I use Haskell a lot, but I notice that it's very hard to cross-compile it.

If only cross-compilation became easy so that I can develop on my chip Macs and deploy on x64/AMD Linux servers.

>statically linking Haskell binaries is quite a challenge

>build requirements really slow down the process. I have to use dockers to help cache dependencies and avoid recompiling things that have not changed, but it is still slow and puts out large binaries.

Also, the Docker-based deployment takes a lot of time as it needs to recompile each module. While you can cache some part of it, it's still slow.

Meanwhile with Go it's painless. And i am not the only one having this issue:

https://news.ycombinator.com/item?id=47957624#47972671

Such a shame Haskell is beautiful and performant language still build is slow.

Impressive scale! Would love to hear more about their testing strategy and how they handle dependency management at that size.

Cool article, wish it wasn’t AI written/edited.

This is a great read. The "observable by construction" principle especially building traceability in from the start rather than bolting it on later is something more teams should internalize.

Hell yeah, I made a hyperbolic PDE solver in a bizarre constrained space in haskell! Unboxed vectors and performant algebriac systems out of functional programming is a blast.

huh i love haskell soooooo muchhhh

I'm not particularly into Haskell or functional languages, but I came here to say it warms my heart to hear that people in finance actually embrace it (also J/APL). It seems like a good choice for banking.

The Mercury site also looks way better than most other banks I have ever used (load speed is also very good.) On the danger of seeming like a shill (I'm not), I'm tempted to try them out.

> written using the mosaic theory of information and a range of journalistic tools

what does that mean?

Mercury has been awesome, I've been using them for my business account for years and recently started using them for personal as well. I didn't know they used Haskell until well after I started using them, but it definitely tracks. The quality of their exposed software surface is at least a couple stddev above median.

Just wanted to say that rarely has a text make me want to work for a company as much as this one. This is the way engineering should be done. I have never worked in such an organisation.

Will AI agents turn Haskell into a commercially viable language?

Haskell is like an instrument I can never quite master, yet I find it utterly fascinating and keep trying to learn it without success.

I really believe in FP and Haskell but I want to examine this objectively. Empirically speaking is what Mercury done successful truly because of Haskell? Do they have metrics that demonstrates clear superiority along some vectored trait like complexity, bug count, etc?

>A couple million lines of Haskell, maintained by people who learned the language on the job, at a company that moves huge amounts of money? The conventional wisdom says this should be a disaster, but surprisingly, it isn't. The system we've built has worked well for years, through hypergrowth, through the SVB crisis that sent $2 billion in new deposits our way in five days,1 through regulatory examinations, through all the ordinary and extraordinary things that happen to a financial system at scale.

This one is quite telling. Do people have counter examples?

risky move, what is the talent pool for Haskell devs these days?

I know this is not the point of the article, but I find the anecdote in the beginning about null pointer errors somewhat ironic. Haskell's solution to null pointers are option types (`Maybe x` in Haskell), but these are known to be suboptimal.

In languages with option types, if you want to weaken the type requirement for a function parameter, or strengthen the guarantee for a return type, you have to change the code at every call site. E.g, if you have a function which you can improve by changing

- a parameter Foo to Option<Foo> or

- a return value Option<Bar> to Bar

you would have to change the code at all call sites. Which could be anything between annoying and practically impossible.

In languages that solve null pointer errors instead with untagged union types (like TypeScript or Scala 3), this problem doesn't occur. So you can change

- a parameter Foo to Foo | Null or

- a return value Bar | Null to Bar

and all call sites of the function can remain unchanged, since the type system knows that weakening the type requirement for a parameter, or strengthening the promise for a return type, is a safe change than can't cause a type error.

So yes, option types do avoid null pointer exceptions, but they solve the issue in a very suboptimal way.

A couple million lines of code?

Try a better programming language next time, dagnabbit!!!

(There will be downvotes I suppose. More lines of code the better?)

I just wanted to leave a glowing comment about Mercury itself, since this is one of the few times I’ll be able to.

I’ve been using Mercury for 5 years. In that time, I’ve been able to wire transfer money without having to worry it might disappear (functionally impossible at certain other banks), created hundreds of virtual debit cards each with their own limit and pulling from different accounts, created dozens of accounts (a “place to put money”) named by function (each of my household utilities gets its own account, with an automatic rule to pull in money whenever it gets paid out), and… well, I think that covers everything.

This has given me unprecedented insight into my financial life. I know exactly how much I spend on groceries, on each utility, and on entertainment. I can project ahead and get a burn rate for my household. And my ex wife uses it too, on the same login, which is as easy as “make an account named with her first name” and a corresponding virtual debit card.

I’m convinced the only reason people don’t use Mercury is that they don’t know what they’re missing.

You have to pay for personal banking (a couple hundred a year iirc), but the business banking is free. If you want to try them out, you can start an LLC for a few dollars (at least in Missouri) and get overnight access to Mercury. All that’s required is your EIN.

They’ve been one of the single best products I’ve ever used. The sole wrinkle was when they canceled all their existing virtual cards due to reasons, which threw my recurring billing into chaos. But every great company is allowed at least one mega annoyance, and that one was a blip.

If you’re wondering whether to try them out, the answer is yes, and I’m excited for you to discover how cool it is. https://www.mercury.com

[flagged]

[dead]

[dead]