Epic Games announces Lore version control system

Git is fine for text based files like code, but it's really bad at stuff like textures, 3D models, audio files, and other non-text files that game developers need to collaborate on. For example, one artist might need to obtain an exclusive lock on some art assets while editing them, because there is no sane way to merge two artists' async edits.

The SOTA in this area is Perforce (https://www.perforce.com/products/helix-core), a proprietary system. From what my gamedev friends tell me, when Perforce works it's great, but it hits enough snags that you need a tools engineer to manage it and occasionally fix issues manually. Git LFS is an alternative, but my gamedev friends all prefer Perforce especially when working on team projects beyond like 3-4 people.

    Enumerating objects: 5, done.
    Counting objects: 100% (5/5), done.
    Delta compression using up to 10 threads
    Compressing objects: 100% (3/3), done.
    Writing objects: 100% (3/3), 290 bytes | 290.00 KiB/s, done.
    Total 3 (delta 2), reused 0 (delta 0), pack-reused 0
    remote: Resolving deltas: 100% (2/2), completed with 2 local objects.

From the documentation, it looks like Lore does a bit better in this regard:

    Pushing 1 fragment(s)
    Pushed 1 fragment(s), 124.00 bytes
    Pushing a3f8c2d1... to branch main
    Pushed revision 1 -> a3f8c2d1... to branch main

Perforce definitely needs a challenger. It is not the incumbent because it is particularily simple to use or administer. Git is actually way simpler when it comes to branching operations for example.

The reasons why p4 is often preferred in gamedev have already been mentioned in other comments: large project support, permissions, file locking and so on. Another key reason p4 is the king for Unreal dev is just how well it's supported inside the engine. Not perfect, but it's the best supported VCS because it's what Epic uses. Even the Git plugin is painfully unfinished, because Epic does not internally use it. So with Lore I expect them to give it first class support. I'd recommend Git a lot more if the support in Unreal was better.

(background; I've been in gamedev for almost two decades now, 2-200 person companies, every kind of engine and version control system. I prefer git where I can use it: for Unreal that means small projects and/or tech savvy team members. Pick the tool that is right for the job and the team.)

>Lore, formerly called Unreal Revision Control, is the built-in version control system for UEFN (Unreal Editor for Fortnite), where creators have been using it to version their islands. It is also seeing progressive adoption by internal Epic teams, and is being implemented as the backing store for UEFN’s cook pipeline, where it replaces traditional intermediary storage layers—eliminating redundant file transfers and significantly reducing the time between publishing changes and those changes being playable.

Surprised it is in Rust and not Epic C++ or Verse. I wonder why.

Oof. So this isn't compatible with any tools that move or rename files. I can't see how this will be acceptable for real-world use.

Interesting to see where this goes.

- Pachyderm (Go): https://github.com/pachyderm/pachyderm

- XetHub (acquired by HuggingFace): https://huggingface.co/blog/xethub-joins-hf

- LakeFS (Go): https://github.com/treeverse/lakeFS

- Oxen (Rust): https://github.com/Oxen-AI/Oxen

I guess with AI, anyone can vibe code a content-addressed, chunk-level deduped, versioning system in Rust these days...

But jokes aside, Lore seems really cool! What's interesting is the realization that different domains/industries have similar problems, but they don't seem to be cross-polinating. In this case AI and Gaming both need a storage system that can version control large binary files at scale. I think there's lots of opportunities to share ideas here, but perhaps the lack of idea sharing (currently) creates opportunity!

It does show its years though. We were one of the first users of the Perforce cloud offering, as we were small and didn't want to self-host ourselves, but it was a bit of a rickety experience. You had to register an Azure account in order to be able to access the service, and you had to ask support to modify things like triggers. Coming from the world of GitHub and other SaaS products, you could tell it was an attempt to retrofit an older model into a new skin.

The Git LFS path has some unofficial support as well, but you are on your own when things go poorly. Epic doesn't provide much help there.

Competition in this space is welcome, especially if they're planning to make it fully officially supported by the Engine.

I wrote about why merging files isn't as common in the world of game dev for folks coming from the world of text: https://www.kuril.in/blog/why-game-devs-dont-merge-files/

Passing for now.

Looks very git-ish. But probably better equipped for large binary files.

    echo "Hello, Lore" > hello.txt

    lore stage hello.txt

    lore status --scan

    lore commit "Initial revision"

    lore push

[1]: https://gavinhoward.com/uploads/designs/yore.md

[2]: My WIP VCS has been named Yore for at least two years; I did not copy Lore's name.

I also don't see any sort of GUI client? So the whole art team is going to have to get up close and personal with the CLI

If the roadmap's "Web client and code review tools" could replace gerrit for me, this would be a easy switch.

Moreover, it looks like they designed both the mutable store and immutable store to be able to easily store their state directly on an s3 like system.

There are a number of features that would greatly speed up CI/CD system operations I belive.

That would be very powerful for various use cases.

[0] https://github.com/EpicGames/lore/tree/main/docs/developing/...

[1] https://github.com/EpicGames/lore/blob/main/docs/developing/...

[0] https://github.com/eblanshey/HistoryWorkbench

Helpful page that gets closer to the details

I'm trying to figure out what Lore can accomplish that git+LFS can't. I've read about big binaries chunking, native interface and permission, is there anything else? Weren't those problems already solvable in the git+LFS ecosystem?

Count on it.

Missed opportunity for Lorehub.

Its remarkable that anyone thought this website was fit for release, and it gives off strong slop vibes

I also have absolutely zero trust in a product like version control being provided by a for-profit company. It seems like a terrible idea to tie your software stack to Epic Games of all people, given their track record

????

It’s like anything you do has to talk to the server

Even something as simple as diffing a file will just hang if there are server issues

Would love any feedback on it or contributions if people are interested :)