A 3D fruit fly on macOS desktop powered by the real FlyWire connectome

I appreciate the open source approach over the startup's sensational claims. This is more transparent.

That said, I think it still presents itself as if the fly is being controlled by the connectome, when (if I'm reading the source correctly) it's more accurate to say scripted behaviors have been hooked up to be triggered by the connectome.

Put another way, you might as well have the "escape" trigger open a YouTube video of a fly flying away. That would probably be less misleading too.

Related (fictional):

https://qntm.org/mmacevedo

MMAcevedo (Mnemonic Map/Acevedo), also known as Miguel, is the earliest executable image of a human brain. It is a snapshot of the living brain of neurology graduate Miguel Acevedo Álvarez (2010–2073), taken by researchers at the Uplift Laboratory at the University of New Mexico on August 1, 2031

I have no exposure to connectomes - or really anything quite like this (as opposed to good ol’ fashioned perceptrons) - so forgive my following uninformed question:

Can someone explain if this software is ethical?

> What's modeled vs. measured

> Honesty section:

I wouldn’t mind an human written README

Interesting work! There's NeuroMechFly (https://github.com/NeLy-EPFL/flygym/) available, you might try using that to simulate the fly body in real time as well.

Ah, Mr. Acevedo, nice to see you.

Fall; or, Dodge in Hell comes to mind

No demo video?

At some point in the future would it be possible to simply download a DNA sequence of an entire animal as a "ROM" and boot it into a virtual "reality"?

What detail level of reality would we have to emulate for an egg to grow into an adult animal?

Would we be able to have a "digital human" when we reach the level of creating this kind of brain mapping for the human brain?

Honestly I think we're still discovering how powerful neural networks are and certainly they can do computations that sound very close to intelligence in some contexts, but I still find it hard to think that we can simplify the whole reality of an incredibly complex chemical machinery down to a bunch of matrices multiplications.

Hi, im the author of this repo, thank you for posting here!

This has to be some of the coolest stuff I've seen on HN in a while. I'm running 3 flies on my desktop right now, all powered by realistic brains.

quite interesting, very small neural nets mimicking the brain of a fruit fly

Things like this are why I come to HN! Cool hack :)

does it simulate hormones as well?

Oh nice, might use this to create some art for my home.

https://www.instagram.com/p/DMfi-9Vsl-u/

> Infinite Pain is a computer running a deterministic neural simulation of a microscopic nematode worm’s brain (c. elegans), subjected to regular pain stimuli.

next up: https://openworm.org/

It's cool and all, but I have real ones in my kitchen, and I'd prefer to keep the computer room free of them if it's all the same to you.

Now this is real AI.

I made a housefly simulator on the Mac once, using a visual programming language called Bounce / Body Electric, developed by Chuck Blanchard at VPL Research, originally used for VR simulations with the DataGlove and Eyephone and a pair of SGI workstations to render each eye.

https://en.wikipedia.org/wiki/VPL_Research

The simulated fly's purpose was to annoy the simulations of Rush Limbaugh and Jesse Jackson by landing on their heads, who were watching TV and arguing about the closed caption stream. I wrote to Jaron Lanier about it 27 years ago:

    At 12:31 AM -0400 7/8/99, Hopkins, Don wrote:

    Hi, Jaron. We've met briefly once or twice - I'm a friend of David
    Levitt's.

    What's this about Sun acquiring the rights to the VPL patents, and
    Body Electric?

Jaron Lanier's VPL and Body Electric page:

https://web.archive.org/web/20051217153424/http://www.advanc...

>VPL had problems maintaining quality in our hardware- though I still sometimes find our headmounts and gloves in use. But our software was really great and can still offer a lot. A challenge to VRML people: Don't you DARE turn out more VR tools that aren't at least as good as this old stuff!

    Does Sun actually have the Body Electric source code? What version do
    they have?

    Does anybody at Sun even know how to compile a Mac program?

    Last time I worked there (admittedly a long time ago), they were too
    embarrassed to allow Macs in the building (which might make somebody
    realize how bad Unix sucks in comparison).

    Has anyone tried to rewrite it in Java?

    When I was working with David at Levity and Interval, I totally
    overhauled the source code to Bounce: porting it to the latest version
    of CodeWarrior and the PowerPC, cleaning up C code translated from
    Pascal, that no human had ever touched before, and just generally
    re-indenting and adding white space so it was pretty to look at.

    Then I implemented a new interface for plugging in DM's, based on
    ActiveX (yes, ActiveX runs on the Mac).  I added a new data type that
    you can flow along blue wires: a COM object (aka a plug-in ActiveX
    object in a shared library).  

    Then we made plug-ins modules that produced and consumed the new
    plug-in data types, like strings and polymorphic dictionaries.  With
    these new data types, we were able to model very complex simulations
    as nested trees of dictionaries, strings and numbers, treat
    dictionaries as high level objects, pass them all around on wires,
    reading and modifying them at will.

    The thing that the original Body Electric was missing is a way to
    dynamically model structured data like Lisp s-expressions and
    association lists, which is now possible in Bounce, using
    dictionaries. (The swivel3d trees just don't cut it for representing
    "knowledge".)

    Bounce still had the "M4" Director player rendering engine, but it
    didn't do everything we needed, so I implemented my own graphics
    library for drawing sprites, playing sounds, and stuff like that.  We
    used it to implement a simulation of Rush Limbaugh and Jesse Jackson
    watching TV and arguing over the closed captioning stream.

    A big fat housefly would skitter around the screen, land on their
    faces, and tickle them into waving their hands around and
    pontificating.

    I just got a 400 mhz G3 powerbook, and fired up a 2 year old copy of
    Bounce and the crazy Limbaugh/Jackson demo, and it still works,
    actually like a bat out of hell! I ran into David and John Szinger
    (another Bounce programmer from Interval) a few days ago at a 4th of
    July party, and they really got a kick out of seeing the old demo that
    we worked together on, still running!

    I live in Oakland just south of Berkeley. Drop me a line if you're in
    the bay area, and would like to see what Bounce has evolved into.

        -Don

    From: Jaron Lanier
    To: Don Hopkins
    Sent: Thursday, July 08, 1999 6:13 AM
    Subject: Re: Body Electric lives?

    Hey there, and thanks for writing!

    Yup, Sun owns VPL and Body Electric, and my guess is that if anyone
    looked very closely it would turn out Interval doesn't have rights to
    Bounce.  But no one is likely to look very closely, so let's forget
    about that.  (I don't think Sun is aware of Bounce or the work at
    Interval.)

    I had never asked what was done with Bounce at Interval- it's
    fascinating to hear what you were up to.  I live in NYC for the most
    part, but I'd love to see it sometime when I'm in the Bay Area.  Also
    Chuck Blanchard lives in SF and you and he might want to trade demos
    sometime.

    There IS a community of Body Electric users.  It is STILL building the
    most interactive 3D virtual worlds of any tool (though Alice, from
    Carnegie Mellon, is the other hot contender).  That's SHAMEFUL!  While
    BE sucks in every other way, all the more recent vr design tools,
    especially the vrml ones, simply avoid the problem of deep
    interactivity. How could the community be so whimpy, at this late
    date?

    On the Body Electric side of things, there has also been some work
    updating to CodeWarrior/PowerPC (shame we didn't share that work!), as
    well as support for Quickdraw3D, OpenGL on the Mac, OMS, and some
    other standards.

    There have been some changes to the interface, but not as ambitious as
    yours.  The neatest thing is a debugger/tracer tool that is a pleasure
    to use.  Since BE is used mostly in 3D domain, there are also some
    tools dealing with textures, lights, etc.

    I had always thought the swivel tree was ridiculous, of course, but on
    the other hand I liked the idea that the virtual world and the
    knowledge base were the same thing- that unity encourages the
    visibility and grabbability of the underlying concepts.  I think the
    brain works that way- there isn't some barrier behind which everything
    gets abstract- instead, it's user interface all the way to the bottom!
    What I think would be the coolest long-term destination of BE would be
    extending the scenegraph so that it was as powerful a knowledge base
    as you'd want...

    I'd LOVE to see a fancy BE release, in JAVA, or at least spitting JAVA
    out.  The question, of course, is where the money would come from.
    I've tried to talk Sun into an open source release so the community
    could hack on it voluntarily (the source was included in the patents
    that were granted, so it's actually already released by the US
    government anyway, though only on paper).  Unfortunately, Sun doesn't
    want to do that.  What they've said instead is that I can choose up to
    six sites with free hacking privileges at a given time.  Bizarre!
    With so few sites, I think there'd need to be money to make sure the
    sites stayed focused on it...  You'd have thought Sun would know
    better by now.

    The body electric community is surprisingly NOT entertainment
    oriented, though I and a few others still use it that way.  There are
    people hidden away who are still using it for ergonomic simulations,
    simple surgical planners (because the fancy systems are too rigid to
    model some situations), cognitive test rigs, and some work with kids.

    You should know there was some tension about BE/Bounce at one point.
    The problem was that Chuck Blanchard wasn't credited as the lead
    designer/programmer of BE/Bounce when David brought the program to
    Interval.  Chuck's name was reduced in stature in the "about" and he
    was not mentioned in some important semi-public demos at Interval.  He
    was also offered a pseudo-position to help with Bounce at Interval,
    but without health benefits - and Chuck has MS and absolutely NEEDS
    health insurance.  I at one point yelled at the guy who runs Interval
    (forgot his name..)  about their treatment of Chuck- and David is
    STILL mad at me for making a fuss about it.  But I felt I had to.

    So that's the scoop!  Both sides of the mystery revealed!

    All the best,

    Jaron

Homepage of Jaron Lanier:

https://web.archive.org/web/20050212085044/http://www.advanc...

Here are three goals for "good VR" from David Levitt, with whom I work at Pantomime Corporation, and who used to work on VR at VPL:

https://news.ycombinator.com/item?id=8023893

[dead]

This is absolutely astounding. What an implementation. A connectome is like a full brain scan!

I am asking Claude to port this to the browser and I'll host it, so you can try it without installing anything.

Some details about the port: Claude is able to do porting tasks like this. I'll put up a link when you can try it in your browser. I was able to get it to port the complete BitNet b1.58 2-billion parameter model several versions ago (in February, so this would have been Opus 4.5) to browser, so it can do so. This will run on the browser. I'll have it make it so that on mobile you can tap the fly or turn on the camera and it sees you, and same on desktop actually. Since the connectome is only 668 neurons, this will run super fast. Actually I'll make it allow for several flies, and let you drop food. I did something like this before here[1] but using the real connectome is something very different and very amazing. I'll have it add an experimental 3D mode with threejs, I'm not sure if that is also possible or not, we'll have to see it. This size of task usually takes Claude a few hours, so check my comment in the morning for the implementation, I'll reply here once it's done and hosted.

[1] https://stateofutopia.com/papers/1/evolving-brains-cull-long...

It's weird to write MacOS specific software in this era.