Ollaya – Ollama for open-source, Jev-style decision models

I'm not sure what this means for AI startups if their innovations can be copied by OSS so quickly (what, like 2 weeks?). There's "consumer surplus" for everyone, to borrow an economic concept. But we do ideally want some of the surplus to flow to the innovator, too. I know there were precursors, but that's fine - it's hard to have a totally novel idea in such a popular field. I don't know what the end game is for TypeSafe - they'd need to demonstrate perpetually better results, or compete in another axis: UX, support, custom solutions, etc. So much of the time, someone proving a concept, or it simply getting enough publicity, is enough for a "Cambrian explosion" of follow-ups and copies. Famously, that was true for "Attention is All You Need", and the general idea of "next-token prediction" being so powerful.

We've stumbled into general differentiable models..

Has anyone actually seen better or the same results with Laya compared to Jev? From my experience, Laya performs significantly worse. It's less confident and often makes wrong decisions with more complex queries.

Guys I have a real q, what is the difference between an instruct based re-ranker and laya/jev I just don't see it.

Edit: One is that jev/laya are tuned to have better probabilities, but a reranker can be fine tuned to do that as well. And jev/laya use RLCD?

I installed it, I tried the examples, it works.... But forgive my lack of imagination... what is this useful for?

Like, their example is of classification for a support interface.... `refund_requested`. Pretty convenient bool given the example is about a refund- what if 99% of submissions don't ask about a refund? Also, is that user not a `churn_risk`? What could possibly qualify as a churn risk if not a user asking for a refund?

https://ollaya.dev/library/laya The examples suffer the same problem of why I'd prefer to use a string column vs an enum. Changing an enum means you need to update the db, using a string you can do whatever.

I'm not trying to be negative, I genuinely want to know about some practical examples (that don't require tons of backwards maintenance).

For everyone dismissing Jev's innovation as being trivial, no it's not.

It is definitely not the MNIST classifier you had trained in 2019.

The difference is that you only train it once and the modern LLM machinery sort of takes care of that with large contexts.

It's great that Jev proved this is a viable product. I'd expect a great many research innovations coming from making this work better/faster/cheaper, and around interfacing modern agents with it.

>Run decision models locally.

>example is a text classification task instead of a decision

It would be really cool to have LLMs and System One in a single tool - in this case, if Ollama implemented it.

Ollama is for large language models, so this is for large language... yodels?

It would be good to list 1) zero-shot accuracy and 2) latency on the models page . The LLM-based models' latency is probably much higher than the BERT approaches I would assume.

Also curious, it seems from looking at the accuracy scores you gave that it seems to be NLI > Gliclass > Laya (for Bert types)? Why do you seem to feature/recommend Laya more - is Laya better in some way?

Laya is pretty easy to set up on its own without ollaya. I just did that and replaced my current jev API usage to laya running on a GTX 970 with 4GB of vram.

Very small context window, but for some existing small llm work I was doing, it was a drop-in replacement and it makes me happy I can get use out of old hardware I have running.

Sounds good on latency but how is its actual decision quality vs. Jev?

Are there many models that are comparable to Jev for generic decision making?

Smarter move if you have an eval set is to just train a classifier and call it a day.

Cool... but this does seem undermined by the fact that Ollama can add support for decision models at any time.

FAQ[1] says:

> It is an independent project, not affiliated with Ollama.

[1]: https://ollaya.dev/docs/faq

I am fairly confident if Jev-style decision models are seen as prominent (which, they seem to be), Ollama will support them. Surprised the team hasn't implemented this already.

Would be great if you supported CUDA 12; I don't feel like paying $15K to upgrade my GPU right now

Does anyone know what laya multi lang is faster than laya en? I would have thought focusing on a single language would be faster.

I have also tried this and its really awesome

great project for empowering open-source alternatives.

Nice work! Making open models easier to run locally is valuable on its own. Keeping the API compatible with Jev is a thoughtful touch, too.

Why do you need another model-type specific Ollama? Can't Ollama be made to support these models?

[dead]

Hey Claude, make ollama for Jev like models. Make no mistakes /s

This inference engine is soooo much faster btw: https://github.com/tamnd/kime