Typesafe.ai released JEV, a model that works 100 times faster than LLMs at 10% of the cost1. The catch? It does not answer with a long text but only with a set of desired outcomes.
You cannot ask JEV to fix your fridge based on a few pictures but it can tell you if you should buy a Toyota or a Mercedes based on your financial status, the amount of debt you owe and where you live.
You cannot ask JEV what are the top five games for the Xbox Series X but it can tell you if, based on your preferences, you should play Cyberpunk 2077 or It Takes Two.
JEV needs only two things: (1) a state and (2) a list of choices. The model makes a decisions based on the state and the potential choices. It does it very fast and cheap.
For example, if you build a safe-driving car, you provide:
state: the position of the car, the destination coordinate, what the camera sees (obstacles), the previous actions
choices: accelerate, brake, park, turn left, turn right
It extremely fast. So fast that it can play doom or control a car.
I think this is a major change and will enable a wild range of new applications.
You can now get a contextual, high-value decision (1) in milliseconds and (2) at a negligible cost. JEV is another steps towards making the switch from deterministic algorithm (think if/then/else) to probabilistic ones (e.g. think model call) transparent in terms of user experience. Nikita said it better than me.
JEV is only one breakthrough that will enable more AI-native applications. Where LLMs were not reliable enough, too expensive or too slow, developers can use JEV. We should expect a few more breakthrough in the coming years, either on the software side (new models, new paradigms) or hardware side (faster processing). The end result is AI-native applications that run locally on your device.
Software engineers that were skeptical of AI should see the release of JEV as a second wake-up call and learn how to build AI applications.
Please excuse if I am slightly off