Great question! We explored local LLMs (including llamafile-type solutions) in our early development, but found that the reasoning capabilities and consistency weren't quite there yet for our specific needs.
That's why we currently optimize for cloud AI models while implementing intelligent plan caching to significantly reduce API costs. This approach gives you the best of both worlds: high-quality execution plans with minimal API costs, plus much faster performance for similar actions.
Running a 7b coder in laptops with 4060 is possible and with very good results. Orra looks like a very good tool to be integrated with any IDE. Take a look at this: https://github.com/huggingface/llm.nvim -- it has a backend option. Ollama exposes a REST API, I think you guys should support it :)
According to eurostat [1], Germany has one of the most expensive electricity (if not the most expensive due to a negative tax in Ireland) in Europe at ~0.35 to 0.40 EUR per kWh.
28.72ct/kwh is the cheapest for my location and 45.51ct/kwh if im in the Grundversorgung(if i fall out for whatever reason out of my regular contract this is the fallback)
From the demo on the homepage, I feel like, it would be much easier to call the trusty old mail(admin, new, registration) func on that trigger... than to go through all that hustle and bustle.
reply