I like the approach that Apple seems to be taking with fine tuned small models that handle routine tasks and then defer to larger off device models for things they can’t confidently do. I imagine you could construct a training set that contains examples that should produce low confidence answers where you could add an output that is essentially a “call for help” option so you could train it to choose that. Smaller models also means you could have more running in parallel and use another to route requests to the appropriate expert.
Reading emails, replying to emails, scheduling tasks, using apis for services.
Basically everything which doesn't need knowledge but actions.
"Tell my wife i'm late" and it will use some configured magic to talk to service xy and just does it.
Siri is very good in doing homeautomatistaion without the internet, the old google agent and alexa were absolutly not and i don't think they were ever available offline.
This basically gives you a local (local-first!) good working assistent
Would be very nice to have my schedule automatically managed by Siri. Already has a few nice things but I genuinely have trust issues, especially with AI.
You can get very far with the Shortcuts app by the way. Some examples: using your current location to estimate when you should leave to get to your next meeting on your calendar,
letting those included in the calendar event know you’re running late. Highly highly recommend it, the learning curve isn’t much, a bunch of drag and drop!
It can be fine tuned for device related actions. In other words, with all the capabilities of your device applications or services, the small model can virtually have the same capabilities. It can always dispatch a user request in way of “natural language” to those applications, and orchestrate the applications. It can dispatch user requests beyond the device capabilities to a cloud model. This is powerful since it changes how you interact with your devices.
I tested the Google AI on my phone, I had the browser open and asked it to read the page to me and it responded that it does not have access to the internet.
So I would like an AI assistant that:
1 can understand english and my native language
2 that is aware that runs on Android(or KDE/Linux) and can understand commands like "open the Android Settings , Application section " or "read the page that is opened in the browser" or "read the text in the popup that is now opened". Basically to be integrated with the OS via public and open APIs. Big AI companies could compete on selling us better assistants especially for multi lingual people.
3 the model should be small , it should not know geography, history, music bands etc, for tasks where the user asks question there should be an option for the model to forward the question to a search engine or even an online LLM.
It could power simple agents like Siri under the hood. Helping with natural language understanding, intent classification, retrieval, and other agent tasks.
