Hacker News new | past | comments | ask | show | jobs | submit login

Have you thought about integrating the macOS accessibility API for either reading text or performing actions?



No, my thought process never really stretched outside of what I built. I had this particular idea, then sat down to build it. I had some idea of getting OpenAI to respond with keyboard shortcuts that the application could execute.

E.g. in Photoshop: "How do I merge all layers" --> "To merge all layers you can use the keyboard shortcut Shift + command + E"

If you can get that response in JSON, you could prompt the user if they want to take the suggested action. I don't see myself using it very often, so didn't think much further about it.




Consider applying for YC's Spring batch! Applications are open till Feb 11.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: