Show HN: Ggml.js – Serverless AI Inference on Browser with Web Assembly

[anonymousd3vil]

https://rahuldshetty.github.io/ggml.js/

What is ggml.js?

Run any ggml ported ML models directly on your web browser with ggml.js

This project provides JavaScript bindings on ggml models, so that you can embed GGML models to your web apps to build serverless AI inferencing.

Currently the framework supports following model types: - Dolly v2 - GPT2 - GPT J - GPT NEO X - MPT - Replit - StarCoder

You can find more details about the framework: https://rahuldshetty.github.io/ggml.js/

For live demo examples, you can visit here: https://rahuldshetty.github.io/ggml.js-examples/

Source Code: https://github.com/rahuldshetty/ggml.js

Please do provide any feedback/questions/suggestions in the comments. I'll be happy to take it up :)

[louis030195]

Can I run useful llms in the browser?

[anonymousd3vil → louis030195]

Yes, the support is limited at the moment. I am planning to port llama.cpp models in the future.