On: Ask HN: Is anyone doing anything cool with tiny la...

[politelemon]

Could you recommend a tiny language model I could try out locally?

[mettamage → politelemon]

Llama 3.2 has about 3.2b parameters. I have to admit, I use bigger ones like phi-4 (14.7b) and Llama 3.3 (70.6b) but I think Llama 3.2 could do de-anonimization and anonimization of code

[RicoElectrico → mettamage]

Llama 3.2 punches way above its weight. For general "language manipulation" tasks it's good enough - and it can be used on a CPU with acceptable speed.

[seunosewa → RicoElectrico]

How many tokens/s?

[iamnotagenius → seunosewa]

10-15t/s on 12400 with ddr5

[OxfordOutlander → mettamage]

+1 this idea. I do the same. Just do it locally using ollama, also using 3.2 3b