Speculative decoding does this to an extent - using a smaller model to generate its own predictions and putting them in the batch of the bigger model until they diverge
It doesn’t. It simply trades compute efficiency by transposing matrix multiplications into “the future.” It doesn’t actually save FLOPs (uses more) and doesn’t work at large batch size
Does anyone even care? Really, who cares? The truth is nobody cares. Saving FLOPs does nothing if you have to load the entire model anyway. Going from two flops per parameter to 0.5 or whatever might sound cool on paper but you're loading those parameters anyway and gained nothing.
FUTO is an organization dedicated to the development of software that returns control of computers and technology to the people. We’re particularly interested in giving people more privacy and control from big tech. We also give grants to open source projects in line with our mission (see our site).
We just launched Grayjay, an Android app with a universal subscription feed for all creator video platforms that lets creators and audiences be less reliant on a single platform, and makes directly supporting creators easier than ever.
We're hiring an electron/vue and/or android engineer to help bring the product to more people
FUTO | https://futo.org | Austin, TX or Remote | Full time and interns
FUTO is an organization dedicated to the development of software that returns control of computers and technology to the people. We’re particularly interested in giving people more privacy and control from big tech. We also give grants to open source projects in line with our mission (see our site).
Yes! It's not on the site yet but we recently gave microgrants to VLC, FFMpeg, the Blender Foundation, as well as other non-US projects. We also just had a fellow[1, applications now open] from Lithuania. (I work for FUTO)
That's interesting, some time ago I saw a video about a live captioning system in Louis Rossmans (youtube id r09Hm2zd2lY) where someone asked the same question and the answer was that they only accepted local applications.
Did I misunderstand his answer or is this something that changed?
Asked him and he said he was talking about having to travel here to do the fellowship. "If you are international, we will accept you as a fellow and help you come to texas, but the fellowship program will not allow you to be in the fellowship program if you want to stay in your home country throughout the program."
