I wonder which would actually faster - binary search on the standard ordered alphabet or going linearly in order of letter frequency.
Even better - some hybrid algorithm that goes through the most common letters then goes into binary search for the remaining letters (many of which appear at approximately equal frequency)
You could probably also have a tree of the ~32 most common words. At the very beginning you could have a branch to choose whether you want to pick the 'word' tree or the 'letter' tree.
Even better - some hybrid algorithm that goes through the most common letters then goes into binary search for the remaining letters (many of which appear at approximately equal frequency)