On: I made a transformer to predict a simple sequence ...

[civilized]

The AI community has over-learned this bitter lesson and is now unprepared to make useful AI in data-constrained environments.

This is why almost all of the countless enterprise AI demos taking place as we speak are doomed to failure.