Hi, one of the authors here! Thanks for your question – you raise a good point about inference and model size.
We touch a bit on potential applications in section 8 of the report ("Future directions"). One of our main focuses is on autonomous troubleshooting agents that can query and reason about metrics. In this context, the size and cost of Toto is considerably smaller than that of LLMs already in use for similar workflows. In future work, we envision using Toto in a multimodal context where expect the time series backbone to represent a manageable proportion of the overall compute budget.
We are also exploring how to use Toto to improve our anomaly detection and proactive alerting solutions at scale, and you're right that more work remains there in terms of inference efficiency.
The size of the model and the cost of inference seem steep to provide forecasting on metrics. Excited to see the work to optimize this further.