Hacker News new | past | comments | ask | show | jobs | submit login

Huggingface tokenizers (e.g. BertTokenizerFast which can load the BERT model vocabularies) also can provide the offsets into the original text.



Consider applying for YC's W25 batch! Applications are open till Nov 12.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: