On: Finding CSV files that start with a BOM using ripg...

[ChrisSD]

Note that UTF-8 wasn't actually standardized until Unicode 2.0 in 1996. This was at the same time as the surrogate pairs needed for UTF-16. And UTF-8 didn't find its final form until 2003, which was around the time when it really started to gain legs.

However, as you say, by 1996 people were already using the older UCS-2 standard.

[Dylan16807 → ChrisSD]

It wasn't in unicode until later, but there was a published spec in 1993.

And sure they updated it in 2003 but "don't use invalid codepoints" is not a really notable update.