A VSCode person can (and probably will) answer in more detail, but at heart it's simple: if you want to add type-checking goodness to a library that isn't itself written in TypeScript, you can create a thing called a declaration file: https://github.com/DefinitelyTyped/DefinitelyTyped
Microsoft publishes a list of known good declaration files for popular npm packages to npm, under the scope @types: https://www.npmjs.com/~types
The 1.7 release of VSCode helpfully tries to automatically load type declarations for any npm package you use by requesting the equivalent declaration package under @types. When the package exists this is fine, because it's cached in our CDN.
What they forgot to consider is that most CDNs don't cache 404 responses, and since there are 350,000 packages and less than 5000 type declarations, the overwhelming majority of requests from VSCode to the registry were 404s. This hammered the hell out of our servers until we put caching in place for 404s under the @types scope.
We didn't start caching 404s for every package, and don't plan to, because that creates annoying race conditions for fresh publishes, which is why most CDNs don't cache 404s in the first place.
There are any number of ways to fix this, and we'll work with Microsoft to find the best one, but fundamentally you just need a more network-efficient way of finding out which type declarations exist. At the moment there are few enough that they could fetch a list of all of them and cache it (the public registry lacks a documented API for doing that right now, but we can certainly provide one).
> At the moment there are few enough that they could fetch a list of all of them and cache it (the public registry lacks a documented API for doing that right now, but we can certainly provide one)
Might I suggest having a bloom filter containing all the existing type declaration (which would be quite small) and only querying the registry if the bloom filter reports the type declaration as a positive.
Since the filter can be really small it will probably scale a lot better than a complete list of all type-declarations, and a new filter could be downloaded by the clients every now and then.
Depends on the bloom filter, but for the fixed size, fixed hash functions, and other implementations in the same general vein, it would just be XOR of both of the bloom filters. Anything that comes out true is missing from what would be the combined filter. Slightly different for counting filters, where something along the lines of subtraction would get you what you need (anything non-zero is different).
404 generation needs to be efficient for cases like this.
Indeed!
In general, it is an extremely efficient response. It took a huge number of users all hammering on the same set of 404 handling routes to get our attention, and we were able to handle the load, though it wasn't trivial to do so. The end user impact was minimal.
If it hadn't been a known-good actor, we had some options to shut down the flood a bit more forcefully, but we didn't want to inadvertently cause errors for vscode users. Like my colleagues have said in this thread already, we really dig what VSCode is doing, and as operational fires go, this one got put out very swiftly and did very little harm.
All that being said, knowing the npm devops team, this will no doubt be a source of insights for making the registry even more resilient in the future :)
If your CDN supports stale-while-revalidate and stale-if-error, you should consider enabling them -- it will take the load on your servers from O(users * packages) to O(POPs * packages)
"There are any number of ways to fix this, and we'll work with Microsoft to find the best one..."
It's refreshing to read actual engineers' writing. After this, going back to tear-jerking snark-filled twitter and medium gnashing of teeth will be hard.
Yeah, you're right. It was a simple oversight that we hadn't been caching 404s already, since we already have infrastructure in place to bust cache on publishes. It would have been our next step if necessary (it wasn't necessary to mitigate this flood).
You optimize for the use patterns you anticipate or see in normal usage, because, well, see famous saying about premature optimization. The use pattern we see most often is people installing from pre-determined lists in package.json, so 404s aren't all that common ordinarily.
Microsoft publishes a list of known good declaration files for popular npm packages to npm, under the scope @types: https://www.npmjs.com/~types
The 1.7 release of VSCode helpfully tries to automatically load type declarations for any npm package you use by requesting the equivalent declaration package under @types. When the package exists this is fine, because it's cached in our CDN.
What they forgot to consider is that most CDNs don't cache 404 responses, and since there are 350,000 packages and less than 5000 type declarations, the overwhelming majority of requests from VSCode to the registry were 404s. This hammered the hell out of our servers until we put caching in place for 404s under the @types scope.
We didn't start caching 404s for every package, and don't plan to, because that creates annoying race conditions for fresh publishes, which is why most CDNs don't cache 404s in the first place.
There are any number of ways to fix this, and we'll work with Microsoft to find the best one, but fundamentally you just need a more network-efficient way of finding out which type declarations exist. At the moment there are few enough that they could fetch a list of all of them and cache it (the public registry lacks a documented API for doing that right now, but we can certainly provide one).