It still has the knowledge from the main training on data from across the whole internet, so would still know the word Shakespeare...
But you're right - the model finetuned on shakespeare would be good at writing a new play in the style of shakespeare, but would be bad at giving a critique of shakespeare's works.
