No one in TES's fanbase calls it "scrolls", we call it TES ("The Elder Scrolls"), because that's what Bethesda calls it, over and over and over again. Or we mention the particular games by name and leave out the series title: Daggerfall, Arena, Skyrim, etc.
That is my experience as well. I've never heard anyone refer to it as just "scrolls."
I have heard all of the things you mention, though (TES, The Elder Scrolls, Daggerfall, etc.), but never ONCE have I heard it called just "scrolls." Ever.