That's not quite correct. There were also 'Latvian vikings', the Curonians. Latvia was also ruled by Sweden for centuries. There's not much difference there, apart from language. They were a mix of Baltic tribes, some combination of which formed the modern nation states.