Not quite. New elements are inserted at the end of the regular line, but at the head of the fast line. When you first enter the regular line, this doesn't matter, because everybody (fast line or regular line) would've gotten there before you in the regular line. However, every person that arrives in the fast line after you take your place in the regular line goes before you, yet would've gone after you if the fast line didn't exist.
Total waiting time is the same, but that's not the stat you care about. Your waiting time, as someone in the regular line, is. And that's increased by each person that arrives in the fast line while you are waiting. Their short waiting times are compensated by your long waiting times.