The reason L2s can scale that high is every machine doesn't have to process every transaction.
Each L2 is its own chain that uses its own sequencer/s. The blocks from this chain are then compressed into a single blob of data with a ZK proof that the transactions are valid. The validators on L1 only need to verify that ZK proof matches the hash of the submitted data, which can be done in a few ms even if the L2 did 1M+ transactions.
