That would be true if RSA keys were brute forced, but they aren't - e.g. 512 bit RSA takes days/weeks to break on commodity hardware these days, whereas 512 bit brute force (as is essentially needed for ECC these days) takes significantly longer than the estimated age of the universe.
See http://en.wikipedia.org/wiki/Integer_factorization_records