What do you mean when you say that standard memory doesn't support concurrent accesses? Assume you have 2 threads running on 2 (physical) cores on a single-socket machine. In what case would the data access be serialized? How much do you think this serialization would affect the latency of a read from memory?
For current Intel, all cores in a socket share L3 cache, but have independent L1 and L2. If the two are using the same address read-only, both will have the data in their own L1. If they are read-write, performance will drop, but there still won't be any accesses to RAM, and the data will stay in L3.
If you are talking about random access, there can be concurrent access on either 2, 3, or 4 "channels". This is physically partitioned, so sometimes data can truly be accessed in parallel and sometimes it can't. But the maximum wait time for a request to be held up is small relative to the total latency of a memory access.
The "two cores at most" part of the saturation statement is trivially untrue, although correct for many systems. On the Sandy Bridge system I just checked (quad-channel DDR3 1600), I did get a small increase on the Stream benchmark going from 2 to 3 physical cores: 15 GB/s Copy for one core, 23 GB/s for two, 25 GB/s for three.
But this overstates the problem somewhat, as perfectly predictable access with prefetch isn't particularly common. For random access, it is difficult to saturate RAM bandwidth unless you pay very close attention to the parallelization of your requests. For dependent access (each request depends on the previous) I'm doubtful that you'd be able to reach saturation on most processors.
I agree with your sentiment though --- it's tough to get the performance increase you'd hope for unless the threads, and paying close attention to data access patterns is key. But I'd argue (and I'm sure you'd agree) that managing data access is crucial to single threaded performance as well.
For current Intel, all cores in a socket share L3 cache, but have independent L1 and L2. If the two are using the same address read-only, both will have the data in their own L1. If they are read-write, performance will drop, but there still won't be any accesses to RAM, and the data will stay in L3.
If you are talking about random access, there can be concurrent access on either 2, 3, or 4 "channels". This is physically partitioned, so sometimes data can truly be accessed in parallel and sometimes it can't. But the maximum wait time for a request to be held up is small relative to the total latency of a memory access.
The "two cores at most" part of the saturation statement is trivially untrue, although correct for many systems. On the Sandy Bridge system I just checked (quad-channel DDR3 1600), I did get a small increase on the Stream benchmark going from 2 to 3 physical cores: 15 GB/s Copy for one core, 23 GB/s for two, 25 GB/s for three.
But this overstates the problem somewhat, as perfectly predictable access with prefetch isn't particularly common. For random access, it is difficult to saturate RAM bandwidth unless you pay very close attention to the parallelization of your requests. For dependent access (each request depends on the previous) I'm doubtful that you'd be able to reach saturation on most processors.
I agree with your sentiment though --- it's tough to get the performance increase you'd hope for unless the threads, and paying close attention to data access patterns is key. But I'd argue (and I'm sure you'd agree) that managing data access is crucial to single threaded performance as well.