The worse is that this error only happens because we are trying to save 4 bytes. If all arrays would also store their own size we could eradicate this bug with a limited memory impact.
You have overlooked the CPU cost. Given that most memory objects already have at least an over-approximate size somewhere, and how we are more than happy to annotate stacks with canaries, I really don't think anyone is as concerned with the memory cost of adding a size to every array as the CPU cost of adding a read and compare to every array access.
I think that the CPU cost could be reduced if the CPU had some kind of instruction for array access:
if you 'hardcode' that an 'array' has its address stored in register Rx and its length stored in Rx+1, you can easily have an instruction "checked_load Ry <- Rx, <index>" this instruction can have a latency similar to a normal load instruction as it could check the value of the index with Rx+1 in parallel to the load (1).
And if you have a 'load aligned double word' instruction which load two consecutive word in Rx and Rx+1 (aligned to avoid the thorny issue of page missing in the middle of an instruction), you can reduce the cost of having to manage both the array address and its length.
Sure this isn't free: more registers used, more cache used, still it doesn't look THAT bad from a performance POV no? Or very complex to implement. Is-there something I'm not understanding?
1: and if the checks fails there is TRAP executed, which is compatible with C/C++ semantic.
I think the problem a lot of people would have is that introducing hardware, which costs millions to design and bring to market, when it's possible to write code that isn't prone to overflow....just doesn't seem like a good idea.
When I was in school, my professor for CPU Architecture had a good saying: "If you're going down to NAND gates to debug your software you're doing something wrong".
I think another issue is that arrays, trees, and other structures are a complete abstraction. The CPU doesn't need to know a thing about them for them to work, because all it actually sees is addresses and instructions (I get that I'm grossly oversimplifying things), and so introducing that type of "check" at the hardware level is pushing the abstraction down, and I don't think that's a good idea either.
But I do think that's one of the great reasons for using a higher level language, where there are array bounds checks, etc.
To me, C/C++ is like "pretty" assemby language (Now with new features like Undefined Behavior!!!), and I like that about it. But I'm not sure how other people feel.
Anyways, that's just my take on it, take it with a grain of salt.
I am not necessarily suggesting checking array boundaries on every array access, but at least providing an easy and safe way to refer to the array size.
Array boundary checking is I think desirable but I can understand that for some applications it is an unacceptable performance hit.
> Array boundary checking is I think desirable but I can understand that for some applications it is an unacceptable performance hit.
Given the severe potential security implications for not using it, enabling bounds checking is a sensible default in my opinion. People always mention the speed of C for example but ignore what is being sacrificed for it.
You can get pretty tough protection with address sanitizer (it works technically a bit different). It doesn't come for free however.
It's much faster and better than anything that was there before (e.g. valgrind), but the performance impact is still significant. Still I think address sanitizer is an amazing project that should definitely get some more publicity.
What!? Size of the array is always known to the programmer in C. In fact, if the programmer wants to get rid of that information, he/she must deliberately do so. Once one does that, the array becomes useless.
You also haven't explained how would that size information, which we already have, magically solve the problem.
A large part of overflows come from querying an array beyond its size. If the size of the array was already contained in the array (think of C# arrays which always have a Length property), developers could build more robust code that wouldn't rely on the size of the array being inferred from some other parts of the code.
Many, many, many moons ago, C strings won out over Pascal strings. If the competition was somehow re-run from scratch today, with modern machines, it would be no contest: Pascal strings would win. But there was a time when every byte was precious. And not just "a bit precious", but very precious.
You're in the right general direction. The replies to you are misleading and show the common problem of INFOSEC people having no knowledge of what was achieved in the past or present for immunizing systems. Systems far back as 1961 were immune to this with that one using 3 bits to do it [plus code/data separation]. CPU cost when done in parallel is almost zero. See my main comment.
