The idea here is that if you have segment:offset pointers, like x86 real mode, then you (as a compiler writer) don't have to worry about the segment part at all - you can just subtract the offsets and shift, because pointers within the same object will have the same segment part.
Just nitpicking, but in real mode it is at least conceivable to implement arbitrary pointer arithmetic (at some significant performance hit) because segments are spaced evenly every 16 bytes.
The real killer is arbitrary segmentation, where software can say this segment start here and that segment starts there, like, say, the x86 protected mode. Is the OS going to expose absolute segment addresses to the C language to allow some "defensive programming" like checking if a pointer really points to the array you think it does? Hell no, you write your damn code correctly ;)