Think of it like this: if one person only knows part of a system, they can only reveal so much about it. 15 people might know the entirety of one system, but individually they don't know enough to be a threat. This idea can be stretched out to cover entire levels of security clearance and information (as it does).
The pyramid structure that seems to permeate all things //system// has the following salient feature. Say a system pawn is a node at level $L$ and his/her boss is at level $L+1$ and the boss' boss is at level $L+2$. Depending on social skills and intelligence, the pawn might have information about "what is going on" at level $L+1$ by talking to the boss, but for sure he/she has //no// information about what is going on at $L+2$. In a way secrecy (i.e. information non-awareness) is the //essence// of the pyramid.
Assange has this paper [1], which talks about information being the perfect way to choke the system.
[1] cryptome.org/0002/ja-conspiracies.pdf
[ abstract: Consider the Gonspiracy G = (V,E) where V is the
set of conspirators and E is the set of inside information links.
The conspiracy G is embedded in a larger graph, society, S.
Let |G| be the power of the Gonspiracy. The 'good guys'[2] want |G| to be small, the bag guys want |G| to be large. Assange defines the total conspiratory
power as
|G| = sum information flows in e for all e in E
So to fight G, one must either cut edges e, or generally
reduce the flow of information flowing on edges, by
scaring the nodes {v \in V} that they might be found out.
]
Let's flip that around: it makes it easier for "those above" in the hierarchy to deceive a whole pile of compartmentalized people into performing useless work, and thereby consuming lots and lots of government money.
Seems like that would inevitably lead to everyone in the organization unable to determine whether or not their contribution was useful. (Short of them actually saving a life at the bottom level.)
With no hope of finding meaning in your job, it would make a 40-year career to retirement a long career indeed.
Ever heard of Empire Building? It's a common problem in corporations. Mid-level managers become important by having a lot of underlings. I imagine this would be even more important in compartmentalized areas, as mid-level managers would nominally have no other method to signal their importance to superiors or peers.
I can also think of a situation where a contractor might hire a mid-level manager's husband, wife or child. The mid-level manager would be able to crank up his or her headcount to get the relative a sizeable Christmas "bonus". I've heard tell that this sort of thing actually happens.
The most charitable explanation would be that they have no way even to know if they're doing it, and the tendency of large systems under such conditions is to degrade in efficiency.
