There are possible physical answers; it could be lack of inertia on the hair triggers growth. It could be that when you rub your arm, an action as innocent as a yawn, you are also informing the growth of hair. Could it be as simple as looking at your hair or knowledge of shaving causes a subconscious trigger? We already know that these high level shortcuts into low level processes exist (Pavlov's dog anyone?)