It's more the reason why you avoid eye contact. Looking nervous while shifting eyes to avoid eye contact looks suspicious. Avoiding contact because you are otherwise distracted thumbing through forms in a clipboard makes it look like you have a purpose for being there.