The comparison to colorblindness isn't precise: colorblind people don't have relative color perception. In terms of information content, over an entire song the information encoded in the scale is very small compared to the information encoded in the relative pitches of notes (in fact the ratio goes to 0 for long music). A grayscale image contains significantly less information, essentially a 2D chrominance array; the ratio of information is roughly constant and does not diminish for large images, or say a series of images (e.g. a grayscale movie). It would be more like applying unknown hue rotations [1] each time you look at an image.
[1] Example I found: https://cms-assets.tutsplus.com/uploads/users/1251/posts/259...