Milk allergies are different from being lactose intolerant. I developed a mild allergy to dairy in my 30 or so - or so it was discovered then. With lactose intolerance, you can at least take a pill - not so much for an actual allergy. My one child has a much more severe dairy allergy and will get a very upset stomach even with a little dairy.
We both have our appendix.
Though to your point, whether you can develop a milk allergy later in life I am not sure.
