I did some cursory research on how important 'forgetting' is to neural networks. I didn't have conclusive results because I just played with a few ideas, never really making it serious, but the idea has stuck with me as important for a long time now. In a finite space the pigeon hole principle says you eventually must take something out to put something new in so finding algorithms that efficiently forget are likely key to learning. I think this applies to neural networks and, I think, to society as well. You can't preserve every building for history. You can't remember every grievance or act of heroism. Eventually you have to forget to move forward. At least that is the thought.
