I'm really interested in which "intelligence" metrics can't be trained/learned. It seems obvious to me that playing with the specific blocks or diagrams used in an IQ test helps; and that practicing with one set seems to make me better able to do well with another only indirectly related set.
I'm not an expert, but I researched this once. You can train any particular question that's asked at an IQ test, but someone can always think of new IQ test questions that you will fail if you're not smart enough (i.e. your brain adapts to new problems fast enough). There is a training program (I forgot the name) that trains you at solving IQ test problems sort of in general but it's unclear if it just trains people at solving the sort of questions that IQ test people tend to think of.
The way I understand it the most reliable way of training for plasticity if it is possible at all is to learn as many different things as you can. That means: Learn to program, learn a language with a different root (Chinese), learn to dance, learn woodworking, learn to practice law, learn to do team sport, etc.. Basically have as many hobbies as you can. (Switching every X months) making sure they have as little in common as possible.
Here's a talk on how long it takes to become 'good enough' at anything:
You can develop your own IQ tests if you want, but chances are any IQ test you create will have a high concordance correlation with other IQ tests, and chances are the predictions you make from your new test will match the predictions of other IQ tests. In other words, it will probably validate the suspected construct behind IQ tests.
The construct behind the IQ test is known as g, or general mental ability, as opposed to domain-specific ability. While it's true that repeated practice with a specific IQ test will boost performance, it won't boost it that much, and it won't generalize very well -- that kind of domain-specific improvement is not that interesting.
One interesting IQ test you might want to peek at would be Raven's matrices. Raven's matrices involve a 2-dimensional grid of figures which progressively change due to some hidden factors. The goal is to rapidly generate hypotheses as to what factors might be controlling for the figures, and to make a prediction as to what comes next.
On an aside, I think that the golden objective of AI is to develop a general thinking machine. Such a machine, when bounded by a goal, would be able to develop its own hardware drivers or make software better than itself. It would be able to tackle a game like Go without fine-tuning by expert Google employees, and that same machine would be able to look at scientific data and develop its own causal theories -- as opposed to a domain-specific Go machine, or a domain-specific Chess machine.
