TLDR: during training back propagation means we need to send data 2 ways: input data from input => output, and error data (wanted_result - output) from output => input, updating the weights of the network to work better.
This is the main source of performance bottlenecks and also an obvious difference between natural and artificial neural networks. The brain does not even seem to have error signals, and we also can't seem to find any signal going in the opposite direction of propagation. It requires that you have an error signal ... and that means it requires knowing the right answer to the problem the algorithm is trying to solve. Also quite important to some companies: sending data in 2 directions through the network places serious limitations on parallelization of neural network training. It is one of the big causes that Google/Facebook/MS(OpenAI)/... only seem to have a 1y or less headstart over the rest of the industry, despite billions of investment.
Forward - forward tries to do online learning by training the network to differentiate between real and fake-but-really-realistic signals with data going in the same direction every time.
This is the main source of performance bottlenecks and also an obvious difference between natural and artificial neural networks. The brain does not even seem to have error signals, and we also can't seem to find any signal going in the opposite direction of propagation. It requires that you have an error signal ... and that means it requires knowing the right answer to the problem the algorithm is trying to solve. Also quite important to some companies: sending data in 2 directions through the network places serious limitations on parallelization of neural network training. It is one of the big causes that Google/Facebook/MS(OpenAI)/... only seem to have a 1y or less headstart over the rest of the industry, despite billions of investment.
Forward - forward tries to do online learning by training the network to differentiate between real and fake-but-really-realistic signals with data going in the same direction every time.