Yup. Soon after I heard about curve fitting, I thought, maybe we can always fit a polynomial! So, for 2 points, we can use a polynomial of degree 1, 3 points, degree 2, etc. Think a little and see how to fit n points with a degree n-1 polynomial. Can just write it down or just program it. So, I rushed to do that.
I typed in a few points and got a nice fit; the polynomial went exactly through the points. Hmm ... I couldn't be the first to think of that! So, I typed in points something like a square wave and thought, "Let's see a polynomial fit that!". Well, it did fit -- a polynomial fits a square wave -- gotta be kidding.
So, I plotted out the whole thing. Below, spoiler!!!!
Sure, the polynomial went through the points but between the points it shot off toward either positive infinity or negative infinity and made a lot of progress before turning around.
Then there are spline fits, least squares spline fits, multivariate spline fits, and, ..., ironically, neural network fits.
It's important to recognize that the space of all programs that take x,y as input and then output 1 or 0 is equivalent to the mathematical space of all possible functions. We interpet the programs as indicator functions.
Neural networks are usually smooth because they must be differentiable. Stochastic binary neurons and other techniques take us into a larger space of possible mappings, discontinuous ones.
But yeah, ultimately the model we solve for, will affect what kind of interpolation the solution ends up having.
The biggest question is what kind of interpolating techniques the brain uses. The search for the magic model....
I typed in a few points and got a nice fit; the polynomial went exactly through the points. Hmm ... I couldn't be the first to think of that! So, I typed in points something like a square wave and thought, "Let's see a polynomial fit that!". Well, it did fit -- a polynomial fits a square wave -- gotta be kidding.
So, I plotted out the whole thing. Below, spoiler!!!!
Sure, the polynomial went through the points but between the points it shot off toward either positive infinity or negative infinity and made a lot of progress before turning around.
Then there are spline fits, least squares spline fits, multivariate spline fits, and, ..., ironically, neural network fits.