It's both. It's the lambda calculus and the turing machine.
It's useful to understand both the way that the machine works, down the the assembly level, and if possible below, since it lets you write code that fits the way the hardware works well. It's useful to understand the symbolic side, since it lets you reason about algorithms, and write code that fits the way mathematics work.
Heh, that's a good question. I was thinking in the context of von Neumann arch, but it's a good argument that it's symbolic manipulation. In that case, any language will do... but the best programmers I know almost intrinsically know the von Neumann architecture and they're C programmers too :)
OK, but what is "the core of computer science"? Is it CPUs based on the von Neumann architecture, or is it symbolic manipulation?