If you don't care much about performance (and if you are reimplementing a neural network from scratch you're probably doing it more as a learning project than anything else) implementing matrix operations isn't very difficult.
If you're not used to work with matrices simply reading the Wikipedia article might tell you enough to implement them yourself.
If you have an assembler or C compiler you can implement matrix multiplication (GEMM) which usually does most of the heavy lifting in your neural net. Now you correctly alluded that it may not be simple to efficiently implement GEMM but if you have a simple architecture without a complex memory hierarchy then using whatever SIMD facilities are available and some standard tricks will get you in ballpark of peak FLOP/s.
Or, just download a fast BLAS from your hardware vendor...
If you're not used to work with matrices simply reading the Wikipedia article might tell you enough to implement them yourself.