I've not looked much for optimization. First, what you can do is reduce the number of allocations. Instead of creating a new vector3 for each operation, you can do it in place.
Another way to optimize is to make all the ray calculation on the GPU with WebGL shaders. There are a lot of demos of real time small ray tracers. It works well because it's massively parallel and vector operations are hard-wired. The downside is it's harder to debug and you cannot make long operations or it will crash the graphic driver.
Try doing a stochastic spread of the rays sent through each pixel, with attenuation based upon differences in the surfaces reached by the rays. I was able to create an adaptive raytracer with 4 test rays per pixel and up to 56 when adaptively needed. The speed gain is quite significant, with visual clarity equal to simpler renders requiring much more time.
Another way to optimize is to make all the ray calculation on the GPU with WebGL shaders. There are a lot of demos of real time small ray tracers. It works well because it's massively parallel and vector operations are hard-wired. The downside is it's harder to debug and you cannot make long operations or it will crash the graphic driver.