I can't speak for TensorFlow, but I've personally found Theano to be the most easily extensible among the other widespread deep/reinforcement/gradient learning libraries (Caffe, Torch, Keras, Chainer). There does seem to be a performance hit between Theano and Caffe/Torch, but it's great for prototyping.
