Maybe if you could start by persuading a small black hole to go where you want and following in its wake (or lack of wake actually). The black hole swallows everything in your path and grows more powerful. Perhaps if you had an array of three or more black holes you could steer the group by differentially feeding them somehow. If the initial black hole were an orbiting pair your spaceship could orbit a lagrange point and be swept along.
Black holes aren't immune to the effects of gravity from other objects just because they're black holes. It'd be no different than trying to use a trio of stars as your gravity engine (well, more space efficient). I.e. you still have to accelerate the stars, you still have to deal with slower things the stars "eat" slowing them down, if the stars pull your ship it's slowing them down. Effectively, you've just increased the mass of your spaceship by however much mass the black holes have and then still had to find something to accelerate the whole lot instead of your spaceship.
Humans are humble. We only ride our star in about 200 km/s so we can take the scenery route. It takes mere 100 million earth years to get to the other side of our galaxy.
It's not very clear where our black hole in the center of the milky way is taking us though.
