Agreed. The overal design direction (have those glyphs "#" as it resembles thorns for a Game of Thorns inside a tower filled with brambles) won over usability in certain cases.
If this was a full-blown game (and not a 7-day proof of concept), I would try to make the combat UI more robust.
Huh. I didn't see any relationship between the "#" characters and the game theme. The combat mini game stuck out to me anyway, as an abstract puzzle game in the middle of a hack-and-slash adventure game.
Still, if you are set on using #, you could try varying size, boldness, italic, rotation, font, etc.
If this was a full-blown game (and not a 7-day proof of concept), I would try to make the combat UI more robust.