We're accustomed to associating the word "ice" with temperatures less than 273 K, the freezing point of water at 1 bar. It takes even lower temperatures for ammonia or methane to solidify. But it's definitely not freezing deep within a planet, the temperatures are thousands of kelvins! However, the pressures are from thousands up to millions of bars, which is plenty enough to keep pretty much anything solid even at those temperatures.
The allotrope of solid water we're familiar with is called ice I_h (Roman numeral "I" followed by subscript "h" for "hexagonal"). It has the interesting and vital property of being less dense than liquid water and thus you cannot get ice I_h by compressing water. But there are many other, more exotic phases of ice, most of which are denser than liquid water and can thus form under extreme pressures. Indeed, in a sense they're not that exotic after all, seeing that almost all the water in the Solar System is in the form of high-pressure ices!
