I do not believe that this is how refraction works. It would imply that light of frequencies other than those corresponding to the bandgap energies of the material would proceed unrefracted, or be absorbed. Furthermore, it would render coherent light incoherent, since nanoseconds are very large compared to the frequencies involved in visible light, and this incoherence would result in the light being re-emitted in every direction, not a particular refracted direction. In short, although the phenomenon you are describing is real, it is very different from refraction. It is phosphorescence and fluorescence.
Refraction works because the permittivity of the transparent material is different from the permittivity of vacuum. See http://en.wikipedia.org/wiki/Permittivity.