Indeed. There are tons of things that happen in real world lidar. For ex, if object is too close it may not detect. When you are going at high speed, returned light ray may arrive early or late giving bit of error. Also, think about multiple reflections, noise, cone size and so on.
