Manipulation as in Simulation: Enabling Accurate Geometry Perception in Robots