EMPIR project corrects standard formula for group refractive index of air

While developing novel devices for geodesy measurements, project consortium found that a correction is needed for an established formula

The project

EMPIR project Large-scale dimensional measurements for geodesy (18SIB01, GeoMetre) is working to develop measurement techniques for geodesy measurements referenced to the International Terrestrial Reference Frame. Technologies used in satellite navigation, automated machine guidance and surveying require highly accurate measurements of distance and position relative to the Earth’s surface. Geodesy, the science of Earth measurement, applies these technologies to predicting earthquakes, monitoring sea-level rises and ice sheet changes.

From natural hazard monitoring to surveying, users will benefit from significantly better distance and position accuracy than currently available, which will contribute to better understanding of environmental change.

Refractive index correction

When working on the theory for their novel instrumentation, project partners revealed a subtle, but significant, error in the standard algorithm for the determination of the group refractive index of air. The group refractive index is the ratio of the vacuum velocity of light to the group velocity in a medium, in this case air. It is needed for correct geometric interpretation of most optical distance measurements performed in air. The standard formula was published in 1999 and has officially been recommended by the International Association of Geodesy (IAG) for the correction of terrestrial measurements since then. Used as published, measurements systematically deviate by several parts-per-million (or millimetres per kilometre measurement range), which is well within the targeted accuracy for the most critical measurements performed today. The IAG has been informed about the correction needed and is considering options for informing the community. The work was also published by Applied Optics a leading peer-reviewed journal for optics which had also published the original formula.

This EMPIR project is co-funded by the European Union's Horizon 2020 research and innovation programme and the EMPIR Participating States.

Want to hear more about EURAMET?
Sign up for EURAMET newsletters and other information
Follow us on LinkedIn and Twitter