EMPIR research leads to International Association of Geodesy correction for refractive index formula

Image showing land surveyor equipment on construction of highway
Land surveyor equipment on construction of highway

Developing transfer standards disseminated to the SI metre and novel geodetic devices

The project

EMPIR project  Large-scale dimnsional measurements for geodesy (18SIB01, GeoMetre) is working to develop measurement techniques for geodesy measurements referenced to the International Terrestrial Reference Frame. Technologies used in satellite navigation, automated machine guidance and surveying require highly accurate measurements of distance and position relative to the Earth’s surface. Geodesy, the science of Earth measurement, applies these technologies to predicting earthquakes, monitoring sea-level rises and ice sheet changes.

From natural hazard monitoring to surveying, users will benefit from significantly better distance and position accuracy than currently available, which will contribute to better understanding of environmental change.


International Association of Geodesy officially correct their recommendation

As reported in the previous news story EMPIR project corrects standard formula for group refractive index of air, when working on the theory for their novel instrumentation, project partners revealed a subtle, but significant, error in the standard algorithm for the determination of the group refractive index of air. 

As a result of the work in this EMPIR project, the International Association of Geodesy (IAG) has now officially corrected their recommendation of the formula for this issue.

This is reported in the recent IAG Newsletter, where the IAG recommended to use the correction discovered in the EMPIR project.

Specifically, the newsletter says:

“IAG Resolution 3 of the 1999 Birmingham IUGG General Assembly recommended to compute the group refractive index in air for electronic distance measurement to better than one part per million (ppm) with visible and near infrared waves in the atmosphere using the computer procedure published by Ciddor & Hill in Applied Optics (1999, Vol.38, No.9,1663-1667) and Ciddor in Applied Optics (1996, Vol. 35, No.9, 1566-1573).

However, within the 18SIB01 GeoMetre project of the European Metrology Programme for Innovation and Research (EMPIR) in 2020 Florian Pollinger of PTB, Braunschweig, Germany pointed out that in the annex which provides the algorithm, there is a sign error which cause a deviation in results. The resulting group index of refraction deviates in about the correct order of magnitude from the group index but the deviation has the wrong sign. Explanation is published in: Pollinger, F. (2020) Refractive index of air. 2. Group index: comment Appl. Opt., OSA, 2020, 59, 9771-9772 IAG recommends to use the correction published by Pollinger (2020) when using the Ciddor and Hill Formula according to the IAG Resolution #3 of 1999.”

Project coordinator Florian Pollinger, from PTB, said

‘The Ciddor-Hill formula is the established standard for the compensation of the air refractive index in surveying, geodesy and large-volume metrology. It is used in many applications to derive correct conclusions from many optical measurements, e.g., for monitoring critical structures, for surveying properties and also for positioning micrometre-precise boreholes.

The error in the original paper was very unfortunate. It corrected the result in the expected order of magnitude, but with a wrong sign. In consequence, credible, yet faulty results were produced when used as published. The International Association of Geodesy (IAG) is a leading scientific association in the field of geodesy and surveying. The fact that they now officially recommend our correction shows the significance of this discovery. Their announcement makes sure that the professional surveying community is made aware and can in future correctly apply this important formula for their work.’

This EMPIR project is co-funded by the European Union's Horizon 2020 research and innovation programme and the EMPIR Participating States.

Want to hear more about EURAMET?

Sign up for EURAMET newsletters and other information

Follow us on LinkedIn and Twitter

Select your area of interest
EMPIR project on data security sees early uptake and prestigious journal publication

New test standards to support innovative quantum key distribution-based security solutions more

EMPIR project publishes work on quantum effects in a series of prestigious Nature articles

New measurements and manipulation of single electrons for the “second quantum revolution” more

Increased measurement capabilities for neurodegenerative diseases

EMPIR project develops SI-traceable methods to support earlier diagnosis of neurodegenerative diseases more

EURAMET project on passive radiative cooling technologies collaborates with SPACECOOL in Japan

Developing metrological frameworks to standardise new passive cooling materials and test them under real-world conditions more

EMPIR project releases video summarising metrology on new Magnetic Resonance techniques

A project establishing quantitative MRI analysis has released a short video summarising the most important results achieved more

Page 1 of 218.