Validating 3D measurement software

Engineering projects – from the world’s tallest skyscrapers to new transport infrastructure – use complex 3D models to assess safety, efficiency and cost effectiveness. Running millions of survey data points through complex computational simulations to create these models introduces opportunity for error. To provide the confidence needed by such safety critical industries, 3D simulation software developers need standardised methods for checking programming accuracy.

Challenge

Computer aided measurements are an essential part of engineering projects. From predicting if building components can access sites, to having confidence in the structural integrity of walls and containers, reliable 3D models are needed for critical decisions on safety, appearance, and investment.

Computer simulations turn millions of survey data points, collected from measuring these structures, into reliable models. These models rely on many different and complex subroutines. Users making decisions based on these models must trust that the programmes correctly interpret and process the input data. Mistakes can be very costly in these industries, so users need confidence that these models are performing correctly.

Developers often send their software for independent testing. Whilst this increases the chances of programing errors being spotted, there is no coherent framework for testing measurement software - developers must rely on ad hoc approaches, with no way of demonstrating that such methods provide correct results.

An independently validated, standardised method to demonstrate performance of modelling software would reduce errors and give confidence to the many engineers and manufacturers who rely on 3D structural models.

Solution

The EMRP project Traceability for computationally-intensive metrology has developed a validation process to prove the correctness of calculations using complex, numerical algorithms implemented in measurement software. 3D measurement software is a typical application that would benefit from this approach.

This project applies the calibration principle used for measurement instruments – using a well understood reference object to calibrate software, with the reference object being replaced by data sets.

The project developed a range of ‘golden data sets’ – standardised data sets and routines that match the required accuracy needed for software conformance testing. The project’s TraCIM system evaluates and compares the results from the software being tested to idealised values, providing a performance certification process to confirm that test software runs properly and does not produce errors. The system is cloud based and so can be used by the software developer on their own system, making it quicker than previous third party approaches, as well as more reliable.

Impact

Technodigit, part of Hexagon AB, specialises in point cloud processing for various applications such as surveying, digital terrain modelling and civil engineering. The company has validated a new software model as a result of the project, creating a more efficient way to check the integrity of fuel or oil storage tanks at ports and airports.

Technodigit’s 3DReshaper is used to ensure site safety through fuel tank storage integrity checking. This process currently requires two days of manual measurements and data manipulation. Recognising the frustrations in this process, Technodigit developed a new software module which can automatically generate the test report in two hours with minimal human intervention.

Demonstrating that the data points have been correctly analysed has been difficult, undermining confidence in a product, which customers are otherwise keen to embrace. With the TraCIM system, Technodigit has been able to check its software algorithms and receive a test report confirming these are fit for purpose.

This gives 3DReshaper user’s confidence in the results and that the inspection has been appropriately carried out, speeding take-up of the software. The TraCIM system is also being used to continually assess its own in-house testing systems and software components, making it a key part of Technodigit’s new product development and ensuring confidence from day one.

Image showing computer program code on a computer screen

Novel mathematical & statistical approaches to uncertainty evaluation

Developers of software implementing complex calculations increasingly require independent testing to demonstrate that results are accurate.

The EMRP project Traceability for computationally-intensive metrology identified calculations for a number of priority metrology applications and developed approaches and associated test data to assess the performance of software implementing these calculations.

A system, TraCIM, was developed that allows mathematical software to be tested using the internet.

Software developers now have a recognised route for independent product verification reducing risks associated with errors, while customers benefit from increased confidence in the software’s ability to produce reliable results.

  • Category
  • EMRP,
  • New Technologies,
Element 1 Element 2 Element 3 Element 1 Element 1 Element 1 Element 1 Element 1 Logo-Footer