Development of a tool to support the quality assurance of bathymetry data against common standards.
The Challenge
Understanding the quality of bathymetric datasets is critical to ensuring it is fit for purpose. These datasets are large and undergo extensive processing through several stages; collection of raw soundings, to point clouds, and then to gridded data products. Each step can introduce quality issues, and identifying these issues in the early processing stages can potentially avoid the need for costly recaptures. The key challenge is to reduce the effort require to perform quality assurance checks so that they can be performed early and often.
Partners
Geoscience Australia supported QAX as a project sponsor, and through participation in the QAX working group. As domain experts. Throughout development two organisations were engaged as product owners to guide key feature development within QAX. CSIRO was an initial product owner which led to the development of functionality spread across checks for all processing levels of bathymetry data; raw data through to gridded products. Later the Australian Hydrographic Office (AHO) as a product owner focussed on improving usability and the implementation of checks specific to its Hydroscheme Industry Partnership (HIPP) Statement of Requirements (SoR).
A QAX working group was established to provide domain expertise, technical input, testing and usability feedback. This working group was comprised of several Australian hydrographic surveying companies including Reach Subsea, EGS Survey, MMA Offshore, Ocean Infinity, and Precision Hydrographic. Where possible QAX utilised code that was developed by members of the working group. A positive unintended outcome of the working group engagement is the role they would play in supporting broader adoption of QAX with members applying QAX to bathymetric surveys outside AusSeabed and HIPP.
The Solution
QAX development started as a collaborative initiative between AusSeabed and the National Oceanic and Atmospheric Administration (NOAA)/ Center for Coastal and Ocean Mapping (CCOM). This was critical in bootstrapping QAX development given NOAA/CCOM’s previous experience developing QA tools for bathymetry data. Following on from this FrontierSI took over the lead development role and in collaboration with product owners and the working group continued to develop QAX in non-contiguous phases over 5 years.
QAX was build as a desktop application, this was an early design decision to support its use offshore where internet connectivity would be limited. A plugin architecture was implemented with a simple interface that defines what input file and parameters are required by the check, this enabled developers to easily implement new checks without the need to develop custom user interfaces. Several plugins were developed for QAX, each implementing a logical grouping of checks. The Mate plugin implemented a total of 13 checks for raw bathymetry data with support for a range of input file formats including Generic Sensor Format (GSF), and Kongsberg files (.all, .kmall). MBES Grid Checks was added as a plugin to support checks on gridded data products, specifically to ensure density, resolution and total vertical uncertainty requirements were met. The Finder Grid Checks plugin built on the gridded data checks to support identification of holes and fliers within data. Lastly, the GGOutlier plugin was developed in collaboration with an organisation from the working group. GGOutlier is tool that identifies outliers in bathymetry data, as it was release under an open-source license it could be integrated into QAX via a plugin.
The QAX technology stack used Python with Qt (PySide2) for the user interface. Checks were implemented using common open-source geospatial libraries such as GDAL, PDAL, NumPy, and SciPy. Additionally, several specialised open-source bathymetry libraries were integrated, with FrontierSI collaborating with their developers to ensure integration into QAX (and other applications). A GitHub Actions-based continuous integration (CI) workflow was established, packaging QAX into a Windows installer using PyInstaller. This CI workflow significantly reduced the effort required to publish new releases, leading to more frequent updates and timely user feedback.
Impact
At the completion of the FrontierSI-AusSeabed project QAX and the working group were handed over to Geoscience Australia to provide ongoing support and maintenance; in recognition of QAX utility. AHO applies QAX in the quality assurance process it performs on data obtained through HIPP, and several organisations involved in HIPP have also adopted the tool.
QAX has been downloaded over 1000 times and is in active use by many organisations; predominately within Australia, although there has been some international uptake.
AusSeabed is actively scoping future opportunities for QAX.
Contact
To learn more, contact FrontierSI at contact@frontiersi.com.au or connect with Project Manager, Lachlan Hurst, at lhurst@frontiersi.com.au.
