TL;DR — At Clarity, our mission is to empower the world to reduce air pollution. A significant part of that mission is ensuring our customers trust the integrity of their air quality data, so they can confidently use it to make informed decisions and take action to improve air quality. As part of our ongoing commitment to the quality of the data provided by our air quality measurement equipment, we are thrilled to announce the release of automated quality control (QC) features on the Clarity Dashboard. This enhancement marks a significant milestone in our ability to provide cost-effective, accurate, and trustworthy air quality data to our customers globally. 

The importance of accurate and trustworthy air quality measurements

In recent years, there has been a proliferation of lower-cost air quality sensors, making air quality monitoring accessible to a broader audience. While this democratization of air quality data is generally positive, it also brings challenges. Indicative and lower-cost sensors can vary widely in accuracy and reliability, potentially leading to misinformation if not properly managed. 

Clarity’s flagship Node-S measures particulate matter (PM) and nitrogen dioxide (NO₂) with high accuracy. The air quality sensor is solar-powered, FCC/CE certified, UV-resistant, weatherproof, and offers >99% uptime with solar power even in low light conditions.

Clarity’s outstanding collocation results, third-party evaluations, and MCERTS certification demonstrate that we provide some of the most accurate and reliable air quality measurements available from indicative equipment. However, we recognize that not every measurement from our devices will be valid. This is true for all air quality measurement equipment, including Federal Reference Monitors.

Whether due to extreme environmental conditions at the project site, the limitations of the sensing principles used by our devices, or the occurrence of known sensor failure modes, there are bound to be measurements that require closer inspection to confirm validity. 

To address this, we have now implemented automated QC features into our data processing operations and software offerings, including the Clarity Dashboard and OpenMap. This new functionality helps users identify questionable air quality measurements in real time and exclude them from analyses and public data sharing, further distinguishing our air quality monitoring solution from less rigorous alternatives.

Ensuring high-quality data with automated quality control

Our software team has been hard at work redesigning the backend of our air quality data platform to rebuild it with quality control at the heart of our data management process.  

With our new automated quality control, QC flags are applied to measured values when specific conditions arise, and an overall QC assessment is computed based on the severity of each flag. The QC assessment, combined with the Calibration Status, helps users identify and exclude untrustworthy measurements from analyses and public data sharing. This process is simplified by the new Data Cleanup toggle in the Dashboard, which highlights QC invalid values and values missing calibration, providing an easy way to filter them out of view. Similar functionality will soon be released soon through our API and CSV data download tool.

QC flags and QC assessment

QC flags play a crucial role in identifying measurements that may not be trustworthy. Developed based on our in-depth understanding of air pollution, sensing principles, and our sensors' failure modes, these flags are automatically applied to air quality measurements that meet specific conditions by our data processing backend. 

Our backend QC assessment process then reviews all QC flags applied to a measurement to determine if it is valid or invalid. If multiple flags are present, the most severe one decides the final assessment. Additionally, this process ensures that calibrated measurements are accurately categorized by checking the validity of all their inputs and that aggregated measurements are correctly categorized by evaluating all the values being averaged.

Click here for more details on how QC flags and QC assessment work.

Calibration Status

Some metrics measured by Clarity devices require cloud calibration to ensure accuracy. Calibration status is an essential attribute that helps users understand whether the calibration process has been successfully applied to the measured data. The possible values for calibration status can be found here. Calibration Status is used together with QC assessment to determine what measurements are recommended for use in analyses and public data sharing.

Viewing QC assessment and calibration status on the Dashboard 

The new symbols you now see on the Clarity Dashboard are the front-end manifestation of our behind-the-scenes work to provide the highest quality data possible. Air quality measurements are clearly marked for easy identification when they have QC invalid values (red diamond) or values missing calibration (empty blue circle).

The Clarity Dashboard now automatically flags air quality measurements that do not pass our QC assessment with a red diamond, allowing you to further investigate the measurement and determine its validity. 
When you click on the red diamond, the Dashboard provides additional detail on the reason why that air quality measurement was flagged. 
Uncalibrated air quality measurements are also marked on the Clarity Dashboard, with an empty blue circle. We generally do not recommend using uncalibrated data for public data sharing or analysis (except in certain research-oriented applications where raw data are valuable).

‍

Data Cleanup Toggle on the Dashboard

To enhance the usability of QC assessment and Calibration Status, we have introduced a Data Cleanup Toggle on the Dashboard. This feature allows users to filter data, displaying only calibrated, valid values. 

The Data Cleanup Toggle has two modes: Off and On. 

  • Data Cleanup Off: QC invalid values and values missing calibration are included in visualizations, and highlighted appropriately.
When the Data Cleanup Toggle is in the "off" position, all air quality measurements are displayed, but any data points that have been flagged due to QC concerns or lack of calibration will be indicated with a symbol.
  • Data Cleanup On: QC invalid values and values missing calibration are filtered out from all visualizations, creating gaps where invalid data would have been displayed.
When the Data Cleanup Toggle is in the "on" position, measurements that have been flagged due to QC concerns or lack of calibration will be removed from view.

The Data Cleanup Toggle gives you a way to easily identify or clear out any untrustworthy air quality measurements, and applies across various widgets on the Dashboard, including time series charts, scatter plots, tables, and maps.  The cleanup is also applied to the air quality displayed on OpenMap, which receives only valid and calibrated data.

The Clarity advantage

Incorporating automated QC functionality is another step toward Clarity’s goal of providing a robust air quality measurement solution that goes beyond the simple collection of air quality data. These QC tools will help ensure you receive data you can trust — setting our solution apart in a market increasingly filled with lower-cost, less reliable sensors.

We believe these enhancements will significantly improve your ability to trust and utilize the air quality data we provide, reinforcing our commitment to data quality and transparency.

Stay tuned for more updates as we continue to innovate and improve our platform, ensuring that Clarity remains the leader in air quality monitoring and Sensing-as-a-Service solutions — please do not hesitate to contact our team for any questions or further information. Thank you for your continued trust in Clarity.