Quality Control in Digital Photogrammetric Mapping

As we know, quality control in photogrammetric mapping substantially influences the accuracy of the final product. A proper workflow, with a focus on quality, is necessary in any photogrammetric map production process. A detailed procedure for quality control and consistent quality improvement must be inherent throughout the production process in any mapping organization. This article discusses some of the experiences and issues involved in achieving quality objectives, and breaks down the details of some of the production processes.

Specific data quality objectives, standard quality control and data validation procedures must be in place prior to initiating any production activities. In addition, consistent monitoring procedures and quality assurance protocols are needed.

Quality Control Systems
The main objective of a quality control system (QCS) is to obtain a continuous production chain with a goal of "zero errors" and/or "first time right." Gaining an understanding of a client's requirements in this area is crucial. Important components of a QCS are:

Quality control for every activity, such as aerial triangulation, DTM creation/editing, orthophoto generation, feature extraction, and/or data editing
Identifying errors in the production process
Fault and process registering
Corrective measures when error is detected

At the beginning of a project, some product samples should be tested to confirm that they are in agreement with the pre-established specifications.

Two steps are included in our photogrammetry QCS process: defining procedures and quality analysis, both of which are further described below.

Defining Procedures
The key parameters to be considered are as follows:

Technical specifications are translated to the production staff in the form of spreadsheets, graphically indexed with workflow diagrams illustrating the instructions.
The procedures pertaining to technical specifications and production processes are documented to achieve satisfactory quality. This will help make the process less susceptible to unexpected problems that can occur during the production process.
A product prototype should be created and confirmed as being adequate for the planned use.
The procedure documentation describes the steps and efforts carried out to obtain the desired quality.

A well-defined map production procedure should be developed for the digital terrain model (DTM) and vectorization process. Some steps might include the following:

Personnel training
Creating one test model for a production and quality check
Introducing an embedded quality checking system
Checking for silly errors , such as missing buildings or fences, etc.

The project leader should conduct a series of random checks to evaluate the production accuracy.

An important aspect of quality control is getting team members to "own" the responsibility for quality, and quality loops. At the team level, individual teams "own" the plotting and editing processes.

The responsibility for the quality and timely delivery of any assigned project rests within the specific team only. At the individual level, individual plotting and editing operators own the sheets and part-sheets assigned to them, and are solely responsible for producing the sheets correctly with the desired quality, and on time. To close the "quality loop," significant errors found further in the process are referred to the operator who introduced the errors for their correction.

Quality Analysis
Each individual operator should conduct a self-inspection. This is a sound practice to identify and eliminate errors - the earlier an error is caught, the fewer resources are required to rectify it. Operators should carry out inspections of their own work upon completion of a half-model/sheet and again, after completion of the whole model.

To ensure that quality is maintained when a full- or half-sheet passes from plotting to editing, and from editing to delivery, an independent quality inspection should be performed by a dedicated quality controller. This spot check should take about 20-30 minutes. As part of the "quality loop," sheets with errors are returned to the operator for the required corrections.

Permissible limits are established for planimetric and altimetry accuracy and also to avoid the occurrence of "overshoots" or "undershoots." Snapping errors should not be allowed, and tools should alert the operator to such errors. The basic rules should consider parallel lines (road edges, canals, etc.), line perpendicularity (building edges, etc.), non-duplicate lines and symbols, and automatic polygon closing.

Here are some examples of how to proceed with data validation, with varying degrees of automation.

Full manual quality control: The whole model/worksheet is checked thoroughly so that no errors are left in the data. (Total concentration is required when doing the quality check!)
Semi-automatic quality control: A few customized tools are used for certain purposes - errors are identified by a circle, but the rest of the editing work is done manually.
Automatic quality control: To minimize time duration, technical errors in the data can be found using automated tools. Consider the following important aspects when using automated tools:

Linear elements and certain area classes are not interrupted.

Elements are not duplicated.

Elements that should be connected are connected and those that should not be connected are not.

Every building has a closed polygon.

Automatic symbology has been given to different objects.

Reporting Quality Control Results
The quality manager, the person in charge of quality, will ensure that error files are properly communicated to the person who introduced the error, and that corrections are carried out within the scheduled time period. We find it very important that the people working on a project are fully responsible for errors that might have a negative impact on production quality.

Gate Control and Quality Assurance
The quality manager, who is not directly a part of the production organization, carries out "gate control" as a precautionary measure to ensure that no deliveries are made to the client that do not meet the quality standard. We estimate that gate control comprises approximately 10% of the production time, and we build that into the project estimate. As part of the check, the quality manager does a spot check, looking for missing features like masts, manholes and fences; he checks for polygon closure, and generally keeps an eye out for anomalies. Significant errors, reported internally, are documented in a failure report.

When errors are found, it's important to review the procedures involved for possible improvements. It's also important to close the loop by letting the operator know and having that person fix it. Despite using best practices to avoid delivering a product with an error, it does happen. If a product with an error gets out to the client and the client reports the error, it's important to document that as well. A failure report in this case might or might not include specific errors. Generally, it does result in rework and redelivery. We always try to reply within three working days and have corrective actions start as soon as possible.

At the end of the project, we generate a report attesting that the quality standards were achieved. It includes details about errors corrected, date when corrections were made, etc. We also request that the client provide a certificate stating that quality standards were achieved.

The main benefits of applying a QCS are quality improvement, increased productivity and decreased cost. By applying quality control procedures, any organization can assure a very high level of acceptance of its product and reduced risk of rework.

Reference

Sato, S.S. and I. Da Silva. "Brazilian Quality Control Systems for digital Photogrammetry Mapping production," in The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Science, Vol 34, Part XXX.