I don’t know about you, but one of the things that drives me nuts about data is not the data itself, but how we manage it. It would be fine if we were all making and using only our own data, but GISers regularly create data that will be used by others, intentionally or not. We live in a world of shared data, making decisions based on data that someone else likely built for an entirely different purpose.
I made the case in my previous article that data governance is important to our work because it “means the ability to create spatial data of high quality that are standardized, well-understood, well-utilized, and well-governed.” My focus there was to make the case for data governance for our own organizational data, but as I mentioned in the conclusion, it is also important, and perhaps even more so, to have a good data governance plan to ensure success for everyone else.
Metadata
Love it or hate it, metadata should be priority number one for any organization creating and sharing information. It should also be priority number one in our data governance plan because we’re not going to be at our organizations forever. I recently read that the average tenure for a GIS technician is 1-2 years! We must prioritize this part of the process because we’re depending on a technician to create these data, and if they’re likely to move on in a few years, we’d better make sure that we have all the documentation we need before they go!
Metadata are crucial in any situation where data will be reused, whether these data are public, or private and internal to the organization. No one can responsibly re-use or interpret data without metadata. Metadata tell us about a dataset: how it was formed, why it was created, and what the limitations of the dataset are. As our datasets get bigger and more robust, so should our metadata.
Metadata are often given short shrift and treated as low priority in the data development workflow because creating and maintaining them are difficult. There is always something that seems more pressing, more interesting, or more worthy of resources. There is always another map to create, another model to generate, another analysis to be run. Let’s do better: make more metadata!
Where is it? Build a Robust Data Inventory.
One of the questions that I regularly ask myself at my day job is: How do we know what we have? In my organization, we have an enterprise data inventory project that spans across all parts of the organization. Each agency is responsible for their own individual inventory, and the results of all inventories are published together in our public data catalog.
While accountability and transparency are two of the major reasons this project was implemented, another valuable outcome of this, and any inventory process, is understanding what we actually have and where it can be found.
For any organization, public or private, as our data stores increase, a data inventory becomes extremely useful and necessary for keeping track of all of its data asset types, locations and use cases. Quite frankly, “where can I find…” and “who has…” are questions that I’ve been asked about a million times over my career. I can guarantee that this is a regular question in most organizations. Let’s let the analysts focus on their work and help program staff answer their own questions. A data inventory, even a nascent one, can increase operational efficiency of an organization almost immediately by creating a self-service opportunity for program staff. If we reduce the time an analyst has to spend on finding and explaining the basics, we allow analysts to do the analyses they were hired to do . As the man said: help me help you… create efficiency.
Do You Trust Your Data?
We all want to make good decisions. We can only make good decisions if they are informed decisions. Part of making an informed decision is knowing whether you can trust the data you are using. Are these data of good quality? What does quality mean? How do I know?
It’s not enough to say that your dataset is good because Joan Smith created it and you know that she is a good technician. Subjective assessments aren’t enough. Part of a good data governance plan is understanding the quality of the information using objective metrics and reports in conjunction with metadata. You might have 13 metrics or 50; regardless, metrics are generally addressed in the following six dimensions:
- Accuracy: How many errors? What kind of errors?
- Completeness: Are all the critical fields fully populated?
- Consistency: Am I seeing conflicting data values across datasets?
- Timeliness: When was the last time my dataset was updated?
- Uniqueness: Are there duplicates in (or across) my dataset?
- Validity: Do these data conform to an established standard?
“The aggregated scores of multiple dimensions represent the measure of data quality in your specific context and indicate the fitness of these data for use.” Unless you can quantify the quality of your data, you can’t know for sure that it’s safe to use it for prime time.
As I said earlier, we live in a world of shared data, making decisions based on data that someone else likely built for an entirely different purpose. A data governance plan ensures that we know what our data are about, where they come from, and what the context is around a dataset’s purpose. When we have metadata, a data inventory, and objective assessments about our data quality and trustworthiness, we can better support our staff and our decision-making processes. We make good decisions with our eyes wide open, and we can share our data with confidence.
A book and blog about where to find geospatial data, how to know if the data is any GOOD, and societal issues surrounding data such as location privacy, copyright, ethics, and AI: https://spatialreserves.wordpress.com/