It is difficult to measure the success of counter-terrorism efforts.In multiple reports to Congress, senior officials have emphasized the number of arrests and disruptions as a way of indicating success.But that is just one component of a measurement system necessary to properly evaluate the effectiveness, efficiency and results of the Department of Homeland Security's (DHS) efforts.
Measuring success in the war on terrorism requires an understanding of the terrorist's operational environment.Many people have used body-count and kill ratio, two traditional military measurements to gauge the war on terrorism.Others have used the number of terrorism attacks as a measure of the success against terrorism.However, these measures may play a role but do not clearly cover the scope of DHS.
Example: The number of significant terrorist attacks in the United States of America, or on Americans worldwide for 2003 was 208. In 2004 the number went up to 651.In and of itself, this is an important fact but it does not represent the entire picture.
James Jay Carafano, Ph.D., a senior research fellow for defense and homeland security at the conservative think tank, The Heritage Foundation, came up with the following list of measures for the war on terrorism.
- Number of Taliban-style states created since 9/11: 0
- Number of countries that have recognized al Qaeda: 0
- Number of nations that have adopted "state-sponsored" terrorism as an official policy: 0
- Number of states that have voluntarily given up weapons of mass destruction programs since 9/11: 1
- Number of transnational nuclear smuggling networks broken-up since 9/11: 1
- Number of Middle Eastern states that have moved closer to democracy: 5
The question is: How do you measure an organization like DHS? The answer is, you start at the process level.The core processes used to achieve the organization's stated mission became the foundation for a management and measures system.A core process is a collection of participants, activities, events and measures that complete a function which adds value to the enterprise.
Key process attributes:
- Measures - mechanisms that quantify critical performance.There are multiple types of measures:
- Process measures - assess where you are within the process
- Results measures - measures the outcome of a process
- Quality measures - measures errors
- Metrics - measurement
units that quantify
volumes handled within a
process, and has to have the word "per" in the
Before jumping in and developing all sorts of metrics, the leadership team must define the overall approach.Once the groundwork is set, the next step is to identify the right metrics/measures to track the organization's progress toward its organizational goals.In order to design a proper measurement system we first must examine the vision, mission and strategic goals set by the organization.The following represent those areas for DHS.
VisionFor simplicity let's just look at the two primary areas for DHS - prevention and protection.These two areas equate to approximately 76% of DHS activities and budget for FY 2004.Careful consideration must be given to the design of a metrics and measures system.Improper or imbalanced measures can often create bad behaviors and gaming of the system.
Preserving our freedoms, protecting the US ...we secure our homeland.
We will lead the unified national effort to secure the US.We will prevent and deter terrorist attacks and protect against and respond to threats and hazards to the nation.We will ensure safe and secure borders, welcome lawful immigrants and visitors and promote the free-flow of commerce.
Awareness - identify and understand threats, assess vulnerabilities, determine potential impacts and disseminate timely information to our homeland security partners and the American public.
For example, for the DHS, if you were to measure the number of persons arrested without an opposing measure like the number of persons whose arrests were upheld and remanded for trial, or those convicted - you could create a situation where arrests were being made without clear, valid and credible reason to improve the appearance of success.
The ideal is to implement a "balanced performance scorecard" throughout the enterprise, because that framework helps foster alignment between all the individual operational components.The scorecard structure contains three major categories - effectiveness, efficiency and results.Careful attention has been given so as not to have an imbalance that promotes bad behaviors or gaming of the system.
Once the metrics and measures are identified and collected, the analytics process begins.Key performance ratios along with balancing formulas and trending data are all reviewed and the results prepared for management's review.The ways in which data is collected and analyzed must be documented so that the context of the information is understood by decision makers.Detailed analysis of these performance metrics allows management to fine-tune operations, efficiently apply dollars and align a disparate organization around common goals.The final report must include a summary of current conditions and background, followed by the conclusions from the analysis as well as the balanced performance scorecard graphic representation.
Balanced performance scorecard is a strategic management process that must be used to assess the effectiveness, efficiencies and results of any organization.There are numerous tools that have been created to help organizations automate the process of creating, maintaining, and reporting scorecard results.
You get what you measure! If the metrics and measures scorecard is properly constructed, it can drive integration and collaboration across all the individual entities now operating under the umbrella of the DHS.However, many organizations, particularly government entities, tend to shy away from their performance being measured.Implementing a balanced performance scorecard within DHS will be significant challenge.The fight against terrorism and homeland security is so important maybe it is time for this to be required.