“When you develop your opinions on the basis of weak evidence, you will have difficulty interpreting subsequent information that contradicts these opinions, even if this new information is obviously more accurate.”
~Nassim Nicholas Taleb
Every day we rely on numbers, data, and statistics to understand the past, take a snapshot of the present, or to make decisions that will impact the future of our business. However, in doing so we run the risk of mistaking data for objectivity – we assume that data represents the ‘truth’ and is not influenced by human error, emotion, or judgment. In fact, the use of data is a complex, subjective, and human-driven process affected by culture, education, technology, fear, incentives, and other human factors. For example, recall the last time you were presented a report containing data or outcomes contrary to your expectations or goals, or results that reflected poorly on your team’s performance. Chances are at least one of the following questions came to mind:
- Is the data accurate?
- Were there any outliers that skewed the results?
- Are the results statistically significant?
- Is the sample size (number of data points) too small to allow meaningful conclusions to be drawn?
- Has the process changed since the data was collected?
- Do the measure(s) exclude certain groups or populations?
- Are there better (more favorable) studies that precede these results?
In order to minimize second-guessing and uncertainty surrounding the analysis and utilization of data, consider the following Five Keys to creating the high quality data needed for sound decision making:
1. Connect the Dots and Plan to Fail – Document how the data will contribute to the achievement of the company mission, strategic plan, or business objective? What are the possible decisions that could be made based on the range of data results? Are all stakeholders in agreement with the overall plan and potential next steps prior to the data being collected, analyzed, and presented? Projects rarely go according to plan. Identify contingencies for everything from simple human errors to changes in project budget, people, equipment, or time allocated to complete each project task.
2. Seek Quality over Quantity – Budget, resources, and project deadlines often dictate the amount of data that can be collected. The number of data points (quantity) collected is moot unless data integrity (accuracy, repeatability, and validity) is achieved. Sampling and measurement error increase the likelihood of making the wrong decision based on the data collected. List these errors and estimate the risks (probability) of them occurring. Once you’ve collected your data, don’t lose it! Include a date in the filename, list the filepath on all hardcopies, and save all electronic files in a back-up location, including your organization’s network server which is more than likely backed-up regularly.
3. Tell a Good Story – Summarize the results and provide context and meaning to them when communicating to stakeholders. Build trust with your audience by describing how the data was collected and analyzed. Graphs generally reveal trends and outcomes more effectively than tables. Re-state the business case as reports have legs – they can be read by others outside the project team (e.g. the CEO). When assigning meaning to the results, understand the roles and performance incentives of others in the organization. Proactively manage the perceptions of senior management by describing what the results do NOT mean.
4. Decide & Take Action – All change requires a problem, a solution, and the political will. Check in with all stakeholders, including senior leadership, to resolve any lingering questions or concerns. If your decision matrix developed in Key #1 has an option for maintaining the current process or repeating the study, be sure to communicate the key lessons learned and to celebrate the successes with those staff involved.
5. Preserve your Knowledge – Accurate and complete documentation will allow you to re-create and understand the project, its outcomes, and the decisions made when it’s inevitably needed many years later. Document project assumptions, decisions made, meeting agendas & minutes, successes, and lessons learned.
For more information and assistance in using data best practices to improve your business, contact us today!