Societal Impact Aspects
A key consideration when selecting solutions for a Trial is to assess whether the solution has any known unintended side-effects or societal impacts that Peter should be aware of. When selecting the solution from the PoS, Peter became aware that no such assessment currently existed for the specific solution since the COP-tool that he wanted to use is new, and thus he decided to make an assessment himself. Based on the selected solutions for the selected scenario, he carries out an assessment using the DRIVER+ Societal Impact Assessment Framework (SIA), which allows him to assess how the use of the COP-solution can potentially have a negative or positive impact on the broader society.
The SIA framework is not tool specific, but is developed for assessing the most common functions that CM tools have. This means that Peter could potentially use the same method for assessing all kinds of solutions that he might be considering. The assessment starts with identifying what kind of functions the solution has (e.g. does it collect or process data, or does it facilitate communication?), and then systematically linking the functions to a predefined set of societal impact criteria. In the PoS, Peter can also look up assessments and concluding recommendations that other users of the PoS have made of other solutions or tools. Thus, if solutions with similar functions as the COP tool have been assessed before, Peter can use these as inspiration.
Preparation phase – Selection of solution
Erik, who is also part of the Trial team, told Peter about the solutions available in the online DRIVER + Portfolio of Solutions. Peter decides to search, evaluate and select a solution that is expected to improve the crisis management performance he wants.
Using key words that describe the crisis management tasks, processes and roles he wants to improve, Peter finds all kinds of solutions including experiences of others and lessons learned.
Together with his Trial team he formulates selection criteria and selects solutions that are worth considering:
- Training for communication and decision making.
- Multiple software tools providing a Common Operational Picture (COP).
- Ways to monitor units, to monitor sensor data and predictive models.
They finally decide to select a Common Operational Picture (COP) solution for Trialing that meets their criteria. Peter reads that the COP is an online software tool providing a shared map that multiple command teams can view and use to share information about incident, units or routes.
Preparation phase – Iteration of research questions
Now that the Trial team has selected a solution, it is possible to further specify the research questions, the measurements and Trial design.
The objective of this specific Trial is to assess the effect of the selected COP tool on communication and shared situation awareness between command teams using the COP tool, and its impact on making effective decisions in a simulated but realistic scenario. Research questions are now reformulated as follows:
- How does the COP tool affect communication between onsite and offsite command teams?
- How does the COP tool facilitate shared situation awareness about incident and response to it?
- How does the COP tool affect decision making about the routing of resources?
Other adjustments (iterations) based on the chosen solution and design
Because the COP tool is new, participants should be trained in using the COP tool and should receive instructions about their task.
Because Peter and his team decided to have two sessions with the same teams, they decided not to use the same scenario twice. To avoid a learning effect, Peter decides that the events in scenario will be similar, but not the same between the sessions with and without the solution, respectively. This is because he wants to use the comparison of measures between the sessions to draw conclusions about the effect of the solution.
Participants will not be instructed about the scenario, because in that case they can respond as they would normally do and have no previous knowledge that might influence their performance.
Specific questions about the COP tool are added to the different measurements (observation protocol, the questionnaire and focus groups). Examples of guiding questions for the focus groups:
What advantages/disadvantages did you experience in using the COP?
Can you provide specific examples?