Measurement: Using observers and the conditions of data collection
Beforehand the Trial committee has defined what the observers are going to observe and how. This was based on the performance measures that were defined.
The observation questions are incorporated in the Online Observer Support Tool. The tool provides Trial-specific pre-made forms (templates) to create observations.
Peter decides to assess communication in two sessions.
He wants participants to experience performance with and without the solution, and he wants to assess the effect of the solution on crisis management performance. Therefore, he will organize:
- one session where participants use the new solution (a Common Operational Picture Tool),
- one session where people work with their own tools and working procedures. The differences between these sessions indicate the positive or negative effects of using the solution.
When participants already know the scenario in advance of the second session, an improvement may not be the result of using the solution, but rather the result of already knowing what will be communicated and what is to be decided. Thus, the scenario will be slightly changed for the second session.
Peter will use the same measurement for both sessions to be able to compare them. He uses self-report measures (e.g. questionnaire, focus group session) and observational methods (using observers) to gain information from different perspectives.
To assess communication in the two sessions, Peter has defined what characteristics of communication he wants to assess in the Trial. He considered duration (is it faster?), the topics shared (it is relevant and complete?), as well as the number misunderstandings and errors (is it accurate?).
Peter searches the DRIVER + Knowledge base to find existing observation protocols that could be used or adjusted for his Trial.