15.3 Measurement in single-systems design research

WRITTEN BY TRMC:

The first step of measuring outcomes is identifying the target problem and operationalizing it in terms of indicators. The operational definition might consist of positive indicators that show the target problem’s absence and/or negative indicators that show the target problem’s presence. The goal is to increase the levels of positive indicators and decrease the levels of negative indicators.

There is no strict rule on how many operational outcomes can be measured. The researcher may focus on one indicator, multiple indicators to study one  target problem/phenomenon, or multiple indicators for multiple target problems. However, it is recommended that researchers study multiple indicators for one target problem at a time. This is to ensure that we record client’s improvement on multiple indicators. The indicators must be frequently occurring so that it can be measured on a regular basis.

Example: Lyla is being bullied by her teammates during her swim-training sessions, which is making her scared to attend sessions in the water and her performance is deteriorating. Her coach/social worker plan multiple interventions consisting of group bonding sessions, punitive measures for bullies, and therapy for Lyla to improve Lyla’s overall performance. To measure the impact of the interventions, the coach/social worker decide to measure indicators for improvement. They decide to measure Lyla’s participation in group activities, her attendance and changes in her performance in the pool. Here, we can see that these indicators can be a result of any of the 3 interventions, and we cannot assess if changes in Lyla’s behavior, performance in the pool or her attendance is occurring due to any one particular intervention. Therefore, it would be advisable to study indicators with respect to one intervention at a time. As an example, if we only consider implementing group bonding sessions as an intervention and assess the 4 indicators, they would be better reflective of the intervention.

However, if we only decide to measure only one indicator to assess the effectiveness of the sessions, it might have some disadvantages. As for example, if we only consider Lyla’s performance, we might not see any immediate improvement, which might have been caused by Lyla’s prolonged absence in the pool prior to the intervention. However, to the researcher, it will erroneously indicate that the sessions are not effective. In the same way, all the indicators have their own advantages and disadvantages. It is therefore advisable to consider more than one indicator to measure one variable, in order to maximize the chance of the hypothesized variations in Lyla’s overall performance to be detected. This is known as triangulation.

Note: The term triangulation does not always imply that 3 indicators have to be measured. Researchers can use more than one indicator and it can be called triangulation. This method can be used in any type of research.

Include section on who measures, obtrusive and inobtrusive observation with advantages and disadvantages.