Data Collection

1. Importance of Data Collection

Collecting accurate data is a foundational part of any top-quality behavior intervention program for patients with autism spectrum disorder (ASD). Data are crucial for a variety of reasons:

1. Data enable a treatment program to be accountable for its effectiveness. A hallmark of applied behavior analysis (ABA) is that it must be effective, and it must be continuously modified and adjusted to maintain maximum effectiveness.
2. Treatment decisions must be made virtually every day, and those decisions must be informed by data. Clinical judgment, experience, and hunches are all relevant, but they cannot supplant data in the decision-making process. Every aspect of behavioral intervention is iterative, which means that it is constantly building on previous treatment and the gains of that treatment. When teaching a skill, we are typically building on prerequisite skills that the patient has learned in the past. Through data, the treatment team can determine whether the patient is actually ready to learn a skill or move to the next phase of learning a skill.
3. Accurate data collection helps maintain consistency across a patient’s program. If a patient’s data are excessively variable, it is often a red flag that their behavior technicians are implementing treatment inconsistently.
4. Data collection is often required to secure and maintain funding. Many funding agencies and state laws require that providers of behavior intervention to patients with ASD keep accurate data throughout the treatment process. If continued progress is not documented on an ongoing basis, many funding providers will try to discontinue funding for treatment. A patient will always be able to make progress in a top-quality behavior intervention program, and these funding issues underscore the need to collect data that demonstrate that progress and alert the BCBA when a treatment plan is falling short.

We take data in order to see objectively whether our interventions are working.

• Without data, do we know for sure whether problem behaviors are decreasing? What if we’re just getting used to them?
• What about skills on acquisition (lessons)? Do we truly know if a skill is increasing?
• How can we tell if a skill is continuing at appropriate levels? Or how can we tell if it’s being used in new situations? (Maintenance and Generalization)

The examples below show two different caregivers telling the same story. The point of these two pictures is to show the importance of data and how crucial data collection is to each patient’s programs, and why we do not rely simply on caregiver or even behavior technician report alone.

Bad Report
Data
Caregiver Report	“Calvin’s aggression has gotten better. I feel like he knows that we’re not going to give in. He definitely doesn’t seem as upset as he used to be.”
Explanation	This is an example of a bad report. We see that the caregiver’s report and the data don’t necessarily coincide. The intervention is having a slight effect, but not to the extent that the caregiver seems to be reporting.

Good Report
Data
Caregiver Report	“Melanie’s aggression has gotten better. I feel like she knows that we’re not going to give in. She definitely doesn’t seem as upset as she used to be.”
Explanation	This is an example of a good report. We see that the caregiver report is more accurate. But we only know the caregiver’s report is accurate because the data show us so.

ABA treatment should be data-driven. BCBAs don’t make decisions based on feeling, or even on caregiver reports. which you have seen can be unreliable. Instead, BCBAs make decisions based on what is actually occurring. Data collection is how you and the BCBA know what is really happening.

What does this mean for you? Well, the data you collect are very important!

Without your data, your BCBA will have nothing on which to base their decisions. Accurate data collection is key; if your data are not accurate, the BCBA won’t truly know what is happening with behavior.

---

2. The Four W's of Data Collection

We have discussed that data are extremely important to ABA, but this leads to additional questions:

• Who should take data?
• What should we take data on?
• Where should data be collected?
• When should data be collected?

2a. Who

Everybody important in the patient’s life can take data! The BCBA will determine details. People who should take data include

• all behavior technicians
• BCBAs
• caregivers (parents, respite workers, grandparents, etc.)
• teachers, speech language pathologists (SLPs), occupational therapists (OTs), etc., as determined by the BCBA

2b. What

Data should be taken on behavior as assessed and determined by the BCBA.

For problem behaviors, some types of data you may take include

• rate per hour
• duration
• percent of occurrence
• whole or partial interval
• ABC data

Regarding target skills on acquisition/lessons, some types of data you may take include

• percent correct
• fequency
• rate per hour

hint

Your BCBA will tell you which behaviors and targets to track and how to track them.

You may also take data on caregiver behavior, technician behavior, and inter observer agreement, or IOA. This is a method used to check the consistency of our data, which will be discussed more in a later part of the training.

2c. Where

Data should be collected anywhere you do therapy.

• at home, including the therapy room, kitchen, or backyard
• in the community, for example, at the store, at a park, at a trampoline place, in church, or in a car
• at school, including the classroom and playground
• during other therapies, like speech, OT, physical therapy

Make sure you take your data collection with you everywhere. It is important to remember that our patients engage in both target skills and problem behaviors regardless of where they are.

hint

Do not forget to bring along your data sheets, iPads for data collection, etc., when you are in the backyard playing, at the park, or at any other location so you do not miss opportunities to collect important information.

2d. When

Data should be collected throughout the sessions. This includes when the patient is on downtime or break as well as during outings. You should be close enough to track your targets correctly.

You should constantly be tracking data unless the BCBA has specifically told you otherwise (e.g., do not collect new data about food and snacking).

In relation to the behavior, collect data as soon after the behavior target as you can. In lessons, data should be taken after each trial, because every second that goes by compromises the accuracy of your data. If you aren’t 100% certain of what the response was, the accuracy is already gone.

When collecting problem behavior data, safety is obviously your number one priority.

EXAMPLE

Don’t let a patient bang their head against a window just so you can mark things perfectly.

In most cases, do what you need to in order to manage the behavior and then mark your data. In some instances, you will be able to track the data as the behavior is occurring.

big idea

Note that sometimes you will need to conduct community data collection very subtly, so do the following:

• Use your iPad (or phone if you have permission from the BCBA).
• Use an inconspicuous type of paper (no patient name or company logo).
• Use a notecard or notebook.
• Use a hand-held clicker to track frequency and rate.

think about it

Why do we need to collect data discreetly?

To conceal the patient’s diagnosis (HIPAA) and to avoid calling attention to the patient, especially when they are interacting with peers. If your BCBA has asked you to be discreet, ask what method of data collection they suggest.

---

3. Accuracy and Consistency

Data should be both accurate and consistent so that we can make the best treatment decisions possible for the patient. As mentioned already, accurate data are very important for BCBA decision-making. Consistency in data collection is very important as well.

All behavior technicians on a team should be consistent with skill acquisition data and behavior management data.

Type of Data	Criteria
Skill acquisition data	What is considered correct? What is not correct? What is considered extraneous or irrelevant behavior?
Behavior management data	What are considered occurrences of the problem behavior? What are non-occurrences of the problem behavior? What is considered one instance or episode of the behavior?

hint

If any of what was discussed previously is not clear to you, talk to your BCBA. Chances are, if it’s not clear for you, then it’s not clear for other team members either.

One method of checking consistency in the data is IOA, or inter observer agreement. All behavior technicians take data on the same behavior at the same time. Data are compared and calculated for accuracy; 80% accuracy is considered good.