Online College Courses for Credit

+
Data Collection

Data Collection

Rating:
Rating
(0)
Author: Capella Partnered with CARD
Description:

This lesson covers:
BCAT E-6: Interobserver agreement (IOA)
RBT A-1: Prepare for data collection.

(more)
See More
Fast, Free College Credit

Developing Effective Teams

Let's Ride
*No strings attached. This college course is 100% free and is worth 1 semester credit.

37 Sophia partners guarantee credit transfer.

299 Institutions have accepted or given pre-approval for credit transfer.

* The American Council on Education's College Credit Recommendation Service (ACE Credit®) has evaluated and recommended college credit for 32 of Sophia’s online courses. Many different colleges and universities consider ACE CREDIT recommendations in determining the applicability to their course and degree programs.

Tutorial

what's covered
This lesson will explore data collection by defining and discussing the following:
  1. Importance of Data Collection
  2. The Four W's of Data Collection
    1. Who
    2. What
    3. Where
    4. When
  3. Accuracy and Consistency

1. Importance of Data Collection

Collecting accurate data is a foundational part of any top-quality behavior intervention program for patients with autism spectrum disorder (ASD). Data are crucial for a variety of reasons:

  1. Data enable a treatment program to be accountable for its effectiveness. A hallmark of applied behavior analysis (ABA) is that it must be effective, and it must be continuously modified and adjusted to maintain maximum effectiveness.
  2. Treatment decisions must be made virtually every day, and those decisions must be informed by data. Clinical judgment, experience, and hunches are all relevant, but they cannot supplant data in the decision-making process. Every aspect of behavioral intervention is iterative, which means that it is constantly building on previous treatment and the gains of that treatment. When teaching a skill, we are typically building on prerequisite skills that the patient has learned in the past. Through data, the treatment team can determine whether the patient is actually ready to learn a skill or move to the next phase of learning a skill.
  3. Accurate data collection helps maintain consistency across a patient’s program. If a patient’s data are excessively variable, it is often a red flag that their behavior technicians are implementing treatment inconsistently.
  4. Data collection is often required to secure and maintain funding. Many funding agencies and state laws require that providers of behavior intervention to patients with ASD keep accurate data throughout the treatment process. If continued progress is not documented on an ongoing basis, many funding providers will try to discontinue funding for treatment. A patient will always be able to make progress in a top-quality behavior intervention program, and these funding issues underscore the need to collect data that demonstrate that progress and alert the BCBA when a treatment plan is falling short.
We take data in order to see objectively whether our interventions are working.
  • Without data, do we know for sure whether problem behaviors are decreasing? What if we’re just getting used to them?
  • What about skills on acquisition (lessons)? Do we truly know if a skill is increasing?
  • How can we tell if a skill is continuing at appropriate levels? Or how can we tell if it’s being used in new situations? (Maintenance and Generalization)
The examples below show two different caregivers telling the same story. The point of these two pictures is to show the importance of data and how crucial data collection is to each patient’s programs, and why we do not rely simply on caregiver or even behavior technician report alone.

Bad Report
Data Data
Caregiver Report “Calvin’s aggression has gotten better. I feel like he knows that we’re not going to give in. He definitely doesn’t seem as upset as he used to be.”
Explanation This is an example of a bad report. We see that the caregiver’s report and the data don’t necessarily coincide. The intervention is having a slight effect, but not to the extent that the caregiver seems to be reporting.

Good Report
Data Data
Caregiver Report “Melanie’s aggression has gotten better. I feel like she knows that we’re not going to give in. She definitely doesn’t seem as upset as she used to be.”
Explanation This is an example of a good report. We see that the caregiver report is more accurate. But we only know the caregiver’s report is accurate because the data show us so.


ABA treatment should be data-driven. BCBAs don’t make decisions based on feeling, or even on caregiver reports. which you have seen can be unreliable. Instead, BCBAs make decisions based on what is actually occurring. Data collection is how you and the BCBA know what is really happening.

What does this mean for you? Well, the data you collect are very important!

Without your data, your BCBA will have nothing on which to base their decisions. Accurate data collection is key; if your data are not accurate, the BCBA won’t truly know what is happening with behavior.

Video Transcription

Download PDF

You're saying that is like a reoccurring thing. Then we can just teach him the rule. You know, if they don't have what you want, you say, oh, I wish they had milk, but OK. You know, teach him the replacement thing to say, and then we'll take data on [INAUDIBLE] is he able to use the replacement behavior that we've taught.

What did you have for snack yesterday at school? What was there, but you didn't like it? What was there? Do you remember?

Peaches.

Peaches. Peaches are good.

And what else?

Peaches. Are peaches good for me?

I love peaches. I think they're so good, and they're very good for you. They have lots of vitamins.

I don't like them!

That's OK. You can just say, I don't like peaches. So you give a rule, have him practice, and then we'll take data on if he's able to implement the rule.


2. The Four W's of Data Collection

We have discussed that data are extremely important to ABA, but this leads to additional questions:

  • Who should take data?
  • What should we take data on?
  • Where should data be collected?
  • When should data be collected?
2a. Who
Everybody important in the patient’s life can take data! The BCBA will determine details. People who should take data include
  • all behavior technicians
  • BCBAs
  • caregivers (parents, respite workers, grandparents, etc.)
  • teachers, speech language pathologists (SLPs), occupational therapists (OTs), etc., as determined by the BCBA
2b. What
Data should be taken on behavior as assessed and determined by the BCBA.

For problem behaviors, some types of data you may take include

  • rate per hour
  • duration
  • percent of occurrence
  • whole or partial interval
  • ABC data
Regarding target skills on acquisition/lessons, some types of data you may take include
  • percent correct
  • fequency
  • rate per hour
hint
Your BCBA will tell you which behaviors and targets to track and how to track them.

You may also take data on caregiver behavior, technician behavior, and inter observer agreement, or IOA. This is a method used to check the consistency of our data, which will be discussed more in a later part of the training.

2c. Where
Data should be collected anywhere you do therapy.
  • at home, including the therapy room, kitchen, or backyard
  • in the community, for example, at the store, at a park, at a trampoline place, in church, or in a car
  • at school, including the classroom and playground
  • during other therapies, like speech, OT, physical therapy
Make sure you take your data collection with you everywhere. It is important to remember that our patients engage in both target skills and problem behaviors regardless of where they are.

hint
Do not forget to bring along your data sheets, iPads for data collection, etc., when you are in the backyard playing, at the park, or at any other location so you do not miss opportunities to collect important information.

2d. When
Data should be collected throughout the sessions. This includes when the patient is on downtime or break as well as during outings. You should be close enough to track your targets correctly.

You should constantly be tracking data unless the BCBA has specifically told you otherwise (e.g., do not collect new data about food and snacking).

In relation to the behavior, collect data as soon after the behavior target as you can. In lessons, data should be taken after each trial, because every second that goes by compromises the accuracy of your data. If you aren’t 100% certain of what the response was, the accuracy is already gone.

When collecting problem behavior data, safety is obviously your number one priority.

EXAMPLE

Don’t let a patient bang their head against a window just so you can mark things perfectly.

In most cases, do what you need to in order to manage the behavior and then mark your data. In some instances, you will be able to track the data as the behavior is occurring.

big idea
Note that sometimes you will need to conduct community data collection very subtly, so do the following:
  • Use your iPad (or phone if you have permission from the BCBA).
  • Use an inconspicuous type of paper (no patient name or company logo).
  • Use a notecard or notebook.
  • Use a hand-held clicker to track frequency and rate.
think about it
Why do we need to collect data discreetly?

To conceal the patient’s diagnosis (HIPAA) and to avoid calling attention to the patient, especially when they are interacting with peers. If your BCBA has asked you to be discreet, ask what method of data collection they suggest.

3. Accuracy and Consistency

Data should be both accurate and consistent so that we can make the best treatment decisions possible for the patient. As mentioned already, accurate data are very important for BCBA decision-making. Consistency in data collection is very important as well.

All behavior technicians on a team should be consistent with skill acquisition data and behavior management data.

Type of Data Criteria
Skill acquisition data What is considered correct?

What is not correct?

What is considered extraneous or irrelevant behavior?
Behavior management data What are considered occurrences of the problem behavior?

What are non-occurrences of the problem behavior?

What is considered one instance or episode of the behavior?

hint
If any of what was discussed previously is not clear to you, talk to your BCBA. Chances are, if it’s not clear for you, then it’s not clear for other team members either.

One method of checking consistency in the data is IOA, or inter observer agreement. All behavior technicians take data on the same behavior at the same time. Data are compared and calculated for accuracy; 80% accuracy is considered good.

Video Transcription

Download PDF

One way that instructors make sure data is reliable is called inter observer agreement. Inter observer agreement or IOA measures data reliability by calculating the degree by which two people agree on a behavior that they have both observed, measured, and recorded. The data that they collect is then compared to determine the level of similarity. If both individuals have recorded data in an identical or very similar way, then the data is then considered to be reliable.

However, if there is low level of agreement or many differences in the data sets between the two observers, then agreement is low and their data collection results are considered to be unreliable. Inter Observer Agreement, or IOA, is something that should be measured and incorporated regularly into ABA programs in order to ensure that the data being collected by instructors is reliable and that everybody is measuring the same behavior in the same ways. The type of IOA procedure used for any particular client will greatly depend on the type of data collection procedure being used.

Your program supervisor or BCBA will advise you in regards to what type of IOA procedures will be used for your particular client. Let's look at an example of how IOA works. Here we see an individual within a discrete trial training session. In order to determine if data collection is reliable, two different observers would watch the same responses from the individual and record those responses.

At the end of the discrete trial activity, the two data sets would be compared. If there is agreement between the two data sets per trial and that agreement is high, then we say that the data is reliable. However, if there are discrepancies between trials of each of the data sets, then the behavioral data is said to have low agreement. In other words, there are problems with reliability.

This may mean that additional training is required for all of the staff in order to make sure that data collection procedures are being utilized appropriately. It may also mean that adjustments need to be made to the data collection procedures. In any case, IOA is something that should be occurring on a regular basis within any ABA program to ensure that the data being collected is accurate and reliable.

summary
In this lesson, you learned about the importance of data collection, noting that it enables a treatment program to be accountable for its effectiveness, everyday treatment decisions must be informed by data, accurate data collection helps maintain consistency across a patient’s program, and data collection is often required to secure and maintain funding. You explored the four W's of data collection, examining who should take data, what we should take data on, where data should be collected, and when data should be collected. Lastly, you learned about the importance of accuracy and consistency in data collection to ensure we make the best treatment decisions possible for the patient.