By Elizabeth Ahmann, ScD, RN, ACC, of the ACO Research Committee
A number of studies conducted over the past twelve years have reported beneficial outcomes of ADHD coaching for students. These various studies illustrate a number of different research designs, four of which will be described here: a case series, a design using a comparison/control group, a pretest-posttest design, and a randomized controlled study.
The following studies on coaching for students with ADHD are included in the research design discussion below:
- In 2001, Zwart and Kallemeyn demonstrated that students engaged in peer “coaching” experienced increased motivation and reduced levels of stress as compared to peers not receiving coaching.
- Parker and Boutelle (2009) studied coaching for college students with ADD and LD, finding that coaching can support a student’s emerging autonomy, help them with self-regulation, and promote their confidence about future success.
- Maitland and colleagues (2010) reported results of a small pilot study examining coaching for college students, finding positive effects.
- In 2011, Parker, Hoffman, Sawilowsky and Rolands published a study suggesting that students find coaching both supportive and effective and that it led to improved goal attainment skills as well as an improved sense of well being and improved self-regulation.
- Field, Parker, Sawilowsky, and Rolands, (2013) found that, in comparison to a group not receiving coaching, students with ADHD receiving coaching services had significant improvements in learning and study skills, self-regulation as well as a sense of well-being and positive emotional states.
- A 2013 study by Parker, Hoffman, Sawilowsky and Rolands suggests that ADHD coaching supports substantial gains for college students with ADHD in becoming more self-regulated, which leads to positive academic experiences and outcomes (Journal of Attention Disorders, 17(3), 215-32).
- Prevatt and Yelland (2013) reported that an 8-week coaching program for college students led to significant improvement in ten areas of study and learning strategies, on self-esteem, and on measures of symptom distress and satisfaction with school and work.
Case Report/Case Series
A case report is an article that describes an individual and his or her experiences of coaching. A case series is a similar approach that describes the experiences of a number of individuals, often between 10 and 30. Case reports and case series offer a type of anecdotal evidence. They often consist of interviews but can also include questionnaires or other measures to gather information about a situation. For example, researchers might interview a number of students who have had ADHD coaching to see what they found useful about the experience. These reports, even if valid and reliable measures are used, are not considered scientifically rigorous and cannot be used as evidence that an intervention causes a particular outcome. But, because case reports and case series offer descriptive information about experiences, they can play an important role in developing an understanding of an issue. In this way, they can be used to identify topics, questions, and hypotheses that may deserve more rigorous scientific exploration.
Parker and Boutelle’s (2009) report is one example of a case series. Students who had already been receiving coaching services were interviewed to determine “why these students choose ADD (“executive function”) coaching to enhance their academic success when more traditional forms of campus support already offer this help” (Parker & Boutelle, p. 204). Parker, Hoffman, Sawilowsky, and Rolands (2013) similarly, interviewed 19 undergraduate students who had received ADHD coaching to examine their “perceptions of the impact of ADHD coaching on their academic success and broader life functioning” in a case-series type of design (Parker, et al., p. 215).
A second study design makes use of a comparison or control group as a way to examine the impact of an intervention. Study participants are assigned to two separate groups: one group receives the coaching intervention while a comparable group of subjects does not. Variables of interest are then measured and compared between the two groups to examine efficacy of the intervention. Careful choice of an appropriate comparison group is of key importance. Validity of the study results depends in part on how closely matched the case and comparison groups are on characteristics that might impact the outcomes studied.
Because individuals in a comparison group will undoubtedly have pre-existing characteristics that make them different from individuals in the case group, these differences can confound interpretation of the results. For example, let’s imagine that a researcher offers coaching to 20 students with ADHD identified through a college disability office. For a comparison group, 20 students of the same ages are selected from the student body. The researcher then examines self-efficacy in the students who did and did not receive coaching. The researcher finds no difference in self-efficacy between the two groups. Is this because coaching was not effective in boosting self-efficacy? Or could it perhaps be because the students without ADHD have higher self-efficacy than students with ADHD to begin with, and the coaching effectively improved self-efficacy for students with ADHD so that it matched the higher level of the students without ADHD? A better research design would have been to select a comparison group of students with ADHD. Even so, other possible differences between the two groups (degree of impact of ADHD, co-existing anxiety or depression, co-existing learning disabilities, previous coaching experience, and so forth) could still impact validity.
Zwart and Kallemeyn’s (2001) study of peer coaching is one example that used a comparison group. In their study, thirty-five first-year students with ADHD, a learning disability, or pronounced academic struggles, identified through a college disability service office, were invited to participate in a coaching program. Participants completed measures both of self-efficacy and of learning and study strategies. The comparison group consisted of thirty-five additional students, also recruited through disability service office, who were asked to complete the same study measures but were not offered coaching. Zwart and Kallemeyn compared the two groups on gender and year in college, finding no differences, but disability diagnoses did vary significantly between the two groups. While this would typically confound the results of a study, these authors were able to control for the differences in the two groups by using a pretest-posttest design in addition to the comparison group. This combination of approaches makes a comparison group study into a stronger study design.
Several of the coaching studies mentioned above use another study design called a pretest–posttest design. In a pretest-posttest study design, a researcher hypothesizes what factors might be impacted by a coaching intervention and then attempts to measure the variable or variables of interest both before and after the coaching intervention and compare the two measurements. In a sense, the study participants serve as their own comparison group from before the intervention to afterwards. The pretest-posttest design enables researchers to examine the change in the measurement of specific variable(s) from before the intervention to after the intervention, and the difference in pre- and posttest scores is used to explore efficacy of the intervention. Improved outcomes occurring from pre- to posttest suggest that the intervention is a likely reason for the change.
While this is a very strong study design, a caution in the interpretation of results is none-the-less warranted. The results of a pretest-posttest study suggest, but do not prove, that the intervention (e.g. coaching) is the cause of the change. One reason for this is the possibility that other events may occur between the pre-test and the post-test that influence the outcome. As an example, in a study of coaching among college students, let’s imagine that the school offered a series of free classes on study skills, time management, and the like. If students participating in the study also attended these classes, it would be hard to be sure that that coaching, rather than the classes, led to beneficial outcomes. Another reason a pretest-posttest design does not prove causation is that sometimes factors intertwined with an intervention, rather then the intervention itself, can cause the outcome. For example, in a pretest-posttest study of coaching, how can the researchers be confident that it is coaching that causes beneficial outcomes rather than simply the increased personal attention students receive over the course of the study?
Three of the above studies provide examples of the pretest-posttest design. Zwart and Kelleman (2001) administered measures of self-efficacy, as well as of practices and attitudes about learning and studying, both before and after students had a number of peer coaching sessions. Additionally, Maitland and colleagues (2010) examined executive functioning skills, self-determination, and life satisfaction in students both before and after eight sessions of coaching. More recently, Prevatt and Yelland (2013) compared self-esteem, learning and study strategies and several other variables before and after student participation in an eight-week coaching program.
Randomized Controlled Study
A randomized controlled study, or trial, (RCT) is considered the “gold standard” in research design. The key feature of a typical RCT is that study subjects, after meeting criteria for participation, but before the intervention of interest begins, are randomly allocated to receive the intervention (coaching in this case) or not to receive it (in some studies, for ethical reasons, an alternative treatment is offered). The most important advantage of random assignment is that it minimizes bias in selection of group participants. That is, at least in theory, randomly assigning students with ADHD into coaching and non-coaching groups should distribute any pre-existing differences in individual group members in a random, or non-biased, way between the two groups. Of course, researchers also typically compare the two groups on factors known to impact the outcome(s) of interest to assure that randomization has resulted in balanced groups on those factors. Random assignment strengthens confidence that any difference found in outcomes between the coaching and comparison group is indeed due to the effects of coaching rather than any pre-existing differences between the groups. Randomized-controlled studies are less common than other studies because the complexity of their design often makes them more difficult and costly to carry out.
In a study conducted for the Edge Foundation, and published in 2013, Field, Parker, Sawilowsky and Rolands recruited college students with ADHD on several campuses and randomly assigned them to participate in a coaching group or a non-coaching comparison group. “Coaching” students received 24 weeks of phone-based coaching sessions and additional check-ins from the coaches. Pre-test measures for coaching students examined learning and study strategies. Previous semester GPA and other school data were collected for both groups for comparison purposes. Both groups completed the same learning and study strategies measure as well as a measure of well-being late in the school year when GPA and other factors were again collected. This enabled comparing the two groups on the “outcome” measures of interest as a way of determining if the coaching was indeed effective.
Field, S., Parker, D.L., Sawilowsky, S. & Rolands, L. (2013). Assessing the impact of ADHD coaching services on university students’ learning skills, self-regulation and well-being. Journal of Postsecondary Education and Disability, 26(1), 67-81. Abstract available at: http://connection.ebscohost.com/c/articles/90141858/assessing-impact-adhd-coaching-services-university-students-learning-skills-self-regulation-well-being
Maitland, T., Richman, E., & Parker, D. (2010). The Impact of Coaching on Academic Success: A Focus on University Students with Learning Disabilities and Attention Deficit/Hyperactivity Disorder. Presented at Association on Higher Education and Disability Conference, July 14, 2010. http://www.learningace.com/doc/1239880/907bc4a1343e3457874644a922769be2/ahead_presentation_2010_denver_website
Parker, D. and Boutelle, K. (2009). Executive function coaching for college students with learning disabilities and ADHD: A new approach for fostering self-determination. Learning Disabilities Research & Practice, 24(4), 204–215. http://onlinelibrary.wiley.com/doi/10.1111/j.1540-5826.2009.00294.x/abstract
Parker, D., Hoffman, S.F., Sawilowsky, S., & Rolands, L. (2011). An examination of the effects of ADHD coaching on university students’ executive functioning. Journal of Postsecondary Education and Disability, 24(2), 115-132. http://eric.ed.gov/?id=EJ943698
Parker D., Hoffman, S.F., Sawilowsky, S., & Rolands, L. (2013). Self-control in postsecondary settings: Students’ perceptions of ADHD college coaching. Journal of Attention Disorders, 17(3), 215-232. Prevatt, F. and Yelland, S. (2013) An empirical evaluation of ADHD coaching in college Students. Journal of Attention Disorders, e-pub ahead of print, March 18, 2013.
Zwart, L. and Kallemeyn, L. (2001). Peer-based coaching for college students with ADHD and learning disabilities. Journal of Postsecondary Education and Disability, 15(1), 1-15. http://eric.ed.gov/?id=EJ653965
Note: This review was written as a service to ACO members by Elizabeth Ahmann, ScD, RN, ACC, of the ACO Research Committee and represents the perspective of that author. You can contact the author at LizAhmann.com