Disseminating sleep education to graduate psychology programs online: a knowledge translation study to improve the management of insomnia

Abstract Study Objectives Despite the negative impact of poor sleep on mental health, evidence-based insomnia management guidelines have not been translated into routine mental healthcare. Here, we evaluate a state-wide knowledge translation effort to disseminate sleep and insomnia education to graduate psychology programs online using the RE-AIM (reach, effectiveness, adoption, implementation, and maintenance) evaluation framework. Methods Using a non-randomized waitlist control design, graduate psychology students attended a validated 6-hour online sleep education workshop delivered live as part of their graduate psychology program in Victoria, Australia. Sleep knowledge, attitudes, and practice assessments were conducted pre- and post-program, with long-term feedback collected at 12 months. Results Seven out of ten graduate psychology programs adopted the workshop (adoption rate = 70%). The workshop reached 313 graduate students, with a research participation rate of 81%. The workshop was effective at improving students’ sleep knowledge and self-efficacy to manage sleep disturbances using cognitive behavioral therapy for insomnia (CBT-I), compared to the waitlist control with medium-to-large effect sizes (all p < .001). Implementation feedback was positive, with 96% of students rating the workshop as very good-to-excellent. Twelve-month maintenance data demonstrated that 83% of students had used the sleep knowledge/skills learned in the workshop in their clinical practice. However, more practical training is required to achieve CBT-I competency. Conclusions Online sleep education workshops can be scaled to deliver cost-effective foundational sleep training to graduate psychology students. This workshop will accelerate the translation of insomnia management guidelines into psychology practice to improve sleep and mental health outcomes nationwide.

Despite these updates to practice guidelines, CBT-I has yet to be translated into routine mental healthcare.Mental healthcare providers, such as psychologists, do not view sleep problems as a treatment priority [51], nor do they routinely use evidence-based insomnia treatments [52][53][54].This is not surprising, given the ongoing challenges of translating healthcare research findings into clinical practice [55][56][57].Knowledge translation in healthcare (defined as using research evidence to inform health and healthcare decision-making [57]) is slow, with an average of 17 years from publication of high-quality clinical practice guidelines to translation into routine clinical care [58,59].While excellent work has been done to train mental healthcare providers in sleep and circadian rhythms management , there are major barriers to implementing CBT-I in clinical practice.In particular, the demand for trained CBT-I practitioners far exceeds the supply, with only 752 CBT-I specialists worldwide, predominantly based in the United States [81][82][83][84].In Australia, only 7 out of the 35 315 registered psychologists are internationally recognized as CBT-I specialists [82,85], which is insufficient to treat the estimated 3.8 million Australians with insomnia [86].Although some digital interventions are starting to be recognized as sufficient for people with uncomplicated insomnia, people with comorbid insomnia and mental health conditions will continue to need interventions provided by mental healthcare providers around the world [87][88][89][90][91][92].Therefore, more psychologists must be trained in CBT-I.
A second barrier to CBT-I dissemination is that our mental healthcare providers lack general sleep and insomnia knowledge due to the limited sleep education taught within university-based healthcare training programs [52-54, 81, 93].Sleep education is scarce in both medical and psychology programs due to limited time, space, and expertise in the curriculum [69,78,[94][95][96][97][98][99][100][101][102][103][104][105][106].On average, medical students worldwide receive only 2.5 hours of sleep education across their medical degree [107], much less than, for example, the 19.6 hours for nutrition education [108,109].Sleep education provided to psychology students is even lower.Graduate psychology students in Australia receive a median of only 1 hour of sleep education, with 47% reporting no sleep education at all [104].The data are similar in the United States, with only 6% of clinical psychology programs offering a formal course in sleep [100].Without foundational sleep education, most graduate psychology students enter the workforce lacking critical CBT-I knowledge and thus incorrectly believing that sleep hygiene is an evidence-based treatment for insomnia [78,104].
To address these knowledge gaps, sleep education needs to be integrated into the graduate psychology curriculum.Three small studies have developed a sleep education curriculum for graduate psychology programs with promising results [78,110,111].For example, we successfully developed and piloted a 6-hour behavioral sleep medicine education workshop and observed significant increases in graduate psychology students' sleep knowledge and self-efficacy to manage insomnia in clinical practice [111].Notably, students learned that CBT-I is the first-line treatment for insomnia, not sleep hygiene.Students also provided positive feedback about the workshop, with 100% agreeing that sleep education should be included in all graduate psychology training programs.Sustained clinical behavior change was observed at 6 months post-workshop, with all students routinely asking their clients about sleep and many using CBT-I components in practice.However, this study was small in scale (n = 11) and focused on only one university for a short period (one semester), which is unlikely to bring about sustained change in the number of psychologists with sleep and CBT-I knowledge.
To make large-scale improvements in the translation of CBT-I into mental healthcare, sleep education must be widely integrated into the graduate psychology curriculum.Ideally, this should be aided by a knowledge translation framework to target the barriers to implementation [67,81,112].The current study aimed to evaluate the implementation of a large-scale rollout of our validated behavioral sleep medicine education program, called the Sleep Psychology Workshop [111], into graduate psychology programs across Victoria, Australia, using the reach, effectiveness, adoption, implementation, and maintenance (RE-AIM) framework.

Study design
We used a non-randomized waitlist control design, guided by the RE-AIM Framework, to evaluate the implementation of the Sleep Psychology Workshop into all graduate psychology programs in Victoria, Australia in 2020-2021.A non-randomized design was chosen to provide initial scoping data on the state-wide uptake of the Sleep Psychology Workshop [111] into the graduate psychology curriculum while still enabling the collection of control group data.

RE-AIM Framework
The RE-AIM framework is widely used in healthcare to evaluate the implementation of evidence-based interventions in the real world and has been used across a variety of settings (e.g.public health, behavioral science, community health, and policy) [113,114].Table 1 outlines the RE-AIM framework dimensions along with how these dimensions were operationalized for the current study.

Sleep Psychology Workshop: Better Sleep for Better Mental Health
The Sleep Psychology Workshop: Better Sleep for Better Mental Health is 6 hours, live, interactive online workshop designed to provide graduate psychology students with foundational knowledge and skills in sleep, circadian rhythms, and insomnia management for psychology practice.The 6-hour content could be delivered as 3, 2-hour workshops (normal delivery mode) or 2, 3-hour workshops (compressed delivery mode) depending on the structure of the graduate psychology program.The development and pilot study of the Sleep Psychology Workshop has been described elsewhere [111]; however, in brief, the workshop was developed using a modified Delphi method and contained didactic sleep, circadian rhythms, and CBT-I education, along with interactive/blended learning activities (e.g.completing a paper-based sleep diary, sleep restriction therapy calculations), role-play exercises (e.g.taking a sleep history), video (e.g.two-process model of sleep regulation video on YouTube), webbased resources (e.g.sleep health foundation, sleep hub), readings from evidence-based literature and textbooks, and handouts with insomnia management exercises (e.g.constructive worry) for clients.The workshop was designed to be foundational in nature, and not train students to CBT-I competency.All lectures were delivered live online via Zoom or other university-based video-conferencing software.Interactive activities were conducted via break-out groups between students.A registered psychologist (HM) (master-level qualifications in psychology, internationally certified as a diplomate in behavioral sleep medicine, 6 years of clinical experience working as a psychologist in sleep disorder clinics, and 11 years of sleep research experience) delivered the workshop.

Program selection and participant recruitment
Program coordinators of graduate-level psychology training at Victorian universities in Australia (n = 10) were emailed an invitation by the research team to participate in the Sleep Psychology Workshop trial.Program coordinators were advised that the Sleep Psychology Workshop could be run free of charge for their graduate students as part of their coursework requirements or offered as an extracurricular workshop for students to attend in their own time.Program coordinators were also advised that all graduate students could attend the Sleep Psychology Workshop, but their participation in the research project evaluating the workshop was optional.Ethics committee approvals were obtained for this study (RMIT University CHEAN: 05-19/21939 and Monash University MUHREC: 28828).
Participants were students studying a graduate psychology program in Victoria, Australia, that would enable registration as a psychologist upon completion.To be enrolled in a graduate psychology program in Australia, students must have completed an accredited 3-year undergraduate psychology sequence, followed by a fourth-year psychology program of study (e.g.Honors degree in Psychology).Therefore, all students eligible for this research trial were in their fifth year of psychology study or higher.In Australia, there are two graduate pathways to psychology registration: (1) fifth and six (plus) year sequence of formal psychology study (e.g.Master of Psychology [2 years full-time study], Doctor of Psychology [4 years full-time study], or a combined Master of Psychology/PhD in Psychology [4 years full-time study]); or (2) 5 + 1 Internship, which includes a formal fifth year of psychology study, followed by 1 year of supervised practice, and completion of the national psychology exam [115].
Graduate psychology students (all 5th year students or higher) attending the Sleep Psychology Workshop at their university were invited to participate in the evaluation study.The research team emailed the attending students (details obtained from program coordinators) prior to the workshop to provide information about the voluntary research components of the workshop, along with a copy of the participant information and consent form (also contained within the online study questionnaire).Students were advised that participating in the research study was voluntary and would not impact their university assessments, grades, or relationships with university staff in any way.Students provided their consent to participate in the evaluation study by ticking a consent box in the online study questionnaire.

Pre-to post-workshop
The Graduate Psychology Knowledge, Attitudes, and Practice in Sleep questionnaire (GradPsyKAPS) was used to collect demographic information and measure graduate students' knowledge, self-efficacy, preparedness, and confidence in sleep and insomnia management from pre-to post-workshop (RE-AIM domains = reach and effectiveness) [104,111].A shortened version of the GradPsyKAPS (76 items) was used for the current study to reduce administration time and took approximately 20 minutes to complete (see online Supplementary Material).The primary outcome measure for this study was the 35-item multiple choice Sleep Psychology Knowledge Quiz contained within the GradPsyKAPS.The quiz included items about general sleep and circadian rhythms knowledge (e.g.Current guidelines from the American Academy of Sleep Medicine and Sleep Research Society recommend that per day adults need: (1) at least 6 hours of sleep, (2) at least 7 hours of sleep*, (3) at least 8 hours of sleep, (4) at least 9 hours of sleep) and more clinically focused insomnia knowledge (e.g.When applying Sleep Restriction Therapy, which is the best response for the sleep efficiency threshold used for increasing prescribed time in bed? (1) 100%, (2) ≥85%*, (3) ≥75%, and (4) ≥50%,).The GradPsyKaps was administered through the online platform Qualtrics.To reduce the likelihood of students searching for quiz answers in their notes or online, the quiz was timed, allowing students 45 seconds to answer before progressing to the next question.The order of quiz items was changed post-workshop to reduce learning and order effects [116].In addition, the 9-item self-efficacy (7-point Likert scale), the 2-item preparedness (4-point Likert scale), and treatment confidence (4-point Likert scale) subscales from the GradPsyKAPS were used as additional effectiveness outcome measures.

Post-workshop
Graduate students in the intervention groups also completed an additional post-workshop survey to provide feedback about the Sleep Psychology Workshop (RE-AIM domain = implementation).This survey, which took approximately 10 minutes to complete, assessed students' ratings of the workshop and facilitator, as well as more specific feedback regarding their training experience, Table references [110][111][112][113] learning, and workshop content (see online Supplementary Material).

Long-term follow-up
A long-term online feedback survey was administered to students 12 months post-workshop (RE-AIM domain = Maintenance).This measure collected data regarding students' use of workshop-related sleep psychology knowledge and skills in their clinical practice and took approximately 10 minutes to complete (see online Supplementary Material).

Procedures
Students attended the Sleep Psychology Workshop as part of their graduate coursework or as an extracurricular activity.Students' participation in the research components of the trial was voluntary.Students were allocated to the intervention or waitlist control group based on the university and semester they attended.Graduate programs running the Sleep Psychology Workshop for the first time were automatically allocated to the intervention group in 2020.Graduate programs who requested repeat administrations of the Sleep Psychology Workshop to different cohorts formed the waitlist control group which ran in 2021.All participants were sent a participant ID code via email prior to the study to enable the matching of pre-to post-study data and group allocation.

Intervention group
The intervention group ran in 2020.All students in the intervention group were emailed information about the research components before they commenced the Sleep Psychology Workshop.If they wished to participate in the research trial, they completed the online pre-workshop questionnaire with consent form via Qualtrics on their mobile device or computer at the beginning of the first workshop (T1).Students who did not wish to participate in the research activities were permitted to do other quiet activities while other students completed the online survey.After completing the Sleep Psychology Workshop, students were asked to complete the online post-workshop questionnaire within 2 weeks (T2).Twelve months post-workshop, students were invited via email to complete an online follow-up survey.
Students were informed at the beginning of the study that those who completed both the pre-and post-workshop GradPsyKAPS questionnaire and scored at least 50% on the post-Sleep Psychology Knowledge Quiz would receive a certificate of completion for the workshop.In addition, each university group of graduate students who completed the pre-and post-workshop assessments went in the draw to win a $50 e-gift voucher for participating in the study.At 12 months post-workshop, all graduate students who completed the long-term follow-up survey could enter a draw to win one of the five $100 e-gift vouchers.

Waitlist control group
The waitlist control group ran in the first half of 2021.Graduate students in the waitlist control group were provided with information about the research components via email 3 weeks prior to the running of the Sleep Psychology Workshop.They were asked to complete the first GradPsyKAPS questionnaire 2 weeks before the Sleep Psychology Workshop (T1).Students then completed a second administration of the GradPsyKAPS at the beginning of Sleep Psychology Workshop lecture (T2).Students who completed both assessments went in the draw to win a $50 e-gift voucher for participating in the study.If students wished, they could complete the GradPsyKAPS again after the workshop to receive a certificate of completion if they scored more than 50% on the Sleep Psychology Knowledge Quiz.

Data analysis
Study data were analyzed and visualized using Statistical Package for the Social Sciences 28.0 and GraphPad Prism 9.4.1.Descriptive statistics were used to describe the sample.Baseline differences in continuous Sleep Psychology Knowledge Quiz scores were assessed via one-way Analysis of Variance (ANOVA).Changes in pre-to post-workshop sleep psychology knowledge quiz scores were analyzed using mixed between-within ANOVA.For Likert scale ordinal data (e.g.self-efficacy [7-point Likert scale], preparedness [4-point Likert scale], and confidence [4-point Likert scale] subscales) change scores from pre-to post-workshop were calculated and used to compare groups with Kruskal-Wallis tests and post hoc Dunn's tests using effect size r (r = z/ √N).Statistical significance level was p < .05(two-tailed).Where specified, a Bonferroni correction was applied to account for multiple post hoc comparisons between the waitlist control, intervention groups 1, and intervention group 2 (p < .017).
Post-workshop evaluation items were reported on a 7-point Likert scale and collapsed into overall agreement, midpoint (neither agree nor disagree), and disagreement to simplify data visualization.Descriptive statistics were also used to analyze long-term follow-up data regarding knowledge and skills used by students since completing the workshop.Lastly, students' freetext items reporting workshop feedback were coded and grouped according to content and frequency of responses (qualitative content analysis).

Adoption (graduate program level) and reach (graduate student level)
Seven out of ten graduate psychology programs across Victoria, Australia agreed to participate in the Sleep Psychology Workshop trial (adoption rate = 70%).In total, the workshop was delivered 11 times across the seven different graduate programs.Four graduate programs included the Sleep Psychology Workshop as part of their graduate coursework requirements, while three offered it as an extracurricular workshop for students to attend independently.Four programs chose the compressed delivery mode (i.e. two by 3-hour workshops instead of three by 2-hour workshops).For the three nonparticipating programs, reasons provided for nonparticipation included: (1) being unable to approve the curriculum change in time for the research project, (2) coronavirus disease 2019 pandemic challenges, and (3) no response to the study invitation.
In total, 313 graduate psychology students across these seven graduate programs attended the Sleep Psychology Workshop, with 262 consenting to participate in the evaluation study.Eight participants' data were excluded from the analysis; five due to being enrolled in a Master of Counseling rather that a psychology degree, and three due to missing participant ID codes, leaving 254 for the analysis (participation rate = 81%).Of students who consented to the trial, 177 completed both pre-and post-workshop assessments (70% study completion rate).
To calculate the reach of the workshop to the target population of all graduate psychology students provisionally registered as psychologists in the state of Victoria, Australia, we compared the number of participants in the intervention group in 2020 (n = 172) to the Psychology Board of Australia Registrant data from 2020.This allowed us to ascertain the total number of provisional psychologists enrolled in a Higher Degree Program (n = 982) or 5 + 1 internship (n = 427) in Victoria, Australia (total population N = 1409) [117].The reach of the Sleep Psychology Workshop in 2020 to all provisional psychologists in Victoria studying a Higher Degree Program or 5 + 1 internship was 12%.
Group characteristics are displayed in Table 2. Overall, 86% of participants were female, with a mean age of 31 years.Students had received minimal sleep education during their training to date, reporting a median of 2 hours of sleep education during their undergraduate degree and 0 hours in their graduate degree to date.However, 55% of students endorsed already having worked with clients experiencing sleep disturbances while on clinical placement or in previous work experience.Most students were completing a Master of Psychology (64%), followed by the 5 + 1 internship (30%).Students' year of psychology study ranged from fifth to eighth years; however, only two students reported being in their seventh or eighth year of study (<1%).The majority of students were in their fifth year (80%; n = 202), with 20% (n = 50) in their sixth year of study.As data collection allowed us to capture graduate psychology students at different stages of their training, students completing the Sleep Psychology Workshop were grouped according to their year of study (e.g.intervention group 1 = 5th years, intervention group 2 = sixth+ years).Graduate students in the waitlist control group were all in their fifth year of psychology study (waitlist group = fifth years).

Effectiveness (graduate student level)
Sleep psychology knowledge.
Baseline differences in sleep psychology knowledge were assessed via one-way ANOVA and revealed a significant difference between the groups, F (2, 219) = 10.33,p < .001.Post hoc tests indicated that intervention group 2 (sixth+ years) scored significantly higher on baseline sleep knowledge than both intervention group 1 (fifth years) (p < .001)and waitlist control group (fifth years) (p < .001)(see Figure 1).No differences in baseline sleep knowledge were observed between the waitlist control group (fifth years) and intervention group 1 (fifth years) (p = .51).
To assess changes in students' sleep psychology knowledge from pre-(T1) to post-workshop (T2), a mixed-between within-subjects ANOVA was conducted.There was a significant interaction effect between group and time, Wilks' Lambda = 0.47, F (2, 172) = 102.78,p < .001,partial eta squared = 0.54, indicating that sleep psychology knowledge quiz results across time differed between the intervention and waitlist groups.There was a significant main effect of time, Wilks Lambda = 0.37, F (1, 172) = 294.03,p < .001,with large effect size (partial eta squared = 0.63), as well as a significant main effect of group,  1).
Self-efficacy, preparedness, and confidence in behavioral sleep medicine.
Overall, self-efficacy change scores showed significant differences between waitlist control and intervention groups from pre-to post-workshop (all p < .001;see Table 3).Both intervention groups reported significantly increased comfort using sleep-related assessment tools and empirically supported treatments, compared to the waitlist group (all p < .001).In addition, intervention group 1 and 2 reported an increase in their perceived skills to assess and diagnose common sleep and circadian rhythm disorders, compared to the waitlist control (p < .001).Similarly, both intervention groups reported increased knowledge of common sleep disturbances seen in mental health disorders, knowing where to go to access further training in sleep, sleep disorders, and circadian rhythms, as well as knowing more about sleep than other graduate psychology students, compared to the waitlist group (all p < .01).Both intervention groups reported an increase in how prepared they felt to conduct a sleep evaluation and treat a sleep disturbance using an evidence-based approach (e.g.CBT-I), compared to the waitlist group (p < .001,see Figure 2).Lastly, both intervention groups reported significantly increased confidence in treating insomnia disorder and comorbid insomnia using an evidence-based therapy, compared to the waitlist group (p's < .001,see Figure 3).

Graduate program level
The Sleep Psychology Workshop was implemented across the different universities using the same study protocol; however, some inconsistencies in delivery were noted.Time allocation within the workshop was the main challenge, with some workshop content, mainly the cognitive therapy components of CBT-I, receiving less attention at some universities.For example, more extensive sleep questions by students led to more detailed discussions with some groups than others and consequently reduced the planned didactic content of cognitive therapy strategies (e.g.constructive worry and cognitive restructuring) at these sites.However, all workshops prioritized and covered the delivery of sleep restriction therapy, stimulus control therapy, and the insufficiency of sleep hygiene for treating chronic insomnia.In addition, technical issues at one university led to a delay in commencing the workshop, which again rushed some final workshop content.Of note, these time allocation difficulties occurred more at sites that chose the compressed workshop delivery mode (i.e. the universities that chose 2, 3-hour workshops [n = 4] versus 3, 2-hour workshops [n = 3]).However, no differences in students' post-workshop sleep knowledge quiz  The letters in parentheses in column heads refer to the numbers used for illustrating significant differences in the "Post hoc" test column.Data were collected on a 7-point Likert Scale, from 1 = Strongly Disagree to 7 = Strongly Agree.(n = 177) Change scores from post-to pre intervention were used for statistical analysis.

Postmedian
A Bonferroni correction was applied to post hoc Mann-Whitney U Tests with a stricter α level of 0.017.Effect sizes for post hoc comparisons between groups b versus a ranged from r = 0.56 to.73, indicating a large effect.Effect sizes for post hoc comparisons between groups c versus a ranged from r = 0.39 to.60, indicating a medium to large effect.
scores were identified between the compressed and normal workshop delivery modes (p = .098).Lastly, "break out" group work was not possible at one site due to technical difficulties, so role-play exercises were completed as a whole class rather than in pairs.

Graduate student level
Overall, 138 students from both intervention groups completed the post-workshop evaluation survey (80% completion rate), with 95% of students reporting they attended all 6 hours of the Sleep Psychology Workshop.Students provided positive feedback about the workshop, with 96% rating it as very good-to-excellent (29% = very good, 67% = excellent), and 99% of students also rating the workshop facilitator as very good-to-excellent (11% = very good; 88% = excellent).Additionally, >73% of students endorsed their  overall agreement across the different aspects of training experience, learning, and workshop content items (Table 4).Post-workshop, students recognized the importance of sleep, sleep disorders, and circadian rhythms education for psychologists.Overall, 99% of students believed that training in sleep, sleep disorders, and circadian rhythms was important to their future careers as psychologists, with 100% agreeing that all psychologists should understand the relationship between sleep and mental health (see Supplementary Table S1).Notably, most students endorsed sleep, sleep disorders, and circadian rhythms (100%), and Cognitive Behavioral Therapy for Insomnia (94%) should be a standard part of the graduate psychology curriculum.Lastly, 96% of students planned to continue learning about sleep, sleep disorders, and circadian rhythms in future, with 75% interested in gaining a professional certification in behavioral sleep medicine once registered as a psychologist and 66% open to studying a postgraduate degree in behavioral sleep medicine.
Twenty-nine students (21% of the post-workshop evaluation sample) provided written feedback on workshop limitations.Nine students commented that the workshops were rushed at the end; seven students commented that they experienced zoom fatigue with the online workshop format; and six reported they would have preferred 3, 2-hour workshops, instead of the 2, 3-hour workshops.Five students also suggested extending the time of the workshop (e.g. to 7 hours total).In regard to workshop content, six students requested more interactive activities (e.g.history taking, interventions), more case examples, and time for discussion with the group, with some reduction in the didactic presentation.The workshop improved my confidence to manage sleep disturbances in my clients The workshop improved my competence to manage insomnia in my clients The class exercise of role-playing the two-process model of sleep regulation as client and therapist was useful The class exercise of taking a sleep history was useful 1 14 86 The class audio and case formulation of the client with insomnia ("John") was useful

Graduate program level.
Twelve months after completing the Sleep Psychology Workshop trial, four of the seven graduate programs enquired as to whether the workshop could be delivered to the subsequent years graduate cohort.In addition, two of the graduate programs that did not participate in the research trial contacted the research team requesting the inclusion of the workshop at their university the following year.This indicates an ongoing demand from at least six out of ten universities in Victoria, Australia, for sleep, circadian rhythms, and insomnia education within their programs.Additionally, one graduate program integrated a shortened version of the Sleep Psychology Workshop into their program coursework (with the researcher's permission using a "train the trainer" model).This modified version of the Sleep Psychology Workshop formed the basis of a new placement opportunity for students at that university to undertake sleep assessments for members of the community on the University Open Day, and to provide individualized feedback on sleep-wake patterns, sleep duration, chronotype, insomnia symptoms, risk of obstructive sleep apnea, and refer on to appropriate sleep services if required.
Students reported using foundational behavioral sleep medicine skills at the 12-month follow-up, with 100% routinely asking their clients about sleep and 96% feeling comfortable providing psychoeducation about sleep.However, they were less confident in more advanced skills, such as taking a sleep history, referral processes, and delivering evidence-based interventions (see Table 5).
Graduate students reported strong implementation and maintenance of the knowledge and skills learned in Sleep Psychology Workshop.Overall, 83% of students endorsed using some of the knowledge and/or skills learned in the workshop on placement or in their clinical practice (see Table 5).However, fewer students had put key behavioral sleep medicine assessment skills into practice, such as using a sleep diary (31%), taking a sleep history (25%), knowledge of common sleep disorders (25%), or administering sleep questionnaires (9%) (see Table 6).For insomnia items, students reported good memory (76%) and clinical application (53%) of the knowledge that spending too long in bed is a common perpetuating factor for insomnia.Students overall reported good memory and application for the components of CBT-I, with 68% using psychoeducation about sleep, 65% using sleep hygiene education, 64% using relaxation training, 47% using stimulus control therapy, and 45% using cognitive therapy in clinical practice.However, only 23% endorsed applying sleep restriction therapy in clinical practice since the completion of the workshop (Table 6).

Final workshop feedback.
Twelve months on, 91% of students rated the Sleep Psychology Workshop as very good-to-excellent (very good = 31%, excellent = 60%).There was still strong agreement that all graduate psychology students (99%) and registered psychologists (96%) should receive training in sleep, sleep disorders, and circadian rhythms.Notably, 100% of students endorsed that it was important to address sleep disturbances with their clients in clinical practice.
Since workshop completion, 53% reported they had referred to the workshop materials; mainly lecture slides, sleep diaries, and sleep restriction therapy information.In the overall workshop feedback, it was noted that students would have preferred a take-away workshop manual for easy reference.Some students reported completing additional sleep education activities to upskill in CBT-I since completing the workshop, including accessing online sleep information (e.g.Sleep Health Foundation, Sleep Hub; 32%), reading journal articles/textbooks (25%), and individual or peer supervision (22%).Few had taken an online course (5%) or an in-person workshop (1%).However, 38% of students endorsed wanting to receive further training in CBT-I.[113] to improve the dissemination of sleep and CBT-I knowledge in mental healthcare.
The strong adoption rate of the Sleep Psychology Workshop demonstrates great interest and demand for sleep education within graduate psychology programs.Not only was the Sleep Psychology Workshop adopted by 70% of programs, but after the research trial concluded, 60% of programs reported ongoing demand for the workshop at their university.While most universities wanted the researchers to facilitate ongoing sleep education workshops, one graduate program used a "train the trainer" model to integrate a shortened version of the workshop into their curriculum.Our research demonstrates that graduate programs are willing to adopt a sleep education workshop into their curriculum if facilitated by external experts.Therefore, sustainable annual training models should be established to enable additional sleep education curriculum to be integrated across graduate psychology programs [112].Examples could include an educational trainer (funded by professional sleep associations) who delivers the sleep education to university programs, an educational trainer who provides a more in-depth training program for faculty to facilitate the sustainability of the workshop within the institution, or the workshop could be converted into a selfpaced digital training program to allow for broader dissemination [65,66].Ideally, sleep education should be a hurdle requirement for all graduate psychology students, ensuring that they possess basic behavioral sleep medicine knowledge and skills before graduation.
The Sleep Psychology Workshop reached a meaningful number of future mental healthcare providers (n = 313), representing 12% of all graduate psychology students holding a provisional psychology license in Victoria in 2020.However, factors beyond our control limited our ability to capture a higher proportion of graduate students during the study period.Due to time limitations and coronavirus disease 2019 restrictions, we were only able to run the workshop at some universities for 1 year level (e.g.fifth years) and so did not reach other enrolled students (e.g.sixth years).Furthermore, students enrolled in the 5 + 1 internship are off campus working full-time for their internship year and may not have been informed of, or able to attend, the Sleep Psychology Workshop.Lastly, Psychology Board of Australia registrant numbers include graduate students who are no longer enrolled in classes (e.g.fifth and eighth-year students finishing research components or those awaiting degree conferral), limiting their ability to attend the workshop, despite being counted in registrant numbers [117].Therefore, future trials of this workshop should ask graduate programs to advertise the Sleep Psychology Workshop to all students, including those who are awaiting degree conferral, conducting research, or out on internship placements to increase program reach.
Similar to our pilot study [111], and other small sleep education studies [78,97], the sleep education intervention was effective at improving graduate students' sleep and insomnia knowledge, with some educational gains maintained over time.Significant increases in sleep knowledge were found for both fifth and sixth-year students, with most students demonstrating retention of basic sleep knowledge 12 months post-workshop.Together, these findings suggest that introducing the Sleep Psychology Workshop early during the fifth year of graduate school provides trainees with large improvements in important sleep knowledge before commencing clinical placements.Future workshops should also consider additional tangible materials to help students retain and implement sleep knowledge into practice.
Unfortunately, knowledge does not equal competency, nor does knowledge alone lead to behavior change [61,67].This was seen in our long-term maintenance data, which identified that not all students had implemented the sleep-specific skills they had learned into their clinical practice.Notably, students reported increased self-efficacy, preparedness, and confidence to manage sleep disturbances, like insomnia, post-workshop.However, while almost all students asked their clients about sleep, only 25% had taken a sleep history and 9% had administered common sleep questionnaires.Furthermore, basic interventions, such as sleep psychoeducation, sleep hygiene, and relaxation training had a higher uptake than more specialized CBT-I skills, such as stimulus control and sleep restriction therapy.While students had worked in a range of clinical settings focusing on other disorders (e.g.depression, anxiety), 87% of students endorsed working with clients experiencing sleep disturbances in clinical practice at 12 months post-workshop.Therefore, students had adequate opportunities to practice more advanced sleep skills, but our long-term maintenance data suggest that students did not put these skills into practice.These findings are consistent with the training literature, which outlines that didactic training in mental health treatment alone is not always associated with substantial improvements in clinicians' treatment skills [118][119][120][121][122].However, while it is clear that 6 hours of sleep education is not enough to achieve CBT-I competency, the Sleep Psychology Workshop is clearly more effective than the minimal sleep education that graduates psychology students currently receive [104].More "hands-on" competency training and assessments (e.g.case-based training tools and structured oral examination), integration of sleep interventions on clinical placements, or ongoing professional consultation (e.g. group supervision/telephone consultation) appears crucial for bridging the CBT-I evidence-practice gap [60,61,72,94,118,122].
Lastly, successful implementation was reflected in the overwhelmingly positive feedback students provided about the quality of the workshop, as well as their learning and training experiences.Furthermore, students were very positive about the importance of sleep and the integration of CBT-I education into all graduate psychology programs.Unlike our pilot study, the role-playing exercise for the two-process model of sleep regulation received a lower quality rating in this study (73% vs. 87.5% in the pilot study) [111].This difference is likely a result of differences in implementation, with the pilot study delivered in-person (vs.online), allowing the workshop facilitator to listen to students' role plays and provide prompts/suggestions.It was not possible to "drop-in" on all break-out group role plays, which limited students' ability to receive facilitator assistance and feedback, and may have resulted in students not appreciating the full potential of this exercise.In addition, time allocated to the cognitive therapy components of CBT-I differed as a result of time spent engaged in discussions with students over the course of the program.Future workshops should build in additional time for student questions/engagement, adhere more strictly to the timing allotted to each component, and/or increase the time for workshop delivery.

Strengths
We assessed the real-world implementation of a sleep education workshop within healthcare professional training programs using the validated RE-AIM evaluation framework.By using RE-AIM, we moved beyond individual-level effectiveness data (e.g.changes in students' sleep knowledge from pre-to post-workshop) to setting-level adoption, implementation, and maintenance data.Setting-level data provided valuable evidence that sleep education can feasibly be implemented into real-world healthcare provider training programs at a state-level.The RE-AIM framework has highlighted the need to focus on the long-term maintenance of the sleep education within graduate psychology programs and provide more practical sleep training opportunities for students to achieve CBT-I competency.

Limitations
Limitations of this work include that we did not use a randomized-controlled design or include objective clinical competency measures.Also, only one facilitator conducted the workshop, which may have affected workshop efficacy.Future studies would benefit from increasing the number of facilitators conducting the workshop.Also, future studies should allocate graduate programs to the intervention or waitlist group in a randomized fashion, ensuring that all groups have an equal distribution of clinical psychology students and fifth versus sixth-year students.Additionally, including objective competency assessments (e.g.objective structured clinical examination) would allow for the assessment of students' practical sleep skills in post-workshop.Lastly, students were advised at the study outset that they receive a certificate of completion for scoring ≥ 50% on the post-workshop Sleep Psychology Knowledge Quiz, which may have influenced their motivation to retain the sleep information presented in the workshop.

Conclusion and Future Directions
The goal of this innovative study was to widely disseminate an online sleep education workshop to graduate psychology students in Australia.The Sleep Psychology Workshop was effective at both the individual-and setting-level for improving students' sleep knowledge and skills with a strong adoption rate by graduate psychology programs.To improve the long-term maintenance of this program within graduate psychology programs, we are collaborating with the Australasian Sleep Association (ASA)'s Behavioral Management of Sleep Disorders Education Subcommittee to strengthen the workshop and drive this dissemination initiative.We also plan to improve the implementation and maintenance of students' sleep knowledge and skills by offering ongoing group consultation/supervision through a Behavioral Sleep Medicine Online Community of Practice (BSM Online CoP) established through the ASA.Together, we can drive the increase of sleep knowledge and skills within the mental healthcare workforce and build the number of psychologists with CBT-I expertise.For the field of CBT-I dissemination, these are "exciting times, indeed!" [80] Funding

Figure 2 .
Figure 2. Graduate students' feelings of preparedness to evaluate (A) and treat (B) sleep disturbances across groups from pre-to postintervention (n = 177).T1 = pre-workshop; T2 = post-workshop.Items were recorded on a 4-point Likert scale from 1 = not prepared to 4 = very prepared.Error bars indicate 95% confidence intervals.A Bonferroni correction was applied to post hoc Mann-Whitney U Tests with a stricter α level of 0.017.Effect sizes for post hoc comparisons between intervention group's 1 and 2 versus waitlist control ranged from 0.65 to 0.83, indicating a large effect.

Figure 3 .
Figure 3. Graduate students confidence to treat insomnia disorder (A) and treat comorbid insomnia (B) using an evidence-based therapy (n = 177).T1 = pre-workshop; T2 = post-workshop.Items were recorded on a 4-point Likert scale from 1 = not confident to 4 = very confident.Error bars indicate 95% confidence intervals A Bonferroni correction was applied to post hoc Mann-Whitney U Tests with a stricter α level of 0.017.Effect sizes for post hoc comparisons between intervention group 1 and 2 versus waitlist control ranged from 0.59 to 0.83, indicating a large effect.

2 12 86 I 4 96 I
am more likely to try to assess and manage sleep disturbances in my clients now, than I was before the workshop 1 am interested in learning more about sleep, sleep disorders,

Table 1 .
A Description of the RE-AIM Framework Dimensions With Operationalized Definitions

RE-AIM dimension Definition Operationalized study definition
For individual-level implementation, we explored students' evaluation and feedback about the Sleep Psychology Workshop.At the graduate program level, we reflected on the implementation of the Sleep Psychology Workshop across the different graduate programs and report on the consistency of delivery across sites.Maintenance Maintenance includes to medium-to long-term sustainability of an intervention or program, such as whether the behavior is sustained by individuals after the intervention and whether the program becomes a part of routine practices or policies at the setting level (setting and individual level).At the student level, we assessed students' use of the knowledge and skills taught in the Sleep Psychology Workshop 12-month post-workshop.At the program level, we recorded programs interested in continuing the Sleep Psychology Workshop or integrating the content into their curriculum.

Table 4 .
Evaluation of the Sleep Psychology Workshop (Percentage of Responses; n = 138)

Table 5 .
Self-Efficacy in Behavioral Sleep Medicine Skills at the Long-Term Follow-up Survey Compared to the control condition, the workshop was highly effective at improving graduate students' sleep psychology knowledge and their self-efficacy, preparedness, and confidence to manage insomnia in clinical practice.Notably, students maintained their sleep knowledge and skills 12 months

Table 6 .
Graduate Student's Memory and Use of Sleep Knowledge and Skills 12 Months After Completing the Sleep Psychology Workshop