17 Using Quality Improvement (QI)-Focused Evaluation to Redesign Direct Home- and Community-based Services During the COVID-19 Public Health Emergency

Moriarta, M., Cahill, A., & Fredine, H. (2021). Using Quality Improvement (QI)-Focused Evaluation to Redesign Direct Home- and Community-based Services during the COVID-19 Public Health Emergency Developmental Disabilities Network Journal,  (2), 221–237. https://doi.org/10.26077/8ECF-F29F

Using Quality Improvement (QI)-Focused Evaluation to Redesign Direct Home- and Community-based Services during the COVID-19 Public Health Emergency PDF File

Introduction

An estimated five million people receive some form of home and community-based services in the U.S. Approximately 4.7 million receive services through one of the federal Medicaid Waiver or state plan programs, many with intellectual or developmental disabilities (Musumeci & Chidambaram, 2020). Home-visiting programs serving families with pregnant women or young children serve approximately 350,000 families (National Home Visiting Resource Center [NHVRC], 2020), many of whom are from lower socioeconomic groups or minority communities (Sama-Miller et al., 2018). One factor that clients in all of these programs have in common is that they are members of vulnerable populations, which have been shown to be at higher risk of infection from COVID-19. For example, a recent analysis of 64 million patient records from 547 healthcare organizations found that people with intellectual disabilities were 2.5 times more likely to contract COVID-19 (Gleason et al., 2021). A study by the Centers for Disease Control and Prevention (CDC) found that pregnant women are more likely to both contract the infection and are at greater risk for severe outcomes (Zambrano et al., 2020). A 2020 analysis of over 1 million people who contracted COVID-19 published in the Journal of Public Health found that lower socioeconomic status (SES) and race were both significantly positively associated both with initially contracting the virus and side effects (Hawkins et al. (2018).

Using the Association of University Centers on Disability (AUCD) definition for reporting about service programs, home and community-based services are defined as specialized, non-clinical services offered with to enhance the well-being and status of the client or family receiving them. These services are offered by many Centers within the national network of University Centers for Excellence in Developmental Disabilities Education, Research and Service (UCEDDs). These services have been identified as playing “…a crucial role in keeping people…safely at home and in their communities. During the COVID-19 pandemic, these services are both more important, and under more stress, than ever” (Edwards, 2020).

UCEDDs design and implement home- and community-based direct service programs that reflect the unique needs and target populations within the state or local area they serve, the requirements of evidence-based protocols, as well as specific mandates of funding partners and related federal and state regulatory requirements. The Center for Development and Disability (CDD) at the University of New Mexico (UNM) offers a variety of home- and community-based direct service programs with a diverse range of client populations that include case management, consultation, prevention, and intervention services traditionally delivered via face-to-face, in-person meetings in family homes, and other community settings. These nonclinical programs include the following.

• Family-centered nurse case management services provided statewide for children and adults with developmental disabilities who have been diagnosed with long-term chronic health condition that require skilled care and assessment prior to age 22 (the Medically Fragile Nurse Case Management Program [MFCMP]).
• Consultation services supporting adults with intellectual disabilities who have opted to participate in the state’s Mi Via (“My Way”) self-directed Medicaid waiver program.
• Parents as Teachers (PAT), an evidence- and prevention-based home visiting program provided by trained parent educators providing services to families from the prenatal period through age 5.
• The Nurse Family Partnership (NFP), in which licensed registered nurses provide evidence-based prevention services to families from the prenatal period through age 2.

Together, these programs serve well over 1,000 families at any given time. Two of the four programs serve families statewide (MFCMP and Mi Via), while the others provide services across multiple counties that include both urban and rural communities. For a variety of reasons, providing home- and community-based services to clients in these programs was challenging in New Mexico even before the pandemic struck. New Mexico is the fifth-largest state by land mass (Economic Development Department, 2021) and the sixth least densely populated state or territory in the U.S. (U.S. Census Bureau, 2021a). This results in service delivery challenges for direct-service programs that rely on delivering services in clients’ homes. Before the pandemic, Center staff often drove long distances to serve clients and families.

In addition, lack of access to consistent broadband internet service is common across the state. The U.S. Census Bureau (2021b) ranked New Mexico 48th in the country for the percentage of households with broadband internet connections in 2019. The Executive Director of the state’s Public-School Facilities Authority estimates that nearly 25% of primary and secondary students in New Mexico lack internet access at home (Mckay, 2020). This is especially true in rural, tribal, and more remote areas of the state. In testimony before the U.S. House of Representatives Committee on Energy and Commerce in July of 2020, President of the Navajo Nation, Jonathon Nez, estimated that less than half of Navajo chapters—spread out over 27,000 square miles across three states—have access to broadband internet (Nez, 2020).

New Mexico is also a “minority-majority” state. According to the last U.S. Census, half the population of the state cite Hispanic ethnicity (49%) and 11% of the population report Native American heritage (U.S. Census Bureau, 2021). Both Hispanic and Native Americans in the state have significantly lower incomes (New Mexico Center on Law and Poverty, 2018). The clients and families served by CDD direct service programs are broadly representative of the racial and ethnic diversity of the state with a range of 31% to 60% of clients served in specific programs reporting Hispanic ethnicity and the proportion of Native Americans receiving services in each program ranging from 6% to 16%.

The economic impact of the pandemic has greatly affected the entire population of the state. According to a special report by New Mexico Voices for Children (2021) using data from the 2020 Kids Count Data Book, as many as 34% of New Mexican children were food insecure in 2020, compared to 24% in 2018; 51% of adults in households with children had lost employment income since March 2020; and by the fall of 2020, nearly 30% of adults in households with children had little to no confidence in their ability to pay their next rent or mortgage payment on time.

In March 2020, because of the rapid spread of COVID-19, as in many other states across the U.S., New Mexico’s governor declared a public health emergency and all in-person home and community services were abruptly halted throughout the state. In the first days and weeks following this declaration, all direct-services programs at the CDD required rapid recalibration to use distance-based technologies and virtual platforms for service delivery. As this transition was taking place, CDD faculty and staff simultaneously designed and implemented an ongoing, hybrid quality improvement/evaluation process with two purposes in mind. The first purpose was to gain real-time knowledge of the realities of program implementation in this new environment—given most program staff and clients had little experience with using distance-based methods. The second was to assess the short- and long-term impact of shifting service delivery from in-person to remote service delivery for both the staff and those served by these programs.

Methods

Using a combination of interviews, focus groups, surveys and programmatic data, this effort focused on four primary topics.

1. How did clients rate the quality and efficacy of distance-based services? How did clients/families compare virtual to in-person services?
2. What were the experiences of service providers as the switch to distance-based services occurred, including barriers revolving around technology and altered service-delivery processes?
3. What adjustments did program leaders make to support staff and service delivery?
4. How did programs use technology to deliver distance-based services? How well equipped were Center programs to operate remotely in terms of technology, and how were obstacles overcome?

Using a hybrid program evaluation and mixed-method research design, qualitative and quantitative data were collected via interviews with program leads, focus groups with key staff, and online surveys of direct-service clients from May through September 2020. A total of 36 staff participated in either interviews or focus groups via videoconference. The purpose of the interviews and focus groups was to assess the efficacy of the distance-based services as opposed to face-to-face services. The interviews and focus groups had similar protocols and included questions on the following:

• Previous experience of program staff with distance-based technologies;
• Programmatic changes made to accommodate distance-based services;
• Involvement of funding partners in changes made; and
• Training programs conducted for staff in order to provide services remotely.

The online survey of program clients was offered in both English and Spanish. A total of 270 client surveys were completed, including 25 in Spanish. The survey was anonymous and voluntary. Each program’s participants were surveyed independently, and the average response rate was 40%. Survey questions focused on:

• Communication methods used by program staff;
• Changes in the amount of interaction with staff;
• Perceived changes in effectiveness of distance-based services;
• Overall satisfaction with distance-based services; and
• Preferences for client services in the future.

Basic descriptive statistics including frequencies and cross tabulations were performed in Survey Monkey and Microsoft Excel. Qualitative data were manually reviewed, categorized, and coded thematically.

Prior Research: Using Virtual Platforms to Provide Direct Services in Home and Community Settings

In the past decade, a significant body of research has focused on assessing the effectiveness of “telemedicine”—providing direct health care services using virtual service-delivery platforms. A 2017 scoping review in the Journal of Telemedicine identified over 1,200 studies published between 2013 and 2017 that investigated the efficacy of telemedicine in various medical specialties (Kidholm, 2018). Unlike telemedicine, “telehealth” is a less well-defined term that denotes “the use of telecommunication and virtual technology…outside of traditional health-care facilities” (World Health Organization [WHO], 2018). The term incorporates a range of platforms, including synchronous communication, (individuals communicating in real time using videoconferencing or other technologically based means; Barak, 2018), which is our focus in this article.

While the body of systematic research assessing the use of “telehealth” in home- and community-based services is smaller, it is relatively robust and includes well-planned pilot or feasibility studies, including a number of control group studies that directly compared the efficacy of in-person and telehealth services within one program. Prior research on the feasibility of telehealth has generally focused on four key areas: (1) client satisfaction, (2) comparative effectiveness of telehealth and in-person services, (3) barriers to implementation from both the provider and client perspective, and (4) a comparison of substantive program goals and outcomes by those receiving services in-person versus telehealth. The CDD’s assessment presented here examined the first three.

Prior Findings on Client Satisfaction with Telehealth

In a pilot feasibility study of the PAT home visiting program at the University of Southern California (USC), 74 families participated in telehealth-based services over an 18-month period. Eighty percent reported that they were “very satisfied” with the program (Traube et al., 2020). In a small pilot study of nine families receiving telehealth-based services in the Chicago Health Promotion and Prevention Parent Program, all parents found the telehealth-based services useful and the technology easy to use (Breitenstein & Gross, 2013). A pilot program in Australia that enrolled nine families in a telehealth-based counseling program offered by clinical social workers using computers, tablets, and smartphones reported that, despite technical glitches such as loss of sound and video connections freezing intermittently freezing, parents rated their overall satisfaction as 4.2 on a 5-point scale, with 5 being very satisfied. (Owen, 2020).

Prior Findings on Comparative Effectiveness of Telehealth and In-Person Services and Client Preferences

Eighty-one percent of participating families in the USC PAT pilot reported their experience receiving services via telehealth was the same or better than in-person programs in which they had participated (Traube et al., 2020). All parents participating in the Australian pilot of social work counseling said that they would use telehealth for future counseling. A 2015 study found that some clients actually preferred telehealth to in-person services, particularly when they experience feelings of discomfort when talking about their problems (Stubbings, 2015).

Prior Findings on Barriers to Implementation

Prior studies (Molfenter et al., 2015; see also Adler et al., 2014; Robben et al., 2012) have identified resistance to change by program staff and unfamiliarity of staff with both the technology used and differences in service delivery methods between face to face and remotely as barriers to telehealth service delivery. Within this theme, professionals’ negative expectations about the effectiveness of telehealth interventions and their lack of technology experience and training were also cited as barriers (Backhaus et al., 2015; see also Richardson & Simpson, 2015). Difficulties with reimbursement for telehealth from public funding sources such as Medicaid and Medicare as additional roadblocks to implementing telehealth services (Silva et al., (2015). Several studies identified the lack of broadband access for families in rural or remote areas and/ or among families with limited means to purchase access to broadband internet services as barriers (Kahn et al., 2014; see also LeRouge & Garfield, 2013). Finally, a number of studies identified privacy and security of information shared during telehealth sessions as a barrier (Cherney & van Vuuren, 2012; see also Sinclair et al., 2013).

From Pilots to Everyday Practice

A key difference between these past studies and the reality of moving to telehealth-based direct services in the spring of 2020 is that, without exception, these prior studies were rigorously planned “pilots” or feasibility studies, and telehealth-based interventions were carefully developed prior to implementation. These studies took place over a predetermined period of time and preparation included support and training for staff implementing these new models. For example, the PAT pilot study at USC included a rigorous training and reflective supervision program for staff that included telehealth-based practice as a topic.

In the midst of the COVID-19 public health emergency, we were faced with little opportunity to engage in planning and preparation before in-person services were abruptly halted, and telehealth services were initiated out of necessity. In most of the pilot or feasibility programs reviewed, staff and client participation was voluntary, with clients asked to sign informed consent forms before the study began. Further, they included only a subset of staff and clients—some in control group studies—or clients recruited specifically for the telehealth pilot designed and conducted by dedicated experts knowledgeable in research/evaluation methods. Finally, unlike the pilot or feasibility studies reviewed, the move to telehealth by CDD programs required all program staff, regardless of their knowledge of or comfort with technology-based service delivery, to begin offering telehealth services with little to no time for preparation; participation was not voluntary.

However, even with these differences, prior pilot and feasibility studies provide a useful guide to assessing the unplanned move to distance-based service delivery necessitated by the COVID-19 Public Health Emergency across the UCEDD Network.

Findings

The findings presented include qualitative and quantitative data from client impact surveys, one on one interviews with program directors/leads, and group interviews with staff from each direct service program. Generally, our findings are consistent with the results of previous telehealth pilot and feasibility studies reviewed for this article showing clear efficacy for the use of telehealth as a platform for service delivery across diverse programs. In addition, these findings include infrastructure and support needs required to implement virtual service delivery effectively.

Finding One

Although CDD direct service programs varied slightly in their technological readiness to make an unplanned move to telehealth services, all did so successfully and without interruption in services. All CDD programs had access to the basic technological infrastructure required to provide telehealth services, including laptop computers, access to the Center’s servers through the CDD’s virtual private network (VPN), and broadband internet services. All direct service staff had access to work-issued smart phones. Some staff needed cameras and upgraded laptops and the Center’s Technology Services (TS) Unit worked rapidly to meet equipment needs and provide technology support. The CDD’s Medically Fragile Case Management Program Manager noted, “Everyone had the capability to do remote work before the pandemic.” In addition, the Nurse Family Partnership home visiting program had previous experience using telehealth as a “special case” strategy when a home visit might create health concern for the client. However, though permitted, these telehealth visits could not be billed as “telehealth was approved but not allowed as billable [because the funder] considered it supplementary.”

The one consistent need across all direct service programs was individual access to a video-conferencing technology platform to conduct telehealth visits with clients. The University maintained an Enterprise license agreement with Zoom, but prior to the COVID-19 public health emergency, only a handful of CDD employees had designated, individual Zoom accounts. Once the realities of the pandemic became apparent, the University quickly offered access to Zoom accounts to all University staff and by mid-March 2020, all CDD staff had individual accounts. Interviews with program leads and focus groups with program staff revealed that familiarity with videoconferencing and other technology-based work procedures such as using VPN to connect to the Center’s servers varied greatly, confirming findings in other settings discussed above. To overcome this barrier, Center Technology Services staff provided consultation and direct support to program staff regularly.

Finding Two

All programs used multiple methods to provide services. Although eventually, Zoom became the standard platform for most direct service telehealth visits, in the beginning many staff used multiple methods to retain contact with clients and families; they worked diligently by all means available to ensure there were no significant interruptions in service delivery (see Table 1).

In what way have you met with program staff since face-to-face meetings ended? Though all CDD programs utilized multiple methods to sustain contact with clients, the range varied widely across programs. For example, 67% of Mi Via Consultants supporting adults with intellectual disabilities receiving services under the state’s self-directed Medicaid waiver communicated by telephone only. This was in contrast with the PAT Home Visiting Program serving families with infants and toddlers where the percentage of “telephone only” contact was only 6%. The nature of the populations served, and specific services provided by the two very different programs, likely explain these differences, and speak to the tailoring of service delivery to meet client needs across programs. In some instances, difficulties caused by a lack of reliable broadband internet access for families created initial barrier to using videoconferencing. This was particularly true in rural, remote parts of the state and impacted the Center’s statewide programs (Mi Via and MFCMP) more so than the CDD’s the Home Visiting programs, which largely serve families in the metropolitan area in and surrounding Albuquerque where internet access is generally more consistently reliable.

Finding Three

The majority of clients (89%) reported that they had the same or more interaction with program staff since the switch to telehealth services (see Table 2).

Compared to face-to-face visits, how much interaction have you had with program staff since the move to “telehealth”? In fact, across programs, a number of staff noted that many clients and families wanted more contact with them. This was presumably due to the isolation experienced during the pandemic. One program director noted, “It seems that all my participants are happy with [our program] because we’ve had so much more time with them”, while a second observed that “families want to meet more frequently virtually; they’re eager for services—they felt cut-off from all other services.” Finally, another program lead remarked, “Clients needed the program staff more than ever.” One program began more frequent outreach with single parents after observing that these individuals seemed more socially isolated. It was also noted by program leads that it was in fact possible to increase the number of visits because staff no longer needed to drive long distances to meet in person. One staff noted that using telehealth saved “hours to pack and unpack plus over 100 miles plus in travel.”

Finding Four

Health concerns related to the COVID-19 Public Health Emergency underscored many clients’ desire for virtual services while in some cases others simply preferred them to in-person visits. In many instances, across programs, family members identified health risks for their children and themselves during the pandemic as a major driver of preferring telehealth services. One parent remarked, “During this time, I think it is necessary and appreciate the move to telehealth calls.” A second remarked, “[Telehealth] is a good option if there is a need to not have anyone in my house due to illness. Finally, one noted that “For the time being we do prefer telehealth visits for our safety and the safety of our home visitor.”

Staff across programs also noted the phenomenon that occurred in pilot studies: some clients genuinely preferred telehealth. In fact, an average of 45% of clients agreed with the statement that “I would prefer to have visits using Telehealth rather than face-to-face.” One program staff noted that many clients in their teens preferred using the telephone, which seemed to make it easier to discuss sensitive topics: “Not having visual…contact was beneficial in this case, and the teen was able to open up more without showing their face.”

Clients who supported the continued use of telehealth for visits, even after the public health emergency is over, focused on ongoing health risks faced by their children throughout their lives. One parent noted, “Don’t be in a rush to start face-to-face appointments because it puts the high-risk patients at an even higher risk.” Another commented, “We would appreciate the opportunity to have more telehealth visits. It is so much better for our family.” One client stated that the use of telehealth satisfied their personal preferences: “Yes, due to [me] having issues with people in [my] personal space this helps”, referring to virtual visits.

Finding Five

Clients were satisfied with telehealth services and felt that they were as effective as in-person services. However, most clients said that the interpersonal aspects of in-person, face-to-face meetings were valuable and should be part of the “mix” of service delivery when the public health emergency is over.

Across programs, 96.5% of clients said they were either “very satisfied” or “satisfied” with the services they received via telehealth. Over three quarters (82%) said that the telehealth services they received were “about the same” in terms of effectiveness, while just under a fifth said they were less effective (see Table 3).

Overall, how effective were the telehealth visits compared to face-to-face? One parent noted that program staff exhibited “the same excellence” when using telehealth as they did in face-to-face visits. A second observed that there had been no interruption in services: “Disruption of communication between us has been virtually non-existent.”

However, the majority of parents did note that not having in-person, interpersonal communication left them feeling something was “missing.” One parent noted that “Practically, telehealth did the same thing but not as fun, rejuvenating and therapeutic as in person.” Another noted that “[My son] misses seeing [his nurse case manager] in person but technically Zoom visits worked very well for us.” A third comment referenced why the client prefers occasional in-person communications: “[My son] likes to visit with his consultant sometimes in Starbucks where he likes to get hot chocolate.”

One parent commented on the bond that had developed between her child and home visitor, stating, “When you don’t have face-to-face, the quality is reduced. The bond is better with families when you are face-to-face.” Another echoed this sentiment, noting the change in their child with in-person visits: “Our son is more enthusiastic and engaged when meeting in person.” Finally, one program director noted that some staff felt that the switch to telehealth left them unable to read body language and other nonverbal cues as effectively as possible during in-person interactions and this was a disadvantage to using telehealth.

Finding Seven

Given the need to move to telehealth-based services in a one-week period, programs did not have time to search for best practice models for the provision of telehealth services. However, programs directors all increased the frequency of staff meetings for two reasons: to engage in discussions on how to make programmatic changes to their service delivery models using telehealth and to reduce feelings of isolation among staff.

The director of one program noted, “The heaviest lift was how to support staff. We already had the equipment. It was a huge change programmatically and for our clients.” One program increased their regular staff meeting from monthly to weekly during the initial months of the public health emergency and developed job aids for completing Individual Service Plans that were shared with other programs. Another director noted the importance of ensuring that staff remain connected: “Staying connected is critical. Staff have varying levels of stress; staff meetings keep [people] connected.” Finally, another noted that it was important to build confidence of both staff and clients: “Everything we do in home visiting is relationship based and reflective. We had to embrace the change and build staff confidence. Some staff were more comfortable with the technology than others.” The Center’s Home Visiting programs all increased the frequency of reflective supervision sessions with all staff.

Finding Eight

Finally, program leads and staff had to overcome a number of funding, administrative and regulatory barriers, including barriers to reimbursement from funding sources including Medicaid (Silva et al., 2015).

The state’s Human Services Department (HSD) and Center for Medicare and Medicaid Services did not allow Medicaid billing for telehealth services prior to the public health emergency. That barrier dissipated quickly when NM HSD almost immediately issued guidance to provider agencies that for the duration of the public health emergency, Medicaid could be billed for virtual and telehealth visits across programs. Additionally, the state agency overseeing NM’s Home Visiting programs also assured providers that telehealth “home” visits would be reimbursed, reversing the prior stance that virtual visits were seen as supplementary to in-person visits and therefore not funded. Providers were also assured by state and federal guidance related to the Health Insurance Portability and Accountability Act (HIPAA) compliance when transitioning to telehealth service delivery.

As all CDD direct service staff obtained individual Zoom accounts, funding was assured and many systems barriers were rapidly removed, the somewhat “clunky” need to obtain required “wet” signatures for service consents and other documents persisted as an inconvenience across Medicaid-funded programs. This is an issue that is still not completely resolved, but eventually, the NM Human Services Department sanctioned a number of innovative methods to overcome this barrier, including verbal consent and clients signing a document sent via email and returning a jpg file of the signature back to the program.

Privacy and client confidentiality represented a second area that continues to evolve across programs. Program staff have begun using headphones for telehealth meetings to maintain client privacy and confidentiality and programs have developed guidelines for staff and families related to boundaries around virtual visits. As programs at the CDD have refined their practices, staff and families now actively define the frame for a telehealth session – in an effort to further support both family and staff privacy.

Conclusions and Recommendations

The results of this preliminary evaluation of the UNM CDD’s direct service programs’ rapid pivot from in-person home- and community-based visits to virtual encounters provide evidence that telehealth is, in fact, a viable and even robust tool for delivering services across a number of programs and client populations. Prior pilot and feasibility studies examining the use of telehealth to provide direct services also demonstrated that under controlled conditions, using telehealth was a viable tool in lieu of in-person, face-to-face service delivery. By incorporating virtual visits and other technology-supported communication due to the COVID-19 public health emergency, direct service programs at the CDD were able to continue to provide uninterrupted and effective services for the great majority of clients across multiple programs. Clients and families expressed gratitude that, under the circumstances of a pandemic, their safety was prioritized, and services continued to be offered. The majority felt that using telehealth is effective, but many cited the loss of personal connection afforded by in-person contact as a clear downside and in some instances felt it negatively affected the quality of services provided. With that said, the majority of clients and families strongly supported the continuation of telehealth as a service delivery methodology post-pandemic.

Preparedness related to equipment, access to Zoom accounts and technology support was key in the success of all CDD programs effectively making the rapid shift to telehealth service delivery in spite of limited or no prior experience doing so. Initial barriers related to funding, privacy, and signatures for consent were quickly addressed and clients across all CDD programs have continued to have their service needs met. In addition, telehealth provided a means for clients and families to experience reduced isolation by remaining connected to their home visitors, nurse case managers and consultants during a highly stressful and unprecedented time. Telehealth also allowed programs to offer, when needed, more meeting time with clients in particular need, such as single parents.

For staff, increased meeting time and support was also critical to address and refine administrative, technological, logistical and practice elements of their work in real-time as they gained experience providing telehealth services. Opportunities for extra meeting time and reflective supervision also helped staff remain connected with each other and decrease their own sense of isolation.

While the recent gradual reduction in the number of cases in New Mexico and nationwide offers a glimmer of hope that the pandemic is easing, it is clear there will be no return to regular in-person, face-to-face services soon. While this is distressing to consider, it is positive to know that telehealth is a viable and effective option during this extraordinary time. At some point, the virus will be brought under control and in-person services will resume. In preparation for that time, the key question is this: how can the experiences gained during the public health emergency in providing services remotely be used moving forward? The ongoing willingness of funders, including Medicaid to continue to reimburse direct service programs for telehealth and virtual services will be a critical foundation as we anticipate a future in which direct service programs may offer a hybrid model. The ability, post-pandemic, to offer a combination of in-person and remote services, customized to fit the unique needs and preferences of families is recommended. In a rural state like New Mexico, in which staff regularly travel long distances for visits, this would increase the efficiency of programs and could even allow them to increase their caseloads or services.

Based on the experiences of our Center, four issues stand out that deserve attention as we contemplate a future service delivery platform using a combination of face-to-face and telehealth services. The first is defining and operationalizing practice elements for successful telehealth encounters. Many of the pilot and feasibility studies discussed earlier offer “lessons learned” about how to successfully conduct telehealth encounters taking both client and program staff needs into account. As mentioned earlier, some national programs such as NFP and PAT, have developed well-structured operating procedures for conducting home visiting via telehealth, which may be applicable beyond these particular programs. Articulation of “best practice” elements for “home and community” based virtual direct services is recommended as we move forward.

The second is the need to make ongoing and systematic professional development investments in program staff to support their skills and comfort in working with clients and families using telehealth and/or hybrid models. This might include developing standard training in telehealth for new staff as part of their onboarding and orientation processes, as well as ongoing professional development/training opportunities for existing staff. In addition, expansion of the use of reflective supervision models currently common to home visiting practice to other direct service programs may also be promising to support best practice in the use of telehealth for service delivery over time.

Third, the need to sustain ongoing investments in technology over time is critical. Prior to the COVID-19 Public Health Emergency, technology purchases at the Center were made on an ad-hoc basis based on individual program needs and the availability of financial resources. The reliance on well-functioning computer equipment and broadband, as well as the ready availability of technology platforms like Zoom to successfully conduct telehealth direct services, made it clear that a more systematic planning and acquisition process is needed within our Center to ensure that staff have adequate resources to support their virtual work.

Finally, the issue of limited broadband connectivity for some clients (and even some staff) must be acknowledged. There is no easy answer to this problem, as acquiring broadband access in rural and remote areas is a problem requiring a public-private investment strategy to build the technological infrastructure that would allow isolated residences to connect to broadband internet services. Finding a way to provide a family with a “hotspot” is insufficient if there is no broadband with which to connect.

Cases in which broadband internet is available but families do not have the resources to purchase technology to access it may be somewhat less formidable to address. Leadership at the CDD is exploring multiple approaches to access resources for clients and families. These include approaching funders for permission to use program funds to provide families with resources, applying for external grants to purchase technology to be given or loaned to families on a long-term basis, and using simple phone calls when Zoom access is not feasible for clients/families who may consistently or intermittently lack internet access. New Mexico state government and tribal governments across the state are using financial support provided through the CARES Act and subsequent legislation to significantly increase access to broadband services in rural and hard-to-reach areas of the state.

In summary, our preliminary findings suggest that telehealth is a viable, effective and appropriate strategy for delivery of direct services across a diverse set of home- and community-based programs serving a diverse range of clients and families. Initial client perception of telehealth was overwhelmingly positive. The majority of clients and families reported that the ongoing use of telehealth as part of a hybrid of in-person and virtual visits would be welcome – even after the pandemic is over.

Next Steps: Longitudinal Impact Evaluation and Comparison of Clients Receiving Services by Telehealth with Those Who Received “Hybrid” Services

Center Evaluation Services staff are now engaged in additional activities designed to address three questions.

1. What is the long-term impact of telehealth on substantive program indicators? Moving forward, evaluation of the long-term impact of telehealth services on substantive and individual program indicators for success will be needed. At present, both the NFP and PAT Home Visiting programs have national and state performance metrics that are embedded in the assumption that services are delivered in-person. Outcome data is collected for both programs at the state and national levels. We are now in the process of conducting a comparison of program outcomes for these programs, shedding light on whether telehealth services result in the same program outcomes as in-person home visits.
2. What are the medium- and long-term impacts? Given that the current findings only examine the initial response to telehealth direct services, we are unsure of the medium and long-term impacts of using telehealth across programs. The analysis on which findings are reported in this article was deliberately designed as a “rapid cycle” assessment that would capture information on program needs of staff, as well as the immediate impact on clients. We are now implementing a second wave of surveys of clients and interviews with program staff to gauge the extent to which perceptions of both program staff and clients may have changed over time and as the pandemic has continued.
3. How does the impact on clients who had telehealth services only vs. those who received “hybrid” services compare? The CDD direct service programs reported on in this paper have now enrolled two hundred and sixteen new clients since March 2020 when in-person, face-to-face services stopped due to the public health emergency. This will offer a unique opportunity to directly compare perceptions of those who have received both in-person and virtual services with those who have received only distance-based services. In the language of Don Campbell (Cook & Campbell, 1979), this is an example of a “naturally occurring experiment” that occurs when “…a particular intervention has been implemented but the circumstances surrounding the implementation are not under the control of researchers” (Craig et al., 2012, p. 1182).

We are in the process of preparing a new round of client-impact surveys. These new surveys will measure both the longitudinal impact of telehealth services for those clients who were receiving in-person services before the public health emergency as well as capture any differences in perceived impact between those clients and families who experienced a shift from in-person to telehealth services with those who have only experienced virtual service delivery.

References

Adler, G., Pritchett, L. R., Kauth, M. R., & Nadorff, D. (2015). A pilot project to improve access to telepsychotherapy at rural clinics. Telemedicine Journal and e-Health, 20(1), 83–85. doi: 10.1089/tmj.2013.0085

Backhaus, A., Agha, Z., Maglione, M. L., Repp, A., Ross, B., Zuest, D., Rice-Thorp, N. M., Lohr, J., & Thorp, S. R. (2012). Videoconferencing psychotherapy: A systematic review. Psychological Services, 9(2), 111–131. https://doi.org/10.1037/a0027924

Barak, A., Hen, L., Boniel-Nissim, M., & Shapira, N. (2018). A comprehensive review and a meta-analysis of the effectiveness of internet-based psychotherapeutic interventions. Journal of Technology in Human Services, 26(2), 109–160. doi: 10.1080/15228830802094429

Breitenstein, S. M., & Gross, D. (2013). Web-based delivery of a preventive parent training intervention: A feasibility study. Journal of Child and Adolescent Psychiatric Nursing, 26, 149–157. https://doi.org/10.1111/jcap.12031

Cherney, L. R., & van Vuuren, S. (2012). Telerehabilitation, virtual therapists, and acquired neurologic speech and language disorders. Seminars in Speech and Language, 33(3), 243-258. Doi: 10.1055/s-0032-1320044

Cook, T. D., & Campbell, D. T. (1979). Quasi-experimentation: Design and analysis for field settings. Rand McNally.

Craig, P., Cooper, C., Gunnell, D., Haw, S., Lawson, K., Macintyre, S., & Thompson, S. (2012). Using natural experiments to evaluate population health interventions: New medical research council guidance. Journal of Epidemiology and Community Health, 6, 1182–1186. doi: 10.1136/jech-2011-200375

Economic Development Department. (2021). New Mexico climate and geography. Retrieved from https://gonm.biz/why-new-mexico/climate-geography

Edwards, E. (2020). Keep essential care: Home and community-based care. Washington, DC: National Health Law Program. https://healthlaw.org/keep-essential-care-home-and-community-based-care/

Gleason, J., Ross, W., Fossi, A., Blonsky, H., Tobias, J., & Stephens, M. (2021). The devastating impact of Covid-19 on individuals with intellectual disabilities in the United States. New England Journal of Medicine: Catalyst. https://catalyst.nejm.org/doi/full/10.1056/CAT.21.0051

Hawkins, R. B., Charles, E. J., & Mehaffey, J. H. (2018). Socio-economic status and COVID-19–related cases and fatalities. Public Health, 189, 129-134.

Kahn, J. M., Cicero, B. D., Wallace, D. J., & Iwashya, T. J. (2014). Adoption of intensive care unit telemedicine in the United States. Critical Care Medicine, 42(2), 362-368. doi: 10.1097/CCM. 0b013e3182a6419f

Kidholm, K., Clemensen, J., Caffery, L. J., & Smith, A. C. (2018). The model for assessment of telemedicine (MAST): A scoping review of empirical studies. Journal of Telemedicine and Telecare, 23(9), 803-813.

LeRouge, C., & Garfield, M. J. (2013). Crossing the telemedicine chasm: Have the U.S. barriers to widespread adoption of telemedicine been significantly reduced? International Journal of Environmental Research and Public Health, 10(12), 6472–6484. doi: 10.3390/ijerph10126472

Mckay, D. (2020). One quarter of New Mexico students lack internet access. Governing: The Future of States and Localities. https://www.governing.com/community/One-Quarter-of-New-Mexico-Students-Lack-Internet-Access.html

Molfenter, T., Boyle, M., Holloway, D., & Zwick, J. (2015). Trends in telemedicine use in addiction treatment. Addiction Science and Clinical Practice, 10(14). doi: 10.1186/s13722-015-0035-4

Musumeci, M., & Chidambaram, P. (2020). Medicaid home- and community-based services enrollment and spending. San Francisco, CA: Kaiser Family Foundation. https://www.kff.org/report-section/medicaid-home-and-community-based-services-enrollment-and-spending-issue-brief/

National Home Visiting Resource Center. (2020). 2018 home visiting yearbook. Arlington, VA: National Home Visiting Resource Center. https://www.nhvrc.org/wp-content/uploads/NHVRC_Yearbook_2018_FINAL.pdf

New Mexico Center on Law and Poverty. (2018). Racial justice in New Mexico. http://nmpovertylaw.org/wp-content/uploads/2016/08/Report-Racial-Justice-Initiative-NMCLP-2014-05-30.pdf

New Mexico Voices for Children. (2021). 2020 Kids count data book report: Building on resilience. https://www.nmvoices.org/archives/15123

Nez, J. (2020, July). Addressing the urgent needs of our tribal communities. Testimony of Jonathan Nez, President of the Navajo Nation Before the United States House of Representatives Committee on Energy and Commerce, Congress of the United States https://docs.house.gov/meetings/IF/IF00/20200708/110874/HHRG-116-IF00-Wstate-NezJ-20200708.pdf

Owen, N. (2020). Feasibility and acceptability of using telehealth for early intervention parent counselling. Advances in Mental Health, 18(1), 39-49, https://www.tandfonline.com/doi/full/10.1080/18387357.2019.1679026

Richardson, L., & Simpson, S. (2015). The future of telemental health and psychology in Australia: Restoring the psychologically “clever country”? Australian Psychologist, 50, 307–310. https://doi.org/10.1111/ap.12134

Robben, S. H., Perry, M., Huisjes, M., van Nieuwenhuijzen, L., Schers, H. J., van Weel, C., Rikkert, M. G. M. O., van Achterberg, T., Heinen, M. M., & Melis, R. J. F. (2012). Implementation of an innovative web-based conference table for community dwelling frail older people, their informal caregivers and professionals: A process evaluation. BMC Health Services Research, 12(1). https://doi.org/10.1186/1472-6963-12-251

Sama-Miller, E., Akers, L., Mraz-Esposito, A., Zukiewicz, M., Avellar, S., Paulsell, D., & Del Grosso, P. (2018). Home visiting evidence of effectiveness review: Executive summary. Washington, DC: Office of Planning, Research and Evaluation, Administration of Children and Families, U.S. Department of Health and Human Services.

Silva, G. S., Farrell, S., Shandra, E., Viswanathan, A., & Schwamm, L. H. (2015). The status of telestroke in the United States: A survey of currently active stroke telemedicine programs. Stroke, 43(8), 2078–2085. https://doi.org/10.1161/STROKEAHA.111.645861

Sinclair, C., Holloway, K., Riley, G., & Auret, K. (2013). Online mental health resources in rural Australia: clinician perceptions of acceptability. Journal of Medical Internet Research, 15(9): e193. doi: 10.2196/jmir.2772

Stubbings, D., Rees, C., & Roberts, L. (2015). New avenues to facilitate engagement in psychotherapy: The use of videoconferencing and text-chat in a severe case of obsessive-compulsive disorder. Australian Psychologist, 50, 265–270. https://doi.org/10.1111/ap.12111

Traube, D. E., Hsiao, H. Y., Rau, A., Hunt-O’Brien, D., Lu, L., & Islam, N. (2020). Advancing home-based parenting programs through the use of telehealth technology. Journal of Child and Family Studies, 29(1), 44-53.

U.S. Census Bureau. (2021a). New Mexico state population, area and density. https://www.states101.com/populations/new-mexico

U.S. Census Bureau. (2021b). Quick facts: New Mexico. https://www.census.gov/quickfacts/NM

World Health Organization. (2018). Global diffusion of eHealth: Making universal health coverage achievable: Report of the third global survey on eHealt, 2018. Geneva, Switzerland: Author. https://www.who.int/goe/publications/global_diffusion/en/

Zambrano, L. D., Ellington, S., Strid, P., Galang, R. R., Oduyebo, T., Tong, V. T., & Zapata, L. (2020). Update: Characteristics of symptomatic women of reproductive age with laboratory-confirmed SARS-CoV-2 infection by pregnancy status—United States, January 22–October 3, 2020. Morbidity and Mortality Weekly Review, 69(44), 1641-1647. https://www.cdc.gov/mmwr/volumes/69/wr/mm6944e3.htm