INNOVATIVE APPROACHES such as distance treatment have been developed to overcome serious problems of access to mental health services in Canada. Service delays and untimely waitlists are widely reported by the Canadian government1,2 and researchers.3,4 There has been little research to determine the nature or the ability to develop therapeutic relationships using technology to deliver evidence-based treatment.
Although 18% of the pediatric population has a diagnosable mental health disorder, approximately 80% do not receive services.3 The current mental health service structure can be a barrier to timely access because often only severe cases receive specialist services. Children exhibiting mild to moderate symptoms are waitlisted, often for periods up to a year. Similarly, women suffering from mild to moderate postpartum depression symptoms are disadvantaged by limited access to services. Over time, untreated conditions can lead to exacerbation of symptoms causing physical/social impairment to the individual and chaos or marital difficulties within the family unit.5,6 Such healthcare disparity can be very frustrating for families who are in desperate need of services.
The cost (ie, transportation, meals, parking, child care, and missed work for one or both parents) and inconvenience of travel for face-to-face appointments that are typically scheduled during weekday hours can be especially burdensome for those living in rural or remote regions. Furthermore, the stigma associated with receiving mental health services often prevents families from maintaining scheduled appointments.7–10 These obstacles to access result in high attrition rates and ultimately poor health outcomes.
The demand for improved access to healthcare services has been the impetus for the development of new methods of healthcare delivery such as self-help style interventions that do not require face-to-face contact.11,12 Distance intervention programs could provide convenient access to families who have the right to receive timely healthcare services, a potential solution for waitlist issues. The effectiveness of distance delivery systems is uncertain because there is a question of whether a therapeutic alliance can be formed with the therapist, in the absence of visual contact.13 To facilitate the design and acceptance of new delivery systems, we must gain a better understanding of distance therapeutic alliance and factors that may influence the development of therapeutic relationships, in the absence of face-to-face contact.
Traditionally, therapeutic alliance has been used to describe the relationship between a therapist and adult client during face-to-face therapy.14 One of the most recognized theories about therapeutic alliance, developed by Bordin,14 identifies 3 elements (bond, goal agreement, and task agreement) that must mutually exist between client and therapist. A strong, positive client-therapist relationship and collective agreement to the tasks and goals of treatment are fundamental components of therapeutic alliance. Bordin suggests that the strength of the relationship greatly influences the therapeutic change experienced with therapy. In face-to-face therapy, therapeutic alliance has been shown to correlate highly with successful therapy.14–18 However, it is not clear whether Bordin's theory would be generalizable to distance therapeutic alliance constructs.
DISTANCE THERAPEUTIC ALLIANCE
Cook and Doyle13 studied (N = 15) treatment advice from therapists through electronic mail or chat for a variety of adult problems. The Working Alliance Inventory (WAI) scale, developed and validated for face-to-face treatment by Horvath,16,17 on the basis of Bordin's theory (bond, task agreement, and goal agreement) was used as a measure of therapeutic alliance. The results indicated that therapeutic alliance existed and when compared with normative data from the face-to-face literature, distance therapeutic alliance scores may have been possibly enhanced. Nine participants commented on their online experience. The themes that related to the distance relationship included Disinhibition (inapprehension about self-disclosure), Strength of relationship with therapist, and Convenience/flexibility of therapy. However, because of the small sample size, the study lacked statistical power limiting the generalizability of the findings.
Lingley-Pottie and McGrath19 studied the participants in the Family Help Research program to explore the distance therapeutic alliance. The Family Help Research program was designed to provide early intervention for mild and moderate diagnosable mental health problems. Evidence-based psychosocial and behavioral interventions are delivered to families and children in the comfort and privacy of their own home. Treatment includes written material, videos, and a nonprofessional telephone therapist called a coach.8 The WAI scale was administered to 64 adult participants at the end of treatment. The total WAI scores were generally high providing evidence that a therapeutic alliance can exist in distance treatment. The researchers compared the distance treatment mean WAI scores with normative data reported for face-to-face treatment and discovered that the scores were at least comparable to face-to-face treatment.
The purpose of this study was to explore the meaning of the distance therapeutic alliance and the distance treatment experience, described from the Family Help participant's perspective. The researchers hypothesized that the participant's distance alliance experience would reveal constructs that would be congruent to Bordin's theory.14 Furthermore, it was believed that new dimensions not adequately encompassed by the traditional therapeutic alliance definition would emerge.
The participants were adults receiving Family Help treatment for a diagnosable postpartum depression disorder or primary caregivers of children who were diagnosed with mild to moderate behavior disorder, pediatric anxiety, nocturnal enuresis, and recurrent headache and/or abdominal pain.
As described previously, the Family Help Research program teaches evidenced-based skills that are consistently implemented by the participants to enable them to overcome the problem.8,19 All participants were assigned a primary care telephone coach and received written material pertinent to the specific problem area (either in manual form or Web-based format), complimented by an educational video. The treatment program is focused on learning and effectively implementing new skills presented in the material, followed by a weekly telephone session with the coach who is a nonprofessional. Each coach received extensive training to effectively problem-solve, customize the program tasks to meet the specific needs of the family, and provide support during the treatment program. The coaches were trained and supervised weekly by a licensed healthcare professional. Depending on the problem area, the program consisted of 4 to 12 weekly sessions.
The participants resided within Nova Scotia's (Canada) district health authorities 4, 5, 6, and 9, many located in rural areas. Sample size was determined to ensure a variety of participant perspectives were included and redundancy achieved20,21 (ie, mode of treatment delivery; mental health problem area, treatment focused with child/parent, parent only, or direct treatment recipient; and coach/participant gender). Furthermore, inclusion of postpartum women would increase the generalizablility of the findings to unwell adults inflicted with a mental health disorder. Approval was obtained from the applicable research ethics committees.
The researchers created a questionnaire (Table 1) designed to explore the meaning of distance therapeutic alliance and the distance experience. It was telephone-administered at the end of treatment. Careful attention was made to include questions that pertain to the constructs of Bordin's theory (ie, bond, goal agreement, and task agreement). To ensure that the participants' responses were not influenced by wishing to please their coach,22,23 the questions were administered by a research assistant not involved with the participant's care. The participants were also informed that their responses would not be shared with their coach. If the participant had more than one coach during treatment, the questions were focused on the coach with whom the majority of the time was spent.
The open-ended questions were analyzed using content analysis and the remaining question responses were tallied. The primary researcher segmented the sentences reflecting different thoughts.21,23 A codebook was developed on the basis of emerging themes and included the main category name, subcategories and codes, a definition of the subcategory, inclusion and exclusion criteria, and exemplars.21 The codebook was pilot tested with 30 cases using 2 trained independent coders and yielded very good interrater reliability (κ = 0.78).21 Discrepancies were resolved with discussion and the codebook was revised. The final codebook consisted of 5 main categories (ie, program delivery system attributes, program content/design attributes, coach attributes, treatment goals, and hypothetical comments) with a total of 31 item sub- categories.
The full data set was released to the 2 coders. The data were entered into a SPSS database with the entries double-checked prior to running the analysis. Cohen's kappa test was performed to determine interrater reliability. Descriptive content analysis was performed to identify patterns and frequencies.23,24
The sample consisted of 131 participants (mean age = 35.5 years; SD = 4.76) who had completed Family Help treatment. Table 2 includes a demographic description of the study population. The majority of the participants (126) were primary caregivers (124 female, 2 male) whose children were diagnosed with psychological or behavioral disorders (ie, enuresis, recurrent headache and/or abdominal pain, anxiety, and behavior disorder). The other 5 participants were women receiving distance intervention for postpartum depression. Approximately 75.6% of the participants lived in rural areas of Nova Scotia, Canada.
Table 3 includes the closed questions results with accompanying examples of participants' responses. The majority of the participants reported that they agreed with their coach on the goals and tasks of treatment and would prefer the Family Help program if given the choice between face-to-face treatment and Family Help program. When asked about their coach's voice, most mentioned positive comments and believed it made a difference in helping them. Those who felt that voice did not make a difference commented that coach skill and personality were more important. More than half stated they preferred a female voice but almost half did not have a preference. Preference was not relative to the sex of their assigned coach.
The results of reliability testing for the open-ended questions are shown in Table 4. Given the large amount of coding categories and sizeable number of segments in the full data set, good interrater reliability was achieved (κ = 0.66–0.78). If a theme was repeated in the same participant's response, it was counted as one occurrence.
When asked to describe the distance relationship, the majority of the responses were very positive. Figure 1 shows the frequency of the categories reported by the participants. More than half of the participants commented on the quality or strength of the relationship. The majority commented about their coach's attributes (personal and/or technical skill) being components of the relationship. Almost half of the participants reported a sense of being uninhibited enabling them to disclose information freely to their coach. There were no negative comments regarding the formed relationship.
The following excerpts are examples of how participants described the relationship they formed with the coach in the absence of face-to-face contact.
Right off the bat I felt she was easy to talk to. Always professional but within a short amount of time I looked forward to her ideas and examples. I looked forward to talking to her. She was easy to talk to and helped me understand what she was trying to teach me. I just love her. I prolonged my meetings in the end cause I was scared of not talking to her again. I always had someone to talk to about things. She has given me confidence to make the decisions on my own. Even though I never got to meet her I feel like I know her.
Another informant describes her relationship with her coach by saying, “She cared about me. I felt she was like my sister. I didn't have to hold anything back.”
The first example includes all of the themes identified and describes how the coach skill empowered the mom to gain confidence in managing her child's behavior difficulties. Both statements provide a description of a strong and trusting relationship as evidenced by strong words of affection. Moreover, both exemplars denote high levels of self-disclosure.
The goals of therapy that Family Help participants were hoping to attain are illustrated in Figure 2. The most common theme expressed was the desire to gain control. One woman explained that she was: “Trying to learn how to deal with my son, how to handle everyday occasions and difficult times without being frustrated; without wanting to throw the towel in.”
Another parent responded, “How do I deal with the behaviours; for me to learn how to deal with it without physically grabbing a hold of him.”
Figures 3 and 4 depict the main themes identified by participants describing the advantages and disadvantages of the Family Help program versus face-to-face treatment. Although half of the participants reported no disadvantages of the Family Help program, most of them reported disadvantages of face-to-face including inaccessibility (eg, burden of travel, scheduling issues, taking kids out of school/time off work for sessions, and wait times); issues of stigmatization (eg, fear of being judged, forming an opinion on the basis of appearance, being identified, and labeled); apprehension to disclose information to the therapist (eg, intimidated, threatened, shy, and embarrassed); misinterpretation of body language; and cost burden. Fifteen percent reported lack of honesty or quality of information disclosed in the face-to-face setting indicating that it was “hard to admit things,” “less likely to be honest,” and “may tend to put a better spin on things”. Two participants described the disadvantages by stating: “On the phone you can develop your own mental picture of what someone is like. Meeting face-to-face there can be something about the person that you don't feel comfortable with and then the program wouldn't have been effective” and “The length of time to get in. Probably would have never gone further than my family doctor if I had to go face to face. I wouldn't have been honest face to face.”
Interestingly, the inverse of the themes identified as disadvantages of face-to-face treatment were reported as advantages of the Family Help program. Most of the participants reported that accessibility was an advantage of Family Help program as well as cost benefits and specific program design attributes (eg, over the phone, work at own pace, after hours staff availability, and 24/7 on-call services via toll-free line). Moreover, 41% of the participants reported that nonstigmatization or disinhibition (ability to openly disclose information to their coach) were advantages of receiving distance treatment. Analysis of the overall experience indicated that 63% of the participants made comments about feeling uninhibited and/or not judged or not stigmatized with distance treatment. Further analysis revealed that the participants who commented on the inapprehension/nonstigmatizing advantages of distance treatment were not necessarily the same individuals who commented on the apprehension/stigmatizing disadvantages of face-to-face, suggesting that the majority of participants shared concerns about the stigmatizing and inhibiting effects sometimes associated with receiving therapy.
Below are excerpts from a variety of respondents that describe the disinhibition and nonstigmatization distance experience:
I may not have spoken out if face-to-face. On the phone you can disclose a lot. I was in the comfort and privacy of my own home.
I didn't feel as pressured because not face-to-face. I was more willing to give an opinion; more comfortable because not looking at the person.
I felt ashamed before about myself and my parenting. It was easier to talk about this over the phone.
Can be when someone is in front of you, you can get nervous and not say what you are thinking, get shy or nervous.
Face-to-face may bias (body language etc), worry about opening up. On phone you have anonymity.
The majority of the comments about self-disclosure made reference to the openness and honesty of the communications between the parent and coach. Emphasis on the distance environment being less intimidating and a safe place ‘home turf’ where privacy was ensured emerged as a prominent advantage.
The main disadvantage of Family Help program reported was the inability to meet the coach. Of these, 70% (19/27) participants expressed a desire to meet their coach (related to the connection or bond they had made), wishing to put a face to the voice. Examples from male and female informants follow:
I didn't get to meet the person. I would have liked to have known who I was talking to but it's ok the way we did it.
I would have liked to have seen her … feel a little detached on the phone. After a long time the bond forms and it would be nice to have met her.
Potential for that personal involvement, a natural ‘want to see them’ but it doesn't outweigh the convenience of Family Help. The other person gets to share in the success and be able to thank them face-to-face.
The counselor doesn't get to meet the child so I sent a picture to ‘Coach’ of my son. I would prefer to have eye contact but if I had to choose what was available quicker and it was Family Help I would choose it all over again.
A few participants reported program design limitations as disadvantages of Family Help program such as inability to contact coach directly, assessments were too long, wished the program was longer, difficulty getting child to commit to the skills, and wanted the child or other parent more involved during sessions. Approximately 10% (10/13) participants made hypothetical statements about what others may think (ie, some people may find it impersonal) but of these, 7 qualified their response by stating it was not an issue for them.
The main advantage reported about face-to-face treatment was the ability to meet the therapist, however, many of the comments were focused on the desire to meet their Family Help coach. Of the 40 comments made, 26 wanted a visual of their coach and 13 made positive comments about Family Help. A few examples follow:
You get to meet the person, I would love to meet ‘Coach’ and give her a big hug.
“Nice to see what someone looks like” and “Put a face to the voice”.
One mom describes how she would like to meet her coach but feels she developed a mental picture of her:
Just, you would know the person you were working with by seeing them. I felt like I had seen ‘Coach’ because we worked so well together.
Overall, the thoughts and feelings described by this population were very positive about the distance relationship formed with the coach regardless of the age/sex of the participant, coach sex, or type/mode of treatment. One of the objectives of this study was to determine whether the emerging themes would be congruent with Bordin's theory (mutual bond, task agreement, and goal agreement).14 Although Bordin's theory was developed according to professional opinion and specific to face-to-face interactions, this distance treatment study suggests that strong bonds are certainly expressed by a majority of the participants. The results of this study strengthens the existing evidence reported by Lingley-Pottie and McGrath,19 that a therapeutic alliance can exist in the absence of face-to-face contact. Although goal and task agreement did not emerge as themes in the open-ended questions, when asked specifically, the majority of participants reported agreement with the coach on tasks/goals of Family Help treatment. Therefore, Bordin's theory is likely generalizable to distance treatment. However, other important constructs (ie, coach attributes; inapprehension for self-disclosure, and nonstigmatization) were identified through the participants' experiences that are not components of Bordin's therapeutic alliance definition. Consequently, Bordin's theoretical framework may not adequately measure the constructs present in distance treatment.
Therapist attributes such as personal traits (eg, honesty, trustworthiness, warmth, and empathy)25,26 and therapist skill26 have been reported as important factors for the development of a therapeutic bond in face-to-face contact. As shown in the results of this study, the participants report that these coach characteristics positively impact the distance relationship. Moreover, almost all of the participants described positive coach personal traits and voice quality as attributing factors.
Participants' inapprehension (ability to disclose information to coach) and Nonstigmatization as a result of visual anonymity emerged as prominent themes in this study but are not components of Bordin's theory. Other distance treatment studies13,27 have found that participants report the ability to express themselves openly in distance treatment communication. Few studies have examined the role of environmental elements that influence alliance development (such as the exchange of body language and facial expressions in face-to-face contact). In the face-to-face setting, it is unclear how nonverbal cues influence the therapeutic alliance or whether they are necessary. Some professionals believe that treatments with no face-to-face contact may negatively impact the development of a therapeutic alliance if the therapist and client are not able to exchange visual cues during therapy sessions.13 Conversely, as indicated by the results of this study, lack of face-to-face contact could reduce misinterpretation of body language or facial expressions that occur and are not clarified during direct therapy contact, leading to possible benefit or therapeutic gain if the client feels more comfortable in a distance setting.12,28 Furthermore, the privacy and visual anonymity that distance therapy has to offer may prove to strengthen the alliance if the client feels less inhibited and more comfortable disclosing personal information.13,28,29 “Anonymity or perceived anonymity may foster intimacy by increasing the amount of personal, self disclosure in friendships on the internet, where the fear of rejection that may prevent disclosure in face-to-face relationships does not exist.”13(p97)
Ben-Ze'ev30 suggests that interpersonal communication via the Internet offers increased privacy and decreased pressure from societal norms likely because of the sense of perceived anonymity. Joinson29 examined the role of visual anonymity on self-disclosure in computer-mediated communication and found a significant increase in self-disclosure during computer-mediated communication compared with face-to-face communication. Joinson29 and Yao and Flanagin31 explored the theory of private versus public self-awareness as factors influencing increased self-disclosure during computer-mediated interactions. The results of this study suggest that the visual anonymity provided by the distance treatment setting may increase the level and quality of self-disclosure as a result of being less intimidated/threatened or ashamed (decreased public awareness) and less self-conscious or more self-assured with increased autonomy (increased private awareness), which may lead to enhanced distance relationships. In the end, the main disadvantage of Family Help program reported was the desire to meet the coach (with whom a strong bond was formed) to dissolve visual anonymity by revealing identities.
The results of this study enhanced the credibility of the findings reported by Lingley-Pottie and McGrath.19 Through a triangulation research strategy with the earlier quantitative work by the authors,19 this qualitative study provides confirmation32,33 that a positive distance therapeutic alliance does exist from the participant's perspective and is congruent with Bordin's theory. As hypothesized, this study revealed new dimensions of distance therapeutic alliance not encompassed by Bordin's theory (ie, coach attributes [personal traits and skill]; inapprehension for self-disclosure; and nonstigmatization as a result of anonymity). However, since this study did not include a face-to-face treatment group, it is not clear whether these dimensions are unique to distance therapeutic alliance.
Perhaps therapeutic alliance is more complex than originally proposed by Bordin, given that the constructs of the theory were developed from professional opinion where adherence to treatment plan is important to successful therapy. Conceivably, most professionals would agree that therapist attributes (personal traits and skill), client self-disclosure, and client comfort with therapy are important aspects of therapy regardless if it is delivered face-to-face or from a distance. However, these factors are intrinsic to a client's perspective about therapy. Therefore, Bordin's theory of therapeutic alliance may be limited by definition because it is not grounded in the client's opinion and may not thoroughly explain the complexity of the therapeutic relationship dyad.
The dimension of the therapeutic alliance that is unique to distance treatment is visual anonymity. The absence of visual identity offered by the distance setting may cause the clients to feel less nervous about being judged, less intimidated by the therapist, and more comfortable in their home resulting in increased level of self-disclosure and truth telling. The influence that visual anonymity may have on self-disclosure may explain the enhanced distance therapeutic alliance scores reported by Lingley-Pottie and McGrath19 and Cook and Doyle.13
This study shows that participants not only embrace distance treatment as an acceptable access solution but also would select it again over a face-to-face system. The participants' positive distance treatment experience and expressions of strong coach relationships should be sufficient evidence to dismiss scepticism about this mode of healthcare delivery. Moreover, distance treatment modalities can offer the user visual anonymity not possible in traditional face-to-face therapy that may lead to increased treatment compliance and ultimately improved health outcomes.
Individuals with mild to moderate diagnosable mental health disorders should have the right to receive timely care to prevent the symptoms from becoming worse. New, effective distance intervention systems designed to be convenient and easily accessible to the user offer a cost-effective solution to the current wait-time issues. It is our hope that the results of this study will influence primary healthcare reform by facilitating the acceptance and uptake of distance delivery systems to resolve the disparity faced by those who are disadvantaged by limited healthcare access.
One limitation of this study was the lack of member checks performed with the participants to verify accurate interpretations by researchers. However, the purpose of this study was to begin to build a foundation for future qualitative phases.33 In addition, the study was not designed to allow for exploration of responses because no probing questions were implemented. Health outcomes were not available at the time of analysis because the original randomized trial was not completed.
Only 2 male primary caregivers and 5 unwell women with postpartum depression were part of this study population, limiting the generalizability of the results. However, the male responses did not differ from the female respondents. Similarly, the responses from the postpartum women did not indicate any differences in their distance treatment experience compared to the male/female parents. Since the study was not designed to gather the health status of the parents, we cannot make any assumptions of whether they were well or unwell. Given the commonality of the responses, there is some suggestion that distance treatment modalities may be applicable to both the well and unwell adults.
Finally, we had asked the participants to comment about possible limitations of a similar face-to-face treatment system that was not part of this study. Although their comments may be perceived as speculative, we believe there is merit in their experience as adults who have likely encountered limitations with some form of face-to-face healthcare service. Acknowledging their opinions and understanding their experiences may inform future system designs that will more appropriately meet the needs of society.
NURSING IMPLICATIONS AND FUTURE RESEARCH
Telemedicine modalities are becoming very popular modes of service delivery across disciplines, bridging the gap of limited access. Dissemination of distance treatment results to other healthcare professionals will foster collaborative relationships and lead to development of other innovative systems. Distance care delivery programs using trained, supervised nonprofessionals may assist with nursing shortages in some areas of primary healthcare services.
Future research should include distance intervention programs involving children to determine whether similar self-disclosure trends are evident when children interact with their coach in the absence of face-to-face contact. In addition, there is a need for developing sensitive and clinically valid measurement tools (ie, self-disclosure and therapeutic alliance) for use in distance intervention, grounded in the participants' experience. Perhaps the participants' reports about the distance therapeutic alliance may contribute to the overall understanding of the complexity of therapeutic alliance and prompt further exploration using well-designed, grounded theoretical research methodology.
1. Kirby MJ, LeBreton M. The health of Canadians-–the federal role. Final report, volume six: recommendations for reform. The Standing Senate Committee on Social Affairs, Science and Technology.
Ottawa, Ontario, Canada: Canadian Hospice Palliative Care Association; 2002.
2. Health Canada. 2003 First ministers' accord on health care renewal.http://www.hc-sc.gc.ca/hcs-sss/delivery-prestation/fptcollab/2003accord/index_e.html
. Accessed May 14, 2007.
3. Offord DR, Boyle MH, Szatmari P, et al. Six-month prevalence of disorder and rates of service utilization. Arch Gen Psychiatry
. 1987;44:832– 836.
4. Waddell C, Lavis JN, Abelson J, et al. Research use in children's mental health policy in Canada: maintaining vigilance amid ambiguity. Soc Sci Med.
5. Costello EJ, Egger H, Angold A. 10-year research update review: the epidemiology of child and adolescent psychiatric disorders: I. Methods and public health burden. J Am Acad Child Adolesc Psychiatry.
6. Offord DR, Bennett KJ. Conduct disorder: long-term outcomes and intervention effectiveness. J Am Acad Child Adolesc Psychiatry
7. Lipman EL, Boyle MH. Barriers to services promoting child emotional, behavioural, and social health. In: Encyclopedia on Early Childhood Development
[published online]. Centre of Excellence for Early Childhood Development; 2003.http://www.excellence-earlychildhood.ca/documents/Lipman-BoyleANGxp.pdf
. Accessed May 14, 2007
8. Lingley-Pottie P, Watters C, McGrath P, Janz T. Providing family help at home. Proceedings of the 38th Annual Hawaii International Conference on System Sciences.
Washington, DC: IEEE Computer Society; 2005:138a.
9. Cunningham CE, Boyle M, Offord D, et al. Tri ministry project: diagnostic and demographic correlates of school-based parenting course utilization. J Consult Clin Psychol
10. Cunningham CE, Bremner RB, Boyle M. Large group community-based parenting programs for families of preschoolers at risk for disruptive behaviour disorders: utilization, cost effectiveness, and outcome. J Child Psychol Psychiatry
. 1995;36:1141– 1159.
11. Newman MG, Erickson T, Przeworski A, Druz E. Self-help and minimal-contact therapies for anxiety disorders: is human contact necessary for therapeutic efficacy? J Clin Psychol.
12. Rochlen AB, Zack JS, Speyer C. Online therapy: review of relevant definitions, debates, and current empirical support. J Clin Pychol.
13. Cook JE, Doyle C. Working alliance in online therapy as compared to face-to-face therapy: preliminary results. Cyberpsychol Behav
14. Bordin ES. The generalizability of the psychoanalytic concept of the working alliance. Psychother Theory Res Pract
15. Marmar CR, Horowitz MJ, Weiss DS. The development of the therapeutic alliance rating system. In: Greenberg LS, Pinsof WM, eds. The Psychotherapeutic Process: A Research Handbook.
New York: Guilford Press; 1986:367–390.
16. Horvath AO, Greenberg LS. Development and validation of the working alliance inventory. J Counsel Psychol
17. Horvath AO, Symonds BD. Relation between working alliance and outcome in psychotherapy: a meta-analysis. J Counsel Psychol
18. Garcia JA, Weiss JR. When youth mental health care stops: therapeutic relationship problems and other reasons for ending youth outpatient treatment. J Consult Clin Psychol.
19. Lingley-Pottie P, McGrath P. A therapeutic alliance can exist without face-to- face contact. J Telemed Telecare
20. Lincoln YS, Guba, EG. Paradigmatic controversies, contradictions, and emerging confluences. In: Denzin NK, Lincoln YS, eds. Handbook of Qualitative Research
. Thousand Oaks, CA: Sage; 2000:163–188.
21. Ryan GW, Bernard HR. Data management and analysis methods. In: Denzin NK, Lincoln YS, eds. Handbook of Qualitative Research
. Thousand Oaks, CA: Sage; 2000:769–790.
22. Greenhalgh T, Taylor R. How to read a paper: papers that go beyond numbers (qualitative research). BMJ.
23. Murray M, Chamberlain K. Qualitative Health Psychology: Theories and Methods.
London, England: Sage; 1999.
24. Creswell JW. Qualitative Inquiry and Research Design. Choosing Among Five Traditions
. Thousand Oaks, CA: Sage; 1998.
25. Hersoug AG, Hoglend MD, Monsen JT, Havik OE. Quality of working alliance in psychotherapy: therapist variables and patient/therapist similarities as predictors. J Psychother Pract Res
26. Ackerman SJ, Hilsenroth MJ. A review of therapist characteristics and techniques positively impacting the therapeutic alliance. Clin Psychol Rev
27. Day SX, Schneider PL. Psychotherapy using distance technology: a comparison of face-to face, video, and audio treatment. J Counsel Psychol
28. Parks MR, Floyd K. Making friends in cyberspace. J Computer-mediated Commun
29. Joinson AN. Self-disclosure in computer-mediated communication: the role of self-awareness and visual anonymity. Eur J Soc Psychol
30. Ben-Ze'ev A. Privacy, emotional closeness, and openness in cyberspace. Comput Human Behav
31. Yao MZ, Flanagin AJ. A self-awareness approach to computer-mediated communication. Comput Human Behav
32. Maggs-Rapport F. Best research practice: in pursuit of methodological rigour. J Adv Nurs.
33. Mays N, Pope C. Education and debate. Qualitative research: rigour and qualitative research. BMJ.