Secondary Logo

Journal Logo

REVIEW OF LITERATURE

Assessment of Employability Skills: A Systematic Review of the Availability and Usage of Professional Behavior Assessment Instruments

McCallum, Christine A. PT, PhD; Murray, Leigh PT, PhD; Tilstra, Michele OTD, OTR/L, CHT; Lairson, Alexia PT, DPT

Author Information
Journal of Physical Therapy Education: September 2020 - Volume 34 - Issue 3 - p 252-263
doi: 10.1097/JTE.0000000000000152

Abstract

INTRODUCTION/LITERATURE REVIEW

Professionalism is a complex, multifaceted concept which has significant impact on health care practice and health care education.1-4 In practice, employers often report demonstration of professional behaviors (PBs) is valued more than specialized clinical knowledge or skill, regardless of job type.3 While employers often discuss the significance of PBs during work, attention to intentional and explicit development of the knowledge, skills and behaviors related to professionalism is often left to individual interpretation rather than explicit instruction.2,5-7

A plausible reason for the unintentional focus on a necessary part of today's workforce could be attributed to PBs as viewed as being “soft” and possibly, undervalued in academic settings, or it could be attributed to a lack of understanding about how to move an implicit component of a curriculum into the explicit, or intentional component of a curriculum. Dutton and Sellheim7 recognized student physical therapist (PT) PBs and professional development were influenced by opportunities and experiences encountered in and out of the classroom. The authors found development of PBs and attitudes is typically reflected through more casual interactions between faculty and students, or through unintended influence by organizational structure or programmatic culture.7 They concluded, however, development of PBs should be both an implicit and explicit focus for professional physical therapy programs.7 Recognition of influence in all aspects of professional education is a key first step into building a solid curriculum and development of skilled workers to enter our complex health care system.

The concept of professionalism in physical therapy practice and education is not new. The profession, through the American Physical Therapy Association (APTA), coupled with pioneer educators, such as May et al, have led the way in supporting PB development of students and practitioners.8,9 In 2003, in response to changing demands of the physical therapy profession and health care systems, the APTA adopted a vision to include development of professionalism as a key element in its strategic plan. As such, the adoption of core values in physical therapy formally occurred for the profession.8 Prior to this time, May et al first introduced the concept of “generic abilities,” as a requisite skill set of physical therapist graduates in 1995.9 The identification of 10 generic abilities and subsequent behavioral criteria, which provided specific standards for assessment, was novel and innovative. May's Generic Abilities, as it was commonly known, was the first time a skill set was set forth to complement the knowledge and technical psychomotor skills required of graduates prior to entering the profession.

In more recent times, the American Council of Academic Physical Therapy, in collaboration with the APTA, suggested the need for recommendations on best practice in education. An outcome of the 2014 Clinical Education Summit10 was a consensus based study to determine student readiness criteria prior to entry into the first full time clinical education (CE) experience. Through the delphi method, 14 key themes and 95 elements that address the knowledge, skill and attributes required of student PTs prior to full time CE were identified.11 Over 50% of these competencies pertain PB development, including skills related to communication, taking initiative and participating in reflective practice. This work provided a first step for academic programs to examine contemporary competencies required of students as they enter the world of work.

A shift is underway within health care education programs, in particular physical therapy education, to more explicitly address PB in the overall development of student PTs. Recently, a physical therapy professional call to action was made to change the term “soft skills” to describe PBs, to “employability skills,” to level the playing field between technical skills, and those behavioral skills needed to succeed in today's health care environment.12 Englehard quoted Tulgan13 to state “the hard skills get you hired, but it is the soft skills that will get you fired.” In essence, there is nothing soft about developing strong interpersonal skills, initiative, work ethic and professional communication-it takes hard work.

Unless specific, intentional focus within an academic program occurs, PB development will remain hidden or viewed as only part of the implicit curriculum.7,14 A move towards intentional development and assessment of student PT PBs is imperative to guide students in their development of a holistic professional. As such, we believe a need existed to identify PB assessment instruments (PBAIs) that could be used within academic training programs to complement knowledge and psychomotor skill development, and to describe methods of PB development during a professional PT education program, thus the purpose of this study.

METHODS

Study Design

A systematic review was conducted in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines. The protocol was registered on the PROSPERO website and can be found by using the Registration Number CRD42018081303 (Figure 1).

Figure 1.
Figure 1.:
Preferred Reporting Items for Systematic Reviews and Meta-Analyses Flow Diagram

Inclusion/Exclusion Criteria

Articles were eligible for this systematic review if they met the following inclusion criteria: 1) publication in peer-reviewed scholarly journals, 2) pertained to the rehabilitation sciences of physical therapy, occupational therapy, or speech-language pathology educational programs, and 3) assessed student PBs and professionalism. The professions of physical therapy, occupational therapy, and speech-language pathology were selected as the sole disciplines of focus because these professionals often work together in patient care, even though these discipline may award a degree other than a professional doctorate. It was also decided to include any multidisciplinary study in which student PTs were included.

We decided a priori to exclude the Clinical Performance Instrument (CPI), as it was purposefully designed and psychometrically assessed for reliability and validity in the aggregate.15 The CPI does include 5 elements related to PBs, however these elements cannot be separated for individual use due to design.

Information Sources/Search Strategy

The electronic databases searched were CINAHL, ERIC, ProQuest, and PubMed with publication dates of January 1995 to present, as the year 1995 marked the publication of May's Generic Abilities.9 MeSH terms and keywords included program evaluation, assessment, professionalism, PB, physical therapy student, occupational therapy student, and speech-language pathology student. The most recent search was completed on August 15, 2019. Refer to Appendix 1 (Supplemental Digital Content, http://links.lww.com/JOPTE/A91).

Study Selection

After removal of duplicate articles, 2 individuals reviewed titles and abstracts separately to determine relevance based on established criteria. Disagreements were resolved through consensus. The remaining articles were subjected to a full text review. Articles were again reviewed by 2 individuals using the same methods. Additional articles were identified by reviewing references from selected articles through a hand searching and grey literature searches using various resources such as ProQuest Dissertations & Theses, Grey Literature Report, and Google Scholar. Agreement between authors was calculated using an online generated Cohen's Unweighted Kappa for both title and abstract screening and full text screening.16 Level of agreement was then established using the guidelines: kappa value <0.00 = poor strength, 0.00–0.20 = slight, 0.21–0.40 = fair, 0.41–0.60 = moderate, 0.61–0.80 = substantial, and 0.81–1.00 = almost perfect agreement.17

Data Extraction

Two authors extracted all data from included articles and cross-checked to review accuracy. Data was selected based on relevance to the research question. Information extracted included: 1) study type, design, and purpose; 2) study characteristics such as sample size, institution, educational discipline, level of education, year in graduate program when applicable; 3) PBAI psychometric properties and component(s) of PB assessed 4) the intervention setting (classroom or clinic/community) and timing of the intervention and 5) recommendations for use of the tools within an academic program if applicable. We defined the PB assessment process as an educational intervention and the PBAI as the assessment tool used to evaluate student PBs.

Research Design and Methodological Quality Assessment

Each study was evaluated by 3 measures related to research parameters. First, studies were categorized by research type. The education research scale developed by the Institute of Education Sciences classified our studies.18 Second, the research paradigm categorized the studies as either quantitative, qualitative or mixed methods. Delphi method designs were also specifically highlighted. Finally, the methodological rigor/quality of the studies was determined using a modified McMaster Critical Appraisal tool, as developed by Lekkas et al.19 The 14-point binary scale offers a distributed quality measurement; studies were scored for adequate methodological rigor on 14 categories (1 for adequate rigor, 0 for inadequate rigor), for a potential high score of 14, and a low score of 0. Two authors independently assessed each study; disagreements were resolved through consultation.

Data Analysis and Synthesis

Each PBAI was assessed individually to determine: 1) reliability and validity and 2) the PB constructs assessed in the PBAI as identified by Jette and Portney,20 and 3) any processes documented that highlighted a PB advising process. We defined 7 PB constructs by synthesizing definitions and examples from the work of Jette and Portney20 coupled with the descriptors of the APTA Core Values.8 Refer to Table 1. Using the established categories and definitions, 2 authors independently assessed each PBAI to determine the relationship of the components of the PBAI to our established PB criteria. Disagreements were resolved through verbal discussion until consensus was reached.

Table 1. - Operational Definition of Professional Behavior Constructsa
Construct20 Meaning of Construct Explanation of Behaviors-Examples
Professionalism Professionalism requires active acceptance of the responsibility for the diverse roles, obligations, and actions of the therapist including self-regulation and other behaviors the positively influence patient/client outcomes, the profession and the health needs of society.
Professionalism also includes altruism, which is the primary regard for or devotion to the interest of patients/clients, thus assuming the fiduciary responsibility of placing the needs of the patient/client ahead of the therapist's self-interest.
The ability to be accountable for the outcomes of personal and professional actions and to follow through on commitments that encompass the profession within the scope of work, community, and social responsibilities.
Self-presentation, accountability, integrity, professional values (accountability, altruism, professionalism, responsibility).




Providing pro bono services to underserved and underrepresented populations.
Volunteer work, participation in professional organizations, mentoring, projecting a professional image
Critical thinking The ability to question logically; identify, generate and evaluate elements of logical argument; recognize and differentiate facts, appropriate or faulty inferences, and assumptions; and distinguish relevant from irrelevant information.
The ability to recognize and define problems, analyze data, develop and implement solutions, and evaluate outcomes.
Ability to recognize the need for information, to find and analyze information, manage decisions (critical thinking, problem solving).


Effective use of evidence-based practice
Professional development The ability to self-direct learning to include the identification of needs and sources of learning; and to continually seek and apply new knowledge, behaviors, and skills.
Requires the ability to practice consistently using current knowledge and theory while understanding personal limits, integrates judgement and the patient/client perspective, embraces advancement, challenges mediocrity, and works toward development of new knowledge.
Setting professional goals, assuming leadership role, self-assessment abilities (commitment to learning, excellence)
Communications management The ability to communicate effectively (i.e., verbal, non-verbal, reading, writing, and listening) for varied audiences and purposes.
The ability to manage time and resources effectively to obtain the maximum possible benefit.
Management of information, time and resources (communication skills, effective use of time and resources)
Personal balance Steadfast adherence to high ethical principles or professional standards; truthfulness, fairness, doing what you say you will do, and “speaking forth” about why you do what you do. Prioritizing and focusing on commitments, and effective stress management (integrity, stress management).
Abiding by legal and ethical considerations
Interpersonal skills The desire to identify with or sense something of another's experience; a precursor of caring; care is the concern, empathy, and consideration for the needs and values of others.
The ability to interact effectively with patients, families, colleagues, other health care professionals, and the community in a culturally aware manner.
The commitment to meeting one's obligations to provide effective therapy services to patients/clients, to serve the profession, and to positively influence the health of society.
Behaviors related to positive attitude, ability to motivate others, listening skills, demonstrating empathy (compassion and caring, interpersonal skills, professional duty)
Working relationships The ability to seek out and identify quality sources of feedback, reflect on and integrate the feedback, and provide meaningful feedback to others.
The promotion of a mutual trust between the profession and the larger public that necessitates responding to societal needs for health and wellness
Ability to give and receive feedback, demonstration of flexibility, work well with others (use of constructive feedback, social responsibility).
Adherence to legal and ethical regulations
aThe constructs of professional behavior are adapted from the Jette and Portney professional behavior model20 The meaning and explanation of the construct includes descriptors found in the APTA Core Values8 and May's Professional Behavior constructs,27 as well as sample behaviors and items within the Jette and Portney framework.

The additional data extracted from each study was analyzed individually and then in the aggregate. A thematic analysis followed, whereby the timing/indication for use of the PBAI, and recommendations for usage of PBAIs within a health education program were synthesized to determine themes or patterns of assessing PB performance of students. This information was utilized to develop a new conceptual framework for a professional advising plan.

Summary findings are presented in 3 parts. Part 1: Studies and quality assessment; Part 2: Summary of Professional Behavior Assessment Instruments; and Part 3: Recommendations for Professional Behavior Advising Plans.

RESULTS

Part 1: Selection of Studies and Quality Assessment

Selection of Studies

The search initially yielded 6,299 citations. After screening of title and abstract, 106 articles remained for full text review, yielding 10 articles for inclusion. Five additional articles were found through a gray/hand literature search process. The articles originated from Canada (n = 1), the United States (n = 13) and South Africa (n = 1). A total of 7 articles, identifying 5 PBAIs were generated from physical therapy literature, 5 from occupational therapy, and 3 from an interdisciplinary perspective (PT/occupational therapy [OT] and dentistry, medicine, pharmacy, dietetics, and physician assistant). One article was published in 1995, 6 articles between 2002 and 2007, and 8 articles between 2010 and 2019. Refer to Table 2.

Table 2. - Study Characteristics
Outcome Measure Tool Research Type (Ref) Research Paradigm Type of Assessment Sample Sample Size
Physical Therapy
 APBA; Dorsey et al30 Design and development; efficacy Quantitative Faculty Doctoral students 244
 CIET; Fitzgerald et al22 Design and development; effectiveness Mixed methods Clinician Masters and doctoral students Clinical faculty: 26; students: 1,509
 CPBDL; Lucy et al26 Design and development Qualitative (Delphi study) Self Physical therapy graduates; clinical instructor's/clinical coordinators; professional practice leaders; physical therapy faculty 9
 GABA; May et al9 Foundational Qualitative (Delphi study) Self/faculty Undergraduate students Not stated
 PBAT; May et al27 Design and development Qualitative (Delphi study) Self/faculty Doctoral students Not stated
 PTCVSA; Denton et al29 Efficacy Quantitative Self Physical therapy doctoral students First year: 50; second year: 46
 PTCVSA; Anderson and Hall28 Efficacy Quantitative Self Physical therapy doctoral students End of first year: 274
Occupational therapy
 U of I SOT PDT; Carroll et al21 Design and development; efficacy Mixed methods Self Physical therapy graduate students Pilot study: 18; reliability study: 58; focus group: 5
 OTAS; Hubbard et al32 Design and development; effectiveness Quantitative Clinician Occupational therapy undergraduate students; level II fieldwork Phase 1: 178; phase 2: 44
 PRFCL1; Koenig et al23 Design and development; effectiveness Mixed methods Clinician Occupational therapy and occupational therapy assistant undergraduate students; level I fieldwork Clinicians: 75; experts/focus group: 5/5; students: 317
 PDE; Randolph34 Design and development Qualitative Faculty Occupational therapy students (junior) Students: 32; faculty: not stated
 SPBQ; Yuen et al25 Design and development; efficacy Mixed methods Self Occupational therapy students Faculty: 3; first year students: 32; students: 718
Interdisciplinary
 ACCBE; Mpofu and Imalingat33 Design and development Qualitative Faculty Physical therapy faculty/post graduate students; occupational therapy faculty; nursing faculty; social work faculty; public health faculty; dean of faculty; inter-professional representative Faculty: 7; students: 2
 MMOS; Lie et al24 Design and development; efficacy Mixed methods Faculty Occupational therapy faculty; dentistry faculty; pharmacy faculty; physical assistant faculty Faculty (anchor dev): 3; faculty: 16
 PAT; Harris et al31 Efficacy Quantitative Self Athletic training students; occupational therapy students; physical therapy students; physician assistant students; respiratory therapy students Undergraduate athletic training and respiratory therapy students: 65; graduate level physical therapy, physician assistant, occupational therapy students: 115
Abbreviations: AGCBE = Assessment Guidelines for Community-Based Education; CIET = Clinical Internship Evaluation Tool; CPBDL = Comprehensive Professional Behaviours; Development Log; GABA = Generic Abilities Based Assessment; MMOS = Modified McMaster-Ottawa Scale; OTAS = Occupational Therapy Attribute Scale; PAT = Professional Assessment Tool; PBAT = Professional Behaviors Assessment Tool; PDE = Professional Development Evaluation; PRFCL1 = Philadelphia Region Fieldwork Consortium Level 1; PTCVSA = Physical Therapy Core Value Self-Assessment; SPBQ = Student Professional Behavior Questionnaire; U of I SOT PBT = University of Indianapolis School of Occupational Therapy Student Self-Assessment of Professional Behaviors Tool.

Interrater Reliability

Kappa values for study selection after title/abstract review was substantial with a value of 0.65 (95% confidence interval [CI], 0.57–0.73); and moderate for full text review with a value of 0.53 (95% CI, 0.37–0.60), respectively.

Design and Rigor of Studies

The studies were of variable research paradigms: 5 (33.3%) were mixed method21-25; 3 (20%) were qualitative Delphi studies9,26,27; 5 (33.3%) were quantitative28; and 2 (13.3%) were qualitative designs.33,34 The research type of the studies was also variable. Four studies (26.5%) were design or developmental studies26,27,33,34; 3 studies (20%) were efficacy studies,28,29,31 and 1 (7%) was foundational.9 Seven studies had dual purposes: 4 (26.5%) were categorized design/developmental studies coupled with efficacy of the tool21,24,25,30 while 3 (20%) were design/developmental studies coupled with assessing the effectiveness of the Professional Behaviors Assessment Tool (PBAT).22,23,32 Out of the 14 different PBAIs identified 5 (36%) were designed for student self-assessment,25,26,28,29,31 4 (29%) were utilized as a mechanism for faculty to assess student PBs24,30,33,34 while 3 (20%) were primary clinician assessments of student PBs.22,23,32 Two (13%) tools in physical therapy education were considered to be both student self-assessment as well as having the ability of the faculty to use as an assessment of PB.9,27 Refer to Table 2.

The critical appraisal scores for the studies ranged from 1 to 14. Risk of bias for methodological quality was categorized using a scaled approach defined as “high quality, low risk of bias,” “moderate quality, moderate risk of bias,” or “low quality, high risk of bias.” The top studies (n = 3) earned 13–14 points, the middle studies (n = 8) earned 10–12 points, and the lower studies (n = 4) garnered 0–9 points. This categorization has been used in previous systematic reviews19,35 and is able to define the studies that are low or high risk of bias in terms of methodological design quality. Refer to Table 3.

Table 3. - Critical Appraisal Scores-Methodological Quality; Alphabetized by Author
Author (Year) Critical Appraisal Scoresa Total Scoreb
1 2 3 4 5 6 7 8 9 10 11 12 13 14
Anderson and Hall (2018)28 X X X X X X X X X X X X X 13
Carroll et al (2002)21 X X X X X X X X X X X 11
Denton et al (2017)29 X X X X X X X X X X X 11
Dorsey et al (2018)30 X X X X X X X X X X X X 12
Fitzgerald et al (2007)22 X X X X X X X X X X X 12
Harris et al (2018)31 X X X X X X X X X X X X X X 14
Hubbard et al (2007)32 X X X X X X X X X X X X 12
Koenig et al (2002)23 X X X X X X X X X X 10
Lie et al (2015)24 X X X X X X X X X X 10
Lucy et al (2018)26 X X X X X X X X X X X X X 13
May et al (1995)9 X X X X X X 6
May et al (2010)27 X 1
Mpofu and Imalingat (2006)33 X X X X X X X X X 9
Randolph et al (2002)34 X X X X X X 6
Yuen et al (2016)25 X X X X X X X X X X X 11
aCritical Appraisal Category Scoring Key: 1 = Study purposes stated clearly (Quantitative and Qualitative); 2 = Relevant Literature reviewed (Quantitative and Qualitative); 3 = Sample described in detail (Quantitative)/Theoretical perspective identified (Qualitative); 4 = Sample size justified (Quantitative)/Purposeful sample selection described (Qualitative); 5 = Outcome measures reliable (Quantitative)/Sampling until redundancy in data reached (Qualitative); 6 = Outcome measures valid (Quantitative)/Informed consent obtained (Qualitative); 7 = Intervention described in detail (Quantitative)/Procedural rigor used in data collection (Qualitative); 8 = Contamination avoided (Quantitative)/Analytical preciseness (Qualitative); 9 = Co-intervention avoided (Quantitative)/Findings consistent with and reflective of data (Qualitative); 10 = Results reported in terms of statistical significance (Quantitative)/Auditability (decision trail developed and rules reported) (Qualitative); 11 = Analysis methods appropriate (Quantitative)/Transformation of data described (Qualitative); 12 = Educational importance reported (Quantitative)/Theoretical connections described (Qualitative); 13 = Drop-outs reported (Quantitative)/Trustworthiness (triangulation reported for methods) (Qualitative); 14 = Conclusions appropriate (Quantitative and Qualitative).
bTotal Score Key: 13–14 = High quality, low risk of bias; 10–12 = moderate quality, moderate risk of bias; <10 = low quality, high risk of bias.

Part 2: Professional Behavior Assessment Instruments

Refer to Table 4 (see Supplemental Digital Content 1, http://links.lww.com/JOPTE/A98) for synthesis of the PBAIs. Each PBAI is described below, categorized by discipline.

Physical Therapy (7 Articles, 5 Professional Behavior Assessment Instruments)

Academic Professional Behavior Assessment

The Academic Professional Behavior Assessment (APBA)30 was created by the University of Minnesota's physical therapy program. The tool was created using 5 of Jette and Portney's20 identified PB constructs. Face validity was assessed during development. Through an intentional advising process, the tool assisted in the identification and facilitation process of PB development for at risk students prior to full-time CE. Sixty-four percent of identified students successfully completed requirements for CE, while the other 36% required continued PB interventions throughout CE experiences. A positive predictive value of 0.97 indicated that students with scores greater than or equal to 4 (entry-level, demonstrate competency) in each area were 97% more likely to pass the clinical phase of the designated curriculum. The tool was able to correctly identify 71% of students who had difficulty with PBs in the clinic despite advising. There was a significant association (Χ2 = 47.40; P ≤ .0001) between APBA and clinical success.

Clinical Internship Evaluation Tool

The Clinical Internship Evaluation Tool (CIET)22 was created for initial use within the University of Pittsburgh's physical therapy programs full-time CE courses. The CIET consists of 2 distinct parts. Section 1 assesses 18 PBs items and Section 2 assesses 24 patient management items. The PB section was validated by factor analysis, whereby communication, initiative and PB items appeared to be distinct factors measured. Because the PB section did not result in a 1-factor model, total score correlations were not calculated. The CIET is a criterion-based evaluation, meaning all behaviors should always be met throughout a CE experience. The study does not describe whether the 2 sections could be used independently during classroom or part-time CE experiences. However, personal communication with an author stated the intent for the sections of the tool was to be used concurrently rather than individually (personal communication, L. Fitzgerald. August 15, 2019).

Comprehensive Professional Behaviours Development Log

The Comprehensive Professional Behaviours Development Log26 was initially developed by physical therapy faculty at The University of Western Ontario (Canada) in the early 2000s.36 A follow-up study26 updated and revised the tool to reflect current terminology and clinical practice. Additionally, it was assessed to be appropriate for use in other health professions, such as audiology. The log consists of 8 subsections that contain 111 items for student self-assessment: accountability (20 items), ethical/legal practice (10 items), evidence base practice (11 items), communication (17 items), empathy (12 items), client centered care (10 items), critical thinking (16 items) and upholding the profession (15 items). A 6-level scale was developed that ranges from no opportunity to consistently perform the intended behavior to always perform the intended behavior. Academic and clinical faculty may also use the log to assess student behaviors to guide professional development. The log was assessed to be valid through the Delphi method.

Generic Abilities Based Assessment/Professional Behaviors Assessment Tool

The Generic Abilities Based Assessment (GABA)9 tool was created by the University of Wisconsin-Madison's physical therapy faculty to 1) identify generic abilities necessary for clinical practice by physical therapy students and 2) to establish behavioral criteria by which to measure them during completion of a professional program. A Delphi method study identified 10 generic abilities and related behavioral based criteria that categorizes criterion into 3 levels of complexity: beginning, developing and advanced.

The PBAT27 updates and renames the GABA as described above. The PBAT is self-published and did not undergo peer review. This tool was included due to its common use within academic physical therapy programs. The revised PBAT identified and rank ordered PBs expected of new PT graduates in the 21st century. The 10 PBs were identical to the original generic abilities, however the rank order changed. This study revised the 10 PB definitions, behavioral criteria and created new categories for development, including beginning, intermediate, entry-level and post entry-level expectations. No data is provided on the specific research protocol or results used to develop the PBAT.

Physical Therapy Core Value Self-Assessment

Two studies examined test-retest and internal consistency reliability of the Physical Therapy Core Value Self-Assessment (PTCVSA).28,29 The PTCVSA was created by APTA staff to meet the need of academic educators to have a mechanism to assess PBs of students.

Denton et al29 was the first study to assess the reliability of the PTCVSA, knowing that the APTA core values themselves have not undergone a factor-analysis to establish construct validity. The authors assessed 2 different cohorts (first and second year doctor of physical therapy [DPT] students) of students in 1 academic year. Results demonstrated overall acceptable internal consistency reliability (ICC = 0.75). Test-retest values for all individual core values was marginal for first year DPT students (ICC = 0.62) and was acceptable for second year DPT (ICC = 0.80). These authors concluded the PTCVSA is not a reliable or valid measurement tool for students within their first year of a PT professional program due to the abstract nature of the constructs and limited context of beginner students. The authors also analyzed whether the tool measured the constructs of PB; results demonstrated excellent internal consistency (α = 0.96) for both cohorts of students.

Anderson and Hall's28 study analyzed students from 6 cohorts during the years of 2009–2014. They found the PTCVSA has excellent test-retest reliability (ICC = 0.90) and excellent internal consistency (α = 0.97). In addition to the assessment of reliability, the authors also analyzed the minimal detectable change for the PTCVSA. A student needed to demonstrate a 15-point change in score in order to have 95% CI it was a valid change not due to chance.

Both authors suggest the sample indicators of the PTCVSA may be seen as a checklist of rules to follow for novice professionals and could provide them with a framework of how to define and reflect on professionalism. The results do suggest the PTCVSA would be useful for more advanced students in a PT professional program and for professionals once licensed but should not be used as a high stakes assessment regarding readiness or progression in an entry-level program. The authors recommend further study for students across the continuum from first year to pending graduates as well as post-graduation.

Occupational Therapy (N = 5)

University of Indianapolis School of Occupational herapy Student Self-Assessment of Professional Behaviors Tool

The University of Indianapolis School of Occupational Therapy Student Self-Assessment of Professional Behaviors Tool21 was adopted by the Occupational Therapy Program for first- and second-year OT students to self-assess PB development. The instrument was initially developed by Anderson in 2000, however no reference was found to review its development. The tool has 2 subsections, which evaluates 37 behaviors in either the academic (classroom-first year) or clinic (second year). A 7-point Likert scale offers a broad, yet continuous description of the attributes valued by academic and clinical faculty. The model description did not offer any confirmation of quantitative test-retest reliability of the tool; however, they did report it was not significant overall. However, 31/37 individual items were significant as assessed within the academic environment, and 28/37 items were significant during the clinical second year. The timing of student assessment using the tool was shown to impact the overall validity of the instrument. The tool was useful to guide academic advisors in PB development plans for students.

Occupational Therapy Attribute Scale

The Occupational Therapy Attribute Scale (OTAS)32 was developed by clinical educators in South Texas as a supplement to the American Occupational Therapy Association's clinical outcome performance tool during Level II fieldwork experiences. Participants included OT students on level II fieldwork. Assessment of psychometric properties of the tool took place during 2 phases. The tool, a 43 item PBAI which uses a 5-point Likert scale, was evaluated for internal consistency, construct validity, and inter-rater reliability during phase 1, which included a factor analysis on all items of the tool. Before phase 2, the tool was revised based on the factor analysis and then assessed again for internal consistency, test-retest reliability, and predictive validity. Internal consistency was found to be strong in both phases. The tool demonstrated fair inter-rater reliability and strong test-retest reliability (r = 0.81). When analyzed to predict whether the student would be hired, the tool demonstrated 0.94 sensitivity. This suggests that the OTAS could positively predict students that would be hired. The OTAS is designed to be both a formative and summative assessment, in conjunction with the other clinical performance measurement tools.

Philadelphia Region Fieldwork Consortium Level 1 Student Evaluation

The Philadelphia Region Fieldwork Consortium Level 1 (PRFCL1)23 was developed by the Philadelphia occupational therapy consortium. Its purpose is to provide an efficient PBAI clinical educators could use to assess occupational therapy and occupational therapy assistant student PB development during level 1 fieldwork experiences. The 10-item scale was found to be valid (construct and discriminant) regarding age (older vs younger), degree (masters, bachelors, associates), and curricular sequence (first year and second year students). It was also found to have a high degree of internal consistency (α = 0.89) in measuring PBs during early CE experiences (level 1 fieldwork). Specifically, the PRFCL1 student evaluation is supported to have the capabilities of distinguishing students who may need more direct intervention to advance their professional development while having the ability to track progress over time. The PRFCL1 was used a summative assessment tool.

Professional Development Evaluation

The Professional Development Evaluation (PDE)34 was developed by the St. Louis University Occupational Therapy program to help faculty determine students at risk of displaying unsatisfactory PBs in the fieldwork setting. The PDE was adapted with permission from the Occupational Therapy Education Program at the Medical University of South Carolina. The PDE evaluates 28 classroom and laboratory behaviors related to 6 fieldwork and practice goals. Assessment is completed using a 0–5 Likert assessment scale.

Student Professional Behavior Questionnaire

The Student Professional Behavior Questionnaire (SPBQ)25 was developed by the Occupational Therapy program at the University of Alabama at Birmingham. The purpose was to provide a method for entry-level OT students to self-assess their behaviors during their academic training. The SPBQ is a 28-item tool with 3 focus areas: 10 items deal with learning tasks, such as personal responsibility; 9 items pertain to dealing with others, such as social tolerance and respect for others; and 9 items pertain to dealing with oneself. The outcomes of the study demonstrated the instrument has moderate internal consistency (α = 0.79) and has excellent construct validity (CFI = 0.96).

Interdisciplinary (N = 3)

Assessment Guidelines for Community-Based Education

The Assessment Guidelines for Community-Based Education33 tool was developed within the Faculty Community and Health Sciences at the University of Western Cape in South Africa. The tool was created in response to the need for an assessment tool that could be used across multiple disciplines for community-based learning activities. The draft tool was developed after review of 6 existing assessment instruments from the professional programs of physiotherapy, occupational therapy, social work, human ecology, dietetics and nursing. The 64 item PBAI contains facets of learning related to: knowledge (18 items), transferable skills (35 items), attitudes (4 items) and professionalism (6 items). The items are generic enough for use in an interdisciplinary or discipline specific learning experience. A 2-tiered 5-point Likert Scale was developed for either for earlier learners (third year or lower), or for more seasoned students (year 4 or beyond). Specifics pertaining to administration of the tool or any validity or reliability assessment were not documented.

Modified McMaster-Ottawa Scale

The Modified McMaster-Ottawa Scale (MMOS)24 was developed by faculty within the health sciences campus at the University of Southern California to address an interdisciplinary team based Objective Structured Clinical Examination. The authors developed standardized behavioral anchors to assess individual student and interprofessional team performance using a retooled McMaster Ottawa scale. Each competency is rated on an ordinal scale, ranging from “below expected” to “above expected.” The constructs assessed communication, collaboration, roles and responsibilities, collaborative patient centered approach, conflict management and team functioning. The investigators utilized trained students to demonstrate varying levels of performance which were assessed by faculty. Disciplines assessed included dentistry, medicine, occupational therapy, pharmacy, and physician assistant. Utilizing behavioral anchors, the faculty demonstrated the tendency to rate students higher despite them being lower performing students. The authors suggested using the MMOS when at least 2 trained faculty raters are assigned per station, assessing up to 4 student member interdisciplinary teams.

Professional Assessment Tool

The Professional Assessment Tool (PAT)31 was initially created for use with pharmacy students in 2009 by 7 universities. The tool was assessed to be reliable and valid for measuring behavioral professionalism that minimizes the ceiling effect within this profession.6 In 2016, the PAT was assessed by different researchers as a self-assessment instrument for measuring behavioral aspects of professionalism in the professions of athletic training, occupational therapy, physical therapy, physician assistant and respiratory therapy. The instrument is organized by 5 domains of professionalism: reliability, responsibility and accountability (5 items), lifelong learning and adaptability (7 items), relationships with others (9 items), upholding principles of integrity and respect (8 items), and citizenship and professional engagement (4 items). The results of this multi-disciplinary study indicate the tool has internal consistency (α = 0.90–0.95) in occupational and physical therapy students. The authors state the instrument could be used as “one component of a triangulated and longitudinal programmatic assessment plan” for the development of student PBs.

Overall Reliability and Validity

Refer to Table 5. Out of 15 studies, 6 (40%) reported no reliability or validity information9,24,26,27,33,34 and 2 (13%) reported on validity2,30; however, one of those studies did not provide any quantitative data for the section of their tool which assessed PB.22 Four (27%) reported on reliability21,28,29,31 and 3 (20%) reported on both reliability and validity,23,25,32 but all studies were in occupational therapy education programs. When analyzing reliability assessment, 6 (40%) reported on internal consistency,23,25,28,29,31,32 4 (27%) on test-retest21,28,29,32 and 2 (13%) on rater reliability.23,32 Even more limited was the reporting on validity; 1 (7%) reported content/discriminant validity,23 2 (13%) on predictive validity30,32 and 3 (20%) on construct validity.22,25,32

Table 5. - Reliability and Validity of Professional Behavior Assessment Instruments
Outcome Measure Tool Psychometric Property Assessed Values Strength Level Reported
Physical Therapy
 APBA; Dorsey et al30 Validity (predictive; specificity) PPV = 0.97 Those with score ≥4 in each category more likely to pass clinical phase of program (97%)
 CIET; Fitzgerald et al22 Validity (internal)
Validity (external)
Not calculated for PB section
Not calculated for PB section
N/A
N/A
 CPBDL; Lucy et al26 N/A N/A N/A
 GABA; May et al9 N/A N/A N/A
 PBAT; May et al27 N/A N/A N/A
 PTCVSA; Denton et al29 Reliability (test-retest)
Reliability (internal consistency)
ICC = 0.75 (overall)
ICC = 0.62 (1st years)
ICC = 0.80 (2nd years)
α = 0.96
Acceptable
Marginal
Acceptable
Excellent
 PTCVSA; Anderson and Hall28 Reliability (test-retest)
Reliability (internal consistency)
ICC = 0.90
α = 0.97
MDC = 15 points
Excellent
Excellent
Occupational therapy
 U of I SOT PDT; Carroll et al21 Reliability (test-retest) Overall # not reported Text reported overall “not significant”
 OTAS; Hubbard et al32 Reliability (internal consistency) α = 0.86–0.94 (phase 1) Strong
α = 0.91–0.96 (phase 2) Strong
Reliability (inter-rater) 21% Fair agreement
Reliability (test-retest) r = 0.81–0.82 Strong
Validity (external construct) r = 0.74 Strong
Validity (predictive; specificity) PPV = 0.94 Able to positively identify OT students who would be hired
 PRFCL1; Koenig et al23 Reliability (internal consistency) α = 0.89 “High degree””
Reliability (intra-rater) r = 0.88 Adequate
Validity (content/discriminant) Age, degree, curriculum sequence Able to discriminate among students re: 3 areas
 PDE; Randolph34 N/A N/A N/A
 SPBQ; Yuen et al25 Reliability (internal consistency) α = 0.79 Moderate
Validity (construct) CFI = 0.96 Excellent
Interdisciplinary
 ACCBE; Mpofu33 N/A N/A N/A
 MMOS; Lie et al24 N/A N/A N/A
 PAT; Harris et al31 Reliability (internal consistency) α = 0.90–0.95 (PT/OT) Excellent
Abbreviations: AGCBE = Assessment Guidelines for Community-Based Education; CIET = Clinical Internship Evaluation Tool; CPBDL = Comprehensive Professional Behaviours; Development Log; GABA = Generic Abilities Based Assessment; MMOS = Modified McMaster-Ottawa Scale; N/A = not applicable; OTAS = Occupational Therapy Attribute Scale; PAT = Professional Assessment Tool; PBAT = Professional Behaviors Assessment Tool; PDE = Professional Development Evaluation; PRFCL1 = Philadelphia Region Fieldwork Consortium Level 1; PTCVSA = Physical Therapy Core Value Self-Assessment; SPBQ = Student Professional Behavior Questionnaire; U of I SOT PBT = University of Indianapolis School of Occupational Therapy Student Self-Assessment of Professional Behaviors Tool.

Part 3: Descriptions of Professional Behavior Advising Plans

Five studies described the use of a PBAI as a summative and/or formative assessment tool for use during CE experiences21-23,28,32; while 2 studies described a PBAI appropriate for use in interdisciplinary learning experiences such as simulations24 or community-based experiences.33 Two tools were designed for use within a specific discipline, however could be adapted for use in other health professions.26,31 Four instruments were designed to complement other clinical performance-based assessment tools to further delineate PBs displayed during CE experiences.9,26,31,34 One tool was designed for use as a clinical outcome assessment tool.22 Six studies described a PB development9,21,26,27,29,34 or professional socialization30 advising process utilizing a PBAI as the basis of assessment.

Common themes among these PB advising plans included:

  1. PB development should be a purposeful, formative process included within an academic program that complements knowledge and skill acquisition.9,23,26,30,31,34
  2. Inclusion of a formal advising process aides in student development of PBs throughout a profession program.9,23,30,34
  3. Orientation to the values of PBs and clarity of expectations should occur early and often in an academic program.9,21,26,30,32,34
  4. Use of a PBAI helps faculty evaluate student readiness to enter CE experiences and assists in the identification of students who may be at risk for work-place performance issues.9,21,22,26,30,34
  5. PB assessments are based upon student self-assessment and/or faculty observation of student performance and behaviors in both classroom and clinical settings.9,21,26,28,30,34
  6. Students should be assessed by various faculty who interact with them during the didactic portion of their coursework to determine appropriate PBs prior to participation in CE experiences.30,34
  7. Student self-reflection is a required component of a PB improvement plan.9,21,26,28,30,34
  8. Development of a written PB improvement plan guides student development and/or remediation of PBs issues.23,30,34
  9. Feedback between faculty and student is a key factor in the advising process, especially when PB issues are identified.30,34
  10. The advising process includes student consequences for failure to meet PB performance expectations.30,34

DISCUSSION

The purpose of this systematic review was to identify PBAIs that assess student PBs in professional physical therapy programs. A secondary purpose was to identify commonalities in models used to develop PBs of students. Our results reveal PBAIs are available for use and foundational parameters for assessment of PBs in health professional programs were synthesized.

The studies revealed the availability of assessment instruments to measure PBs in both classroom and clinical settings. A strength of our study was the inclusion of PBAIs from multiple professions. While our intent was to identify PBAIs for use within professional physical therapy programs, we found instruments from occupational therapy literature and from an interdisciplinary perspective that we deemed beneficial to build the knowledge base about how PB assessment and development occurs in health professions. In the aggregate, these tools provide support for the constructs identified in Jette and Portney's20 model of PB, which aligns with the broad concept of “professionalism.”

We believe our study is the first to aggregate data from multiple disciplines regarding assessment tools available to assess student PBs. We found the purpose of PB assessments is threefold: 1) to identify areas of student strengths and areas of needed growth in relation to PB development, 2) to identify student readiness to enter CE including students at risk for PB issues during clinical practice (evaluative) and 3) to guide the development of an action plan (intervention) that could direct a student in his/her professional development process. In essence, intentionally focusing on PB development is useful as a diagnostic mechanism to guide development of intervention strategies for student improvement.

Development of affective behaviors related to professionalism is a responsibility of academic programs.9,11,20 Foord-May and May5 provided the profession of physical therapy with the theoretical foundation for the need of a systematic and formal facilitation process to guide student development of PBs, while others have done so for their respective disciplines.6,37,38 Our results revealed common themes for when or how the assessment of student PBs should occur, which supports May's facilitation process5. As such, we propose a revised conceptual framework for PB advising (Figure 2), developed through synthesis of procedures documented in our selected articles. This new conceptual framework is a contemporary creation generated from multi-disciplinary data including athletic training,31 occupational therapy,21,23,25,31,32,34 physical therapy,9,22,27-30 physician assistant31 and respiratory therapy31 and supports educational principles that state learning should be purposeful, intentional and integrated.5,7

Figure 2.
Figure 2.:
Conceptual Framework of a Professional Behavior Advising Process

Our PB advising framework highlights the need for intentional, circular assessment and feedback between academic/clinical faculty and students. Development of PB is not an innate process. The combination of utilizing a PBAI and a formal student-faculty advising process to guide the development of PBs was shown to be predictive of success in workplace responsibilities.30 We believe this is an area of education based research that would benefit from further study.

Overall, our results reveal limited strength of the psychometric properties for most PBAIs. The PBAIs found within the occupational therapy education profession appear to be more extensively studied about reliability and validity than those utilized within physical therapy educational programs. However, in physical therapy education programs, excellent internal consistency and test-retest reliability were demonstrated among several instruments.20,23,29,31,32 This confirms that these PBAIs are consistently measuring PBs. However, assessment instruments should not be considered valid until they are deemed reliable.39 Because these PBAI's have been shown to demonstrate instrument reliability, continued work to evaluate the accuracy and ability to assess the construct of PB is needed. Interdisciplinary tools as well as those identified within the occupational therapy educational programs also have the potential to be utilized or contribute to the development of reliable and valid measures of PB in all health-related professions.

We believe a plausible reason for the limited evidence on reliable and valid PBAIs could be that education research has not kept pace with clinically based research. Academic faculty often implement teaching and learning strategies without necessarily being cognizant of the evidence that is available. Educators should be more aware of the existing PBAIs and attempt to further validate and utilize them in student assessment of PBs. As educational research grows in the area of outcome assessment, a focus on the PBAIs in this review is needed rather than continuing to develop other tools that may not be accurate regarding psychometric properties.

As the consistency and accuracy of outcome tools for PB assessment grows, it is vital to identify key strategies and methods of purposeful intervention for students who are identified as having challenges related to PB. We believe our analysis of the process and the conceptual framework used to purposefully integrate a PB development program within the health professions is adequately supported by the literature, however further study about its impact is needed.

Limitations

While systematic methods were used to identify the selected studies, it is important to note study limitations. While the search strategy was comprehensive, it is plausible a published study was missed. A hand search of the literature in speech therapy was not conducted nor were any studies identified during the systematic search. It would be difficult to conclude that most identified PBAIs could be used across disciplines, although 3 tools were designed to be used by multiple disciplines24,31,34 and one made reference that it could be adapted by other disciplines.26 While these limitations exist, we believe the data generated was enough to make solid conclusions about the PBAIs and the need for a PB advisement processes in health profession education programs.

CONCLUSION

Development of PBs is a necessary component of contemporary physical therapy training programs. Professional behaviors are considered employability skills12 and are just as important to foster development as essential knowledge and technical skills. Our systematic review identifies assessment instruments that can guide programs in evaluation of student PBs during both classroom and clinical settings. We also outlined parameters to consider during the design and implementation of a PB advising model. Continued exploration of PB development would benefit health professions to ascertain sound psychometric properties of PBAIs, to study and refine processes to predict student performance related to PB development and to explore interprofessional PBAIs that could be used in both classroom and clinical settings.

ACKNOWLEDGMENT

The authors would like to thank Drs. Renee Linder, Hayley Stodtbeck and Jordan Marolt for their initial assistance on this project. Without their interest about the topic of student professional behavior development this work would not have been possible.

REFERENCES

1. Bossers A, Kernaghan J, Hodgins L, Merla L, O'Connor C, Van Kessel M. Defining and developing professionalism. Can J Occup Ther. 1999;66:16-21.
2. Davis DS. Teaching professionalism: A survey of physical therapy educators. J Allied Health. 2009;38:74-80.
3. Hayes K, Huber G, Rogers J, Sanders B. Behaviors that cause clinical instructors to question the clinical competence of physical therapist students. Phys Ther. 1999;79:653-667.
4. Tickle-Degnen L. Working well with others: The prediction of students' clinical performance. Am J Occup Ther. 1998;52:133-142.
5. Foord-May L, May W. Facilitating professionalism in physical therapy: Theoretical foundations for the facilitation process. J Phys Ther Educ. 2007;21:6-12.
6. Kelley KA, Stanke LD, Rabi SM, et al. Cross-validation of an instrument for measuring professionalism behaviors. Am J Pharm Educ. 2001;75:1-10.
7. Dutton LL, Selheim DO. The informal and hidden curriculum in physical therapist education. J Phys Ther Educ. 2014;28:50-63.
8. American Physical Therapy Association. Professionalism in Physical Therapy: Core Values. Alexandria, VA: American Physical Therapy Association. https://www.apta.org/uploadedFiles/APTAorg/About_Us/Policies/BOD/Judicial/ProfessionalisminPT.pdf. Updated December 14, 2009. Accessed December 6, 2019.
9. May W, Morgan B, Lemke J, et al. Model for ability-based assessment in physical therapy education. J Phys Ther Educ. 1995;9:3-6.
10. American Council of Academic Physical Therapy. Clinical Education Summit: Summit Report and Recommendations. Alexandria, VA: American Council of Academic Physical Therapy Website. http://www.acapt.org/docs/default-source/pdfs/clinical-education-summit-2014-final-report-1.pdf. Published August 2014. Accessed December 6, 2019.
11. Timmerberg JF, Dole R, Silberman N, et al. Physical therapist student readiness for entrance into the first full-time clinical experience: A delphi study. Phys Ther. 2019;99:131-146.
12. Englehard C. Shall we lead or follow: The elimination of “soft skills”. J Phys Ther Educ. 2019;33:85-86.
13. Tulgan B Bridging the Soft Skills Gap: How to Teach the Missing Basics to Today's Young Talent. Hoboken, NJ: Jossey-Bass; 2015.
14. Hafferty FW. Beyond curriculum reform: Confronting medicines hidden curriculum. Acad Med. 1998;73:403-407.
15. Roach KE, Frost JS, Francis NJ, et al. Validation of the revised physical therapist instrument (PT CPI): Version 2006. Phys Ther. 2011;92:416-428.
16. Lowery R. Kappa as a measure of concordance in categorical sorting. Vasar Stats website. http://vassarstats.net/kappa.html. Accessed December 6, 2019.
17. Landis JR, Koch G. The measurement of observer agreement for categorical data. Biometrics. 1977;33:159-174.
18. Institute of Education Sciences. Common Guidelines for Education Research and Development. Alexandria, VA: US Department of Education and the National Science Foundation. https://ies.ed.gov/pdf/CommonGuidelines.pdf. Published August 2013. Accessed December 6, 2019.
19. Lekkas P, Lasen T, Kumar S, et al. No model of clinical education for physiotherapy student is superior to another: A systematic review. J Physiother. 2007;53:19-28.
20. Jette DU, Portney LG. Construct validation of a model for professional behavior in physical therapist students. Phys Ther. 2003;83:432-443.
21. Carroll VA, Castro ER, DeFranco KM, Fazio RM, Williams LL, Barton RA. Test-retest reliability of a student self-assessment of professional behaviors. Occup Ther Health Care. 2002;16:1-19.
22. Fitzgerald L, Delitto A, Irrgang J. Validation of the clinical internship evaluation tool. Phys Ther. 2007;87:844-860.
23. Koenig K, Johnson C, Morano CK, Ducette JP. Development and validation of a professional behavior assessment. J Allied Health. 2003;32:86-91.
24. Lie D, May W, Richter-Lagha R, et al. Adapting the mcmaster-ottawa scale and developing behavioral anchors for assessing performance in an interprofessional team observed structured clinical encounter. Med Edu. 2014;20:26691.
25. Yuen HK, Azuero A, Lackey KW, Brown NS, Shrestha S. Construct validity test of evaluation tool for professional behaviors of entry level occupational therapy students in the United States. J Educ Eval Health Prof. 2016;13:22.
26. Lucy SD, Deluzio TDB, Bartlett DJ. Updating the comprehensive professional behaviours development log. J Allied Health. 2019;48:293-297.
27. May W, Kontney L, Iglarsh ZA. Professional Behaviors for the 21st Century 2009–2010. Marquette University website. https://www.marquette.edu/physical-therapy/documents/professional-behaviors.pdf. Published 2010. Accessed December 6, 2019.
28. Anderson DK, Hall KD. Reliability, internal consistency, and minimal detectable change of the American Physical Therapy Association professionalism in physical therapy: Core values self-assessment. J Phys Ther Educ. 2018;32:145-150.
29. Denton J, Fike DS, Walk M, Jackson C. Test-retest reliability of the APTA professionalism in physical therapy: Core values self-assessment tool in DPT students. J Phys Ther Educ. 2017;31:2-7.
30. Dorsey LL, Kelly PV, Luetkemeyer PB, Lojovich JM. Use of an academic professional behavior assessment and intervention to promote professional socialization. J Allied Health. 2018;47:210-216.
31. Harris LL, Kelley KA, Apke TL, Cleary D, Varekojis SL, White SE. Validation of an instrument for measuring professional behaviors across students enrolled in allied health professional programs. J Allied Health. 2018;47:190-195.
32. Hubbard S, Beck A, Stutz-Tanenbaum P, et al. Reliability and validity of the occupational attribute scale. J Allied Health. 2007;36:193-200.
33. Mpofu R, Imalingat A. The development of an instrument for assessing community-based education of undergraduate students of community and health sciences at the University of the Western Cape. Educ Health. 2006;19:166-178.
34. Randolph DS. Evaluating the professional behaviors of entry-level occupational therapy students. J Allied Health. 2003;32:116-121.
35. McCallum CA, Mosher PD, Jacobson PJ, Gallivan SP, Giuffre SM. Quality in physical therapist clinical education: A systematic review. Phys Ther. 2013;93:1298-1311.
36. Bartlett D, Deborah Lucy S. Item generation and pilot testing of the comprehensive professional behaviours development log. J Allied Health. 2006;35:89-9.
37. Brown D, Ferrill MJ. The taxonomy of professionalism: Reframing the academic pursuit of professional development. Am J Pharm Educ. 2009;73:1-10.
38. Seipel M, Johnson J, Walton E. Desired characteristics for MSW students and social work employees: Cognitive verses personal attributes. J Soc Work. 2001;47:445-461.
39. Wan TTH. Evidenced-Based Health Care Management. Multivariate Modeling Approaches. Norwell, MA: Kluwer Academic Publishers; 2002.
Keywords:

Professional behavior; Assessment instruments; Validity; Reliability; Advising; Conceptual framework

Supplemental Digital Content

© 2020 Academy of Physical Therapy Education, APTA