Across the spectrum of health and human services, professional case managers on transdisciplinary health care teams are responsible for care management, care coordination, and transitions of care activities. These responsibilities have raised the expectations that professional case managers undertake quality measurement and evaluation of the systems of care delivery and their impact on patient care outcomes and experience of care. The ability of the case manager to fulfill these demands underscores the importance of acquired credentials: educational background, certification, competency, and experience. To provide that assurance of advanced competency, which is vital for the protection of consumers, the Commission for Case Manager Certification (CCMC) offers the Certified Case Manager (CCM) credential. The credential is backed by a rigorous scientific research process conducted every 5 years, known as the role and function study.
As described previously in Part I of this two-part series (Tahan, Watson, & Sminkey, 2015), the CCMC conducts a role and function study on a regular basis to ensure that the CCM certification process and the content of the certification examination remain relevant to and substantiated by current practice, recognizing the increasing complexity of patients' needs across the health care continuum and the expanding dimensions of the transdisciplinary health care team. In this article, the results of the study are applied to the creation of the content and composition of the CCM certification examination.
The role and function study conducted in 2014 used the practice analysis survey method to describe case management practice in diverse settings and from the perspective of various professional disciplines involved. This was consistent with prior role and function studies conducted in 1994, 1999, 2004, and 2009. The practice analysis research method was used to analyze case management practice and delineate both (1) the general roles and functions of the case manager and (2) the related and necessary knowledge areas. On the basis of an extensive literature review, evaluation of prior survey instruments, input from subject matter experts representative of the current practice (professional background, practice setting, and geography), and input from a pilot survey review, a final study instrument was completed (Tahan et al., 2015).
Over a 4-week period of data collection in mid-2014, the survey was sent to a purposive sample of 52,370 potential participants, including both certified and noncertified case managers, from which 7,723 responses were received, and 7,668 responses were deemed appropriate for inclusion in the analyses. The 14.64% response rate was deemed sufficient as a representative sample of practicing case managers and to generalize the findings with a high degree of confidence and precision (99% confidence level and a 1.36 confidence interval). Table 1 includes some high-level characteristics of the study sample; for more details, refer to Part I of the two-part article series. The role and functions study addressed three main research questions (identical with prior years):
- What are the essential activities/domains of practice of case managers?
- What are the knowledge areas necessary for effective case management practice?
- Is there a need to revise the blueprint of the CCM certification examination? And if so, what modifications are warranted?
Responses related to the first two questions are detailed in Part I (Tahan et al., 2015). This article addresses the third question.
Test Specifications of the CCM Certification Examination
At the conclusion of the 2014 role and function study, a final report of results was produced by the researchers, with additional information to guide the development activities for the CCM certification examination blueprint. The report provided in-depth information including sample/characteristics/demographics, mean importance and frequency ratings by item, subgroup analyses using index of agreement (IOA) and factor analysis. It also included pertinent information about the case managers who participated in the survey and their responses about essential activities and knowledge areas related to the practice of case management (Tahan et al., 2015).
Experts refer to the process of developing a certification examination blueprint as test specification. Usually a team of subject matter experts meets and reviews the findings of the practice analysis under the guidance of the researchers and carefully decides on the content areas to guide the certification examination. The researchers selected 11 members for the test specification committee in consultation with the CCMC, using a set of criteria similar to that applied in the selection of the subject matter experts convened for the study instrument development. Selection criteria emphasized relevant diversity, including practice settings, years holding the CCM certification, noncertified case managers, practicing case managers, work settings, practice specialization, professional backgrounds, and geographic location. The subject matter experts consisted of case managers with nursing, social work, vocational rehabilitation, disability management, professional counseling, or workers' compensation backgrounds. They came from various geographic locations across the United States, and worked in settings across the continuum of health and human services delivery (e.g., preacute, acute, and postacute), health insurance plans, workers' compensation, and private/independent practice. Some of the subject matter experts from the instrument development taskforce also participated in the test specification committee. This was important to maintain continuity of the work, while also seeking input from new experts who had not contributed to any aspect of the role and function study before the test specification meeting.
To develop test specifications for the CCM certification examination, the researchers facilitated a meeting of the 11 subject matter experts in August 2014. They met in-person over a 2-day period to finalize which essential activity and knowledge statements would be accepted for inclusion in the test specification and to determine the weights of the knowledge content domains in the certification examination. The subject matter experts reviewed the following:
- The report of the study findings that included details about the mean importance and frequency ratings and standard deviations of each of the 125 essential activity and 94 knowledge statements.
- The study participants' ratings of the comprehensiveness of the study instrument in each of the essential activity and knowledge domains.
- The results of the subgroup analyses demonstrated by the reported IOAs.
These reviews were necessary for the subject matter experts to confidently determine whether a statement should be included in the test specifications and why. The subject matter experts reviewed the results related to each item, determining that statements with mean importance ratings at or above 2.5 were appropriate for inclusion in the test specifications. However, for those with lower than 2.5 ratings (i.e., those rated “slightly below important” or “of no importance”), the subject matter experts needed to deliberate to reach consensus whether to include in the test specifications or reject completely. The use of a cut-point value for accepting or rejecting a statement set at 2.50, which is the midpoint between moderately important and important ratings, was consistent with past studies and conformed to practice analysis research standards (Tahan, Huber, & Downey, 2006). See Part I of this two-part article series (Tahan et al., 2015; see Tables 2 and 3Tables 2 and 3) for the detailed designations of “pass” and “fail” for each item.
Table 2 in this article summarizes the results of the test specification review. Out of a total of 125 essential activity statements, 115 were included in the test specifications. Of these, 106 statements passed the mean importance rating test and 9 failing statements were deemed necessary for inclusion on the test specifications by the subject matter experts. Of these 9, two statements pertained to involvement in activities of denials, appeals, and collaboration with physician advisors. These two statements reflected a main importance rating of 2.37 and 2.48, which are close to the required 2.5 rating. Five statements belonged in the vocational and rehabilitation services domain, covering activities such as arranging assessment for rehabilitation services, identification of services to achieve wellness and optimal functioning, recommendation for interventions on the basis of workers' compensation and disability management guidelines, and coordination of specialized rehabilitation services such as assistive devices. The mean importance ratings of these statements were 2.47, 2.40, 2.44, 1.99 and 2.14, respectively. The remaining two statements were in the outcomes evaluation and case closure domain and focused on collection of health care organization-related outcomes data and analysis of client and health care organization outcomes data. Their demonstrated mean importance ratings were 2.40 and 2.32, respectively.
Of the total 94 knowledge statements, 81 were included in the test specifications, of which 68 statements passed the mean importance rating test and 13 failing statements were deemed necessary for inclusion in the test specifications by the subject matter experts. Of those included despite their low mean importance ratings, two belonged to the case management concepts and strategies domain and addressed case load calculation and program evaluation and research methods. Their demonstrated mean importance ratings were 2.37 and 2.44, respectively, which were close to the desirable 2.5. Another two statements pertained to the health care management and delivery domain, focused on health care analytics such as risk assessment and stratification, and new models of care such as patient-centered medical home and accountable care organizations. These statements revealed mean importance ratings of 2.24 and 2.47, respectively. In the health care reimbursement domain, three items were deemed important by the test specification committee despite having mean importance ratings being below 2.5. These statements addressed the areas of financial resources, military benefit programs, and new reimbursement and payment methodologies such as bundled payment and value-based purchasing; their mean importance ratings were above the moderately important rating of 2.0 at 2.14, 2.02, and 2.20, respectively.
Finally, five failed statements still deemed important by the test specification committee belonged to the rehabilitation and vocational concepts and strategies domain. These statements focused on knowledge of assistive devices, functional capacity evaluation, physical functioning and behavioral health assessment, rehabilitation postinjury or acute hospitalization, and vocational aspects of chronic illness and disability. Mean importance ratings were 2.24, 2.08, 2.48, 2.33, and 1.95, respectively. With the exception of the last statement, all rated above the moderately important rating of 2.0.
Essential activities and knowledge areas related to the vocational and rehabilitation services have been reviewed and analyzed on an ongoing basis since the 2009 role and function study. At that time, after careful examination and consideration, it was determined that the general practice of case management includes rehabilitation, not necessarily limited to vocational rehabilitation and counseling case managers/professionals or settings. It was also recognized that the degree of involvement in rehabilitation-type activities and use of rehabilitation-related knowledge varied on the basis of the professional background, specialization, and work setting of the case manager. As a result, after the 2009 study, the test specification committee updated the vocational rehabilitation domain to reflect the broader rehabilitation services and agreed to continue to carefully examine this domain going forward (Tahan & Campagna, 2010). In the 2014 role and function study, the subject matter experts who participated in the survey instrument development and were also practicing case managers debated the inclusion of survey items pertaining to vocational and medical rehabilitation. They concluded that it was important to have these areas covered on the survey.
In addition, during the test specification committee sessions, subject matter experts, who also were practicing case managers, carefully and thoughtfully reviewed the results of the activity and knowledge statements. They debated the below-2.5 mean importance ratings issue and agreed unanimously that case managers are involved in rehabilitation-related activities and that they apply knowledge of rehabilitation (vocational, medical, and physical functioning) in their practice. In fact, they noted that case managers regardless of practice setting must have general knowledge of rehabilitation to be able to identify the client who would benefit from rehabilitation services, and to assure that referrals are completed in a timely fashion and recommendations for rehabilitation services are incorporated into the client's plan of care. This was necessary for protecting clients' safety and to ensure quality care.
As summarized in Part I of this two-part article series, the researchers shared with the test specification committee the findings of the subgroup analyses using the IOA test statistic. The use of the IOA was essential to determine how similar or different the perceptions of the various participants (subgroups) were relevant to their importance ratings of the essential activities and knowledge areas. Mean importance ratings of items at or above 2.50 indicated an agreement that the content is important; in contrast those rated less than 2.50 indicated an agreement that the content was less important. Any differences in mean importance ratings among subgroups indicated that there was disagreement as to whether the content is important. The IOA computed scores usually range between 0 and 1, with 1 being perfect agreement and 0 being perfect disagreement. IOAs greater than or equal to 0.80 but less than 1.00 meant high agreement; less than 0.80 but greater than or equal to 0.70 indicated moderate agreement; and IOAs less than 0.70 meant disagreement existed among the subgroups' perceptions. A summary of the IOA ranges for essential activities by participant subgroups is as follows (detailed results are included in Part I [Tahan et al., 2015]):
- Job title: 0.20–0.97
- Percentage of time in direct case management services: 0.22–1.00
- Work/practice setting: 0.12–0.96
- Years of experience in case management: 0.84–0.99
- Requirement of work on weekends: 0.95–0.97
- Professional background/discipline: 0.63–0.97
- Presence of CCM certification: 0.93
- Geographic region: 0.93–1.00
- Highest academic degree achieved: 0.93–0.99
- Age: 0.84–1.00
- Sex: 0.95
- Ethnicity: 0.92–1.00
Because test specifications focus more on the knowledge areas than on the essential activities, it was important for members of the test specification committee to critically review the results of the subgroup IOAs for the knowledge areas before final decisions were made about the new blueprint for the CCM. This is because certification examinations usually test for knowledge of practice rather than the frequency (quantity) of the activities of practice. The IOA ranges for knowledge areas (see Table 3) by participant subgroups were as follows:
- Job title: 0.34–0.98
- Percentage of time in direct case management services: 0.46–0.99
- Work/practice setting: 0.36–0.97
- Years of experience in case management: 0.67–1.00
- Requirement of work on weekends: 0.87–0.91
- Professional background/discipline: 0.50–0.90
- Presence of CCM certification: 0.89
- Geographic region: 0.85–0.98
- Highest academic degree achieved: 0.85–0.97
- Age: 0.84–0.98
- Sex: 0.91
- Ethnicity: 0.86–0.99
As was stated in Part I (Tahan et al., 2015), the subgroup IOA analyses that did not show high agreements among the subgroups in both activities and knowledge areas included job titles, practice settings, and professional background. Two other subgroups in the knowledge areas analyses also showed IOAs less than 0.80; they were years of experience and percentage of time spent in direct case management. These subgroups demonstrated some varied degrees of agreement and disagreement. The takeaways from this analysis here will focus on those related to the knowledge areas.
When comparing the central job title subgroup of case/care manager against the other 15 job title subgroups (see Table 3), varied levels of disagreement are noted to exist with consultant, admission liaison, disability manager, insurance benefit manager, workers' compensation, and quality specialist titles. The lower levels of agreement observed may be attributed to small subgroup size, highly specialized practice, or being removed from direct case management service provision, such as with quality specialist and insurance benefit manager titles.
Concerning the primary work/practice settings, if the health insurance subgroup is considered as the central subgroup for the comparative analysis (it being the largest of the subgroups), among the 15 subgroups the vast majority of the IOAs in the knowledge areas analyses were above 0.80, except for the liability and disability insurer subgroup, which showed an IOA of 0.36. Other low IOAs were for the wellness subgroup at 0.72, the independent case management subgroup at IOA 0.77, and independent rehabilitation company and third-party administrator subgroups at 0.78 each.
Upon examination of the professional background subgroups, the disagreements were prominent in the subgroup with rehabilitation backgrounds (i.e., physical therapy, disability manager, and vocational rehabilitation). The IOAs for knowledge areas ranged between 0.50 and 0.90. This is no surprise; it is likely related to the below-acceptable mean importance ratings noted in the vocational and rehabilitation domains of survey knowledge statements. For percentage of time spent in provision of direct case management services, the subgroup that demonstrated disagreement was the no (or 0%) direct involvement subgroup, with IOAs ranging between 0.46 and 0.61 for the knowledge areas. Interestingly, however, eight IOAs were above 0.50, implying a about 50–50 agreement/disagreement between the 0% direct care subgroup and the others. This demonstrates that despite the lack of involvement in provision of direct case management services, this subgroup still agreed 50% of the time with the other subgroups on what knowledge areas were important for the practice.
Subgroup analyses on the basis of years of experience demonstrated acceptable to perfect IOAs in knowledge areas (0.67–1.00), except for the subgroup of less than 1 year of experience. This subgroup had an IOA of 0.67 when compared against the subgroup with 1–2 years of experience. This is likely attributable to being new to case management practice.
Factor/Principal Component Analysis
Analyses of the 2014 role and function study findings also included a factor analysis performed by the researchers to examine the validity and appropriateness of the theoretical domains that composed the case manager role and function study instrument. This analysis is an integral step in the test specification work to inform the content and construct of the CCM certification examination. Factor analysis, also referred to as domain analysis or principal component analysis, is a statistical method designed to reduce data or categorize variables (data) into thematic components (e.g., domains, subject areas, and content areas). This analysis applies the mean importance ratings results into a mathematical test to produce clusters of statements that, when examined carefully, possess similar characteristics and allow higher-level abstractions. This involves clustering micro and unique case management activities and knowledge topics into higher-order functions or knowledge areas.
The researchers tested the appropriateness of the six theoretical activity and six theoretical knowledge domains used in the study instrument development. This process is known as theoretical or forced factor analysis. However, the results were not favorable. Therefore, the researchers then pursued the exploratory factor analysis method whereby all activity statements were combined as one single section and the knowledge statements as another single section, and based on statistical analyses the system then mathematically clustered the statements into groups.
To complete the exploratory factor analysis, the researchers tested a number of different factor solutions (two, three, four, five, and six components). This test ultimately produced an acceptable six-factor solution for the essential activity domains (see Table 4) and five-factor solution for the knowledge domains (see Table 5). Table 6 summarizes the results of the factor analysis and the number of statements included in each factor with their associated Cronbach α computations. Specification of where each of the statements belonged in the factors was based on the exploratory factor analysis results. Notably, statements that were rejected from inclusion in the test specification process because of their being of low or no importance were excluded from the factor analysis as they should have.
Once the factor analysis was completed, the researchers conducted a reliability analysis using Cronbach α (see Table 6), which is a measure of internal consistency and homogeneity of the factor. Internal consistency determines whether several variables are measuring the same construct. The higher Cronbach α is, the more likely the variables are measuring the same construct. Experts have stated that Cronbach α values greater than 0.70 are desirable.
Cronbach α computations ranged between 0.79 and 0.99 for the essential activity domains (factors based on the exploratory factor analysis) and 0.86 and 0.97 for the knowledge domains. Overall, Cronbach α values for activity and knowledge domains were 0.99 and 0.98, respectively—highly acceptable results. The test specification committee then reviewed the results and accepted the domains. Each domain was then named on the basis of the themes covered by the statements included in the domain, as follows:
New Essential Activities Domains
- Delivering case management services
- Accessing financial and community resources
- Delivering rehabilitation services
- Managing utilization of health care services
- Evaluating and measuring quality and outcomes
- Adhering to ethical, legal, and practice standards
New Knowledge Domains
- Care delivery and reimbursement methods
- Psychosocial concepts and support systems
- Rehabilitation concepts and strategies
- Quality and outcomes evaluation and measurements
- Ethical, legal, and practice standards
Test Specifications of the CCM Certification Examination
After inclusion decisions and factor analysis results were finalized, each subject matter expert on the test specification committee was asked to complete an anonymous weighting sheet to assign a percentage (out of 100) for each of the five new knowledge domains. The new domains would become the CCM certification examination content domains. This step in the process focused on knowledge domains only because, as previously stated, certification examinations test knowledge necessary for effective and competent performance in one's role rather than the frequency and type of activities one engages in. Researchers collected the weighting sheets and computed descriptive statistics including measures of central tendency. These consisted of mean, median, standard deviation, mode, and minimum and maximum weights given by domain. The subject matter experts reviewed the results and unanimously agreed on the final recommended test weights for each knowledge domain. Results are shown in Table 7.
A significant finding in the role and function study was the elevation of the importance of two domains: quality management and ethical and legal practice. Previously, following the 2009 role and function study, ethics and quality management were embedded in other domains in the form of subdomains or major knowledge topics. On the basis of the results of the 2014 role and function study, they have been designated as separate domains. The growing prominence of these two areas represents a major shift for case managers in the importance of these knowledge requirements and their associated activities. Case managers are expected to ensure that their activities and interventions adhere to ethical and legal standards at all times. This expectation further attests to the high-functioning scrutiny and analysis required to address the complexity of case management practice and the matters that case managers deal with daily. To set standards guiding the ethical practice of case management, the CCMC first adopted a Code of Professional Conduct in 1996 to assure quality and protect the public interest. Adherence to the Code is mandatory for every board-certified case manager holding the CCM credential. CCMC recently revised the Code and subsequently published it on its website in early 2015 (Commission for Case Manager Certification, 2015).
Preparing for the Case Management Role
The demographic information gathered in the survey on educational background revealed that 70.3% of those surveyed held a bachelor's degree or higher (44.4% bachelor's degree, 24.8% master's degree, and 1.1% doctorate), a 5 percentage point gain from 2009. In addition, 20.7% held associate degrees and 9.0% a nursing diploma. Despite the increase in demand for case managers who are prepared at the bachelor's degree or higher, training of those who assume the role remains a challenge.
Table 8 shows the diversity of the approaches case managers pursued to prepare themselves for the case management role. This is an area of great opportunity for the profession. Despite the increasing demand for case managers and the demonstrated value they offer as evident in hours of work, credentials, and employer's compensation for certifications (Tahan et al., 2015), there is a continued lack of academic programs with special focus on the practice of case management. Table 8 shows that the vast majority (89%) of survey participants said on-the-job training was the primary method used to learn the practice of case management; 6.35% reported to use conferences and seminars, 0.40% used both of these modalities, and 0.81% were self-directed or self-taught. Participants reported to use formal academic programs or a combination of formal and on-the-job training, 1.37% and 0.16%, respectively. This indicates that only 117 of the 7,668 participants completed formal academic training in case management. Despite the recent increase in popularity and acceptance of the value of case management by employers, academicians have yet to fully realize the value of offering formal academic programs in case management. The lack of formal academic preparation of case managers for their roles is primarily related to the limited school or university-based degree granting programs. Treiger and Fink-Samnick (2015, p. 37) reported the availability of only six such programs in the United States, two of which are offered online.
Observations Based on Open-Ended Questions
To gather further input about current case management practice, as well as practitioners' views on how the field continues to evolve, survey participants were asked two open-ended questions:
- “What professional development and/or continuing education offerings could you use to improve your performance in your current role?”
- “How do you expect your role as a case management professional to change over the next few years? What essential activities will be performed and what knowledge will be needed to meet changing job demands?”
Researchers analyzed these responses qualitatively on the basis of the most common themes evident in the comments. In the area of professional development or educational offerings, participants expressed interest in topics related to health insurance, reimbursement methods, regulations, and preferred learning approaches.
- Keeping up-to-date with constantly changing rules and regulations, a case manager must be aware of all policy changes to be effective in their positions.
- Keeping up-to-date with insurance and reimbursement policies and knowing the possible reimbursement strategies for various situations.
- Education on new Medicare/Medicaid policies is crucial because of the growing importance and prevalence of these plans.
- Accessibility and generalization of possible continuing education opportunities such as online or 1-day workshops.
As for the changes in the case manager role expected to occur in few years, participants seemed to focus on reimbursement, Medicare and Medicaid benefit programs, relationships among health care providers, and case management across diverse practice or care settings.
- The expansion of cost-effective reimbursement strategies to deal with current reimbursement policies.
- A higher focus on the complex relationships among health care providers from different professional or educational backgrounds and across the diverse care settings.
- Case management solutions for specialized and nontraditional providers.
- Region-based health care systems, policies, and procedures.
- Case management in the home setting and increased role autonomy of health care advisors.
The role and function study highlights two highly important and interconnected reasons for conducting a case management practice analysis every 5 years. The first is to ascertain the current state of case management practice by surveying broadly among certified and noncertified practitioners in a variety of practice settings across the spectrum of health and human services. The research findings, and in particular the detailed descriptions and weightings of the essential activities and knowledge domains, inform the content and composition of the CCM certification examination. Such assurance is essential not only to the case management professional, but also to the many and varied stakeholders in the health care system, especially the public served by case managers. Certification examinations based on current evidence assure that those involved in the role possess advanced competence that ultimately contributes to safeguarding the public's interest.
The role and function study is also valuable in providing a way to understand how the practice continues to evolve, as well as in what ways it responds to the changing dynamics of health care delivery as a result of socioeconomic and political factors. For example, the results of this study show that case management practice has been positively affected by the Patient Protection and Affordable Care Act of 2010, as well as by value-based purchasing and hospital reduction of avoidable readmission programs.
A frequently-conducted role and function study of case management also contributes an in-depth understanding of the practice to incorporate into both formal and informal professional development activities for case managers (e.g., training, education, and academic programs), which must aim at ongoing advancement of competencies, skills, and knowledge of those involved in the practice.
The authors acknowledge the work of International Credentialing Associates (ICA) and its “Certified Case Manager (CCM) Role and Function Study Report” (October 2014), from which much of the data and findings of this article are derived. In addition, we acknowledge the 2014–2015 CCMC Commissioners, the subject matter experts who participated in the taskforces, and CCMC staff, including MaryBeth Kurland, Chief Operations Office, and Debby Formica, Senior Director of Administration and Operations.