Using Rasch KIDMAP to identify whether China dominates the research area of computer science (CS) based on the specialization index of article citations: Bibliometric analysis

Abstract

Background: 

Scientific comparative advantage is measured by using a specialization index (SI) of article citations. The profile data have been published in the literature. However, no such research has been conducted to determine which countries dominate the field of computer science (CS) (subject category [SC]) using the SI. A KIDMAP in the Rasch model has been applied to the display of individual student performance in school. Based on the SI of article citations, we used KIDMAP to determine whether China dominates the field of CS.

Methods: 

The data were derived from published research in the Web of Science, which included 199 countries and 254 subject categories (SC, between 2010 and 2019). A total of 96 SC related to biomedicine were extracted. We examined 7 factors associated with CS using exploratory factor analysis. Based on the SI in CS under the Rasch model, 1-dimensional SCs on CS were displayed on Wright Maps and KIDMAPs. An analysis of the dominance of CS in China was presented on the basis of a scatter plot.

Results: 

Our findings indicate that (1) CS domains are divided into 2 groups (traditional and advanced domains); (2) no evidence has been found that China dominates CS; based on SI indicators, China was ranked third with -−2.62 and 0.79 logits after Taiwan and Slovenia (-(−2.62 and 9.24 logits in Factors 1 and 2) in the period from 2010 to 2019.

Conclusions: 

There is insufficient evidence to demonstrate that China has a dominant role over other countries/regions despite ranking third in CS. In future studies, it is recommended to include a KIDMAP visual to assess dominant roles in other areas of research, rather than to confine ourselves to CS as we did in this study.

Key points

• • Online KIDMAP has been demonstrated to present the dominant role in a specific research area (e.g., computer science, subject category), which is unique and has not previously been demonstrated.
• • A detailed illustration of the online KIDMAP was provided, and it was displayed on a Google Map. Readers who are capable of drawing the online KIDMAP on their own in the future have been introduced to an online Rasch analysis.
• • It is modern and unique to combine KIDMAP with Wright Map to interpret the dominant role, which can be applied to other relevant research in the comparison of performance with other counterparts.

1. Introduction

The second-largest economy in the world, China, launched its second health reform in 2010 with considerable investments in medical healthcare (MH).^[1] One of the key contributions of this reform is MH inclusion as one of the 8 foundations of health care reform.^[2] In response to a series of policies, the Chinese government and the healthcare industry have invested a large amount of money in hospitals and population health infrastructures. The current state of medical informatics (MI) in China has been described as “hot in industry application, and cold in academic research.”^[3] Although conference output in MI remains low,^[4] whether MI (based on computer science [CS], subject category [SC]) has become a scientific comparative advantage (SCA) in China is still unknown.

China has exhibited higher research achievements (RAs) in the field of bladder cancer than the US.^[5] The current stage of ambitious health reform in China does not have information regarding whether China has dominated the SC in the past decade compared to other counterparts (e.g., USA and UK) in article citations, particularly based on the SCA.

1.1. RAs measured by the specialization index (SI)

A research category is assigned to each journal included in the Web of Science (WoS) Core Collection. There may be up to 6 categories assigned to a journal.^[6] However, the WoS scheme comprises 254 SCs assigned to journal categories in the sciences, social sciences, and arts & humanities.^[7,8] To compare the RA in China in comparison to other countries/regions, the SCA of the SI^[7] should only be considered for SCs related to biomedicine (note: the Journal of Medicine is involved in the SC of Medicine, General & Internal in WoS), particularly in the field of CS.

A first concern is whether composite SI scores (i.e., summation SI across SCs in CS) can be compared between countries (note: the summation score should meet the requirement of 1-dimensional measurement^[9]). Since the data (i.e., 254 SCs or a specific CS field) have multiple dimensions (e.g., composed of individual SCs) that prevent direct comparison of composite SI scores, the comparison is meaningless unless this undimensionality is maintained.^[10,11]

1.2. Undimensionality matters using the Rasch model

An exploratory factor analysis (EFA) used a priori prior to exporting data^[11] to Rasch analysis^[12] should indicate any dimensionality issues when developing new polytomous scales (e.g., converting SIs into ordinal categories from 0 to 5 as Likert-type scales^[10]). It is possible to compare measures after fitting data to the Rasch model because the undimensionality of the data has been maintained.^[10,11]

SI has been used to identify common and distinctive characteristics among nations using radar and bar plots for comparison.^[5] The undimensionality of the data has, however, not been examined in the study.^[5] To determine whether variables fit the Rasch model (i.e., meet the requirement of data undimensionality), fit statistics are commonly used, including infit mean square error (MNSQ) <1.5 for items (i.e., variables or SCs)^[13–15] and outfit MNSQ <2.0 for persons. Item fit statistics with infit MMSQ can be combined with Wright map^[16] to display.^[17] However, no suitable software that is user-friendly has been developed for ordinary users instead of professional researchers.

Furthermore, the national RAs should be evaluated using the Z score (= [observed $-$ expected] $÷$ SD) across items. It is possible to use the KIDMAP^[18] as a proxy to indicate which items are aberrant in responses (e.g., Z scores of items outside the upper limit of 2.0 or lower of $-$ 2.0). As a result, the observed responses are significantly higher (or lower) than the expected scores based on the individual’s ability expressed in logit units.^[19–22] There is a drawback to those KIDMAP^[18–22] in that they do not provide readers with an online version of KIDMAP that can be replicated on their own.

1.3. Research questions were raised

A second health reform was launched in China in 2010 with a significant investment in the MH field.^[2] While the USA has dominated scientific research for decades,^[23] it is unclear whether China has dominated RA in CS over other nations/regions based on citations instead of publications.

To compare the outcomes and citations in the CS field, the SI^[7] should be used. Accordingly, 3 hypotheses were proposed in the current study: RAs can be compared by using composited SI scores; China dominates the CS; and an online KIDMAP can reveal SCA for each country/region.

1.4. Citation cartels should be cautious in author collaborations

Franck wrote an essay in 1999 that defines citation cartels as partnerships between editors and journals, aimed at benefiting both parties.^[24] Inter-journal citations are one of the ways editors can boost their journal’s impact factor.^[25] In recent times, citation cartels have been observed not only between editors and journals but also between editors and authors.^[26] In these cases, members cite each other’s papers for mutual benefit, without necessarily being familiar with the content.^[25]

While modern semantic web tools can help detect citation cartels online,^[26] confirming their existence in the real world can be challenging without a detailed analysis.^[27–29] A temporary indication of a citation cartel may be a self-citation rate that exceeds 20%, 15%, or 10%.^[27–29]

The debate over the correlation between scientific productivity and impact has been ongoing in the scientific community.^[30] Research indicates that academics are often reluctant to simultaneously modify their productivity and journal prestige levels over successive years.^[30] However, the existence of citation cartels within articles remains uncertain. Journal editors and reviewers are familiar with the concept of citation cartels,^[25] and caution should be exercised when assessing an author’s research assessment solely based on article citations.^[31] Greater attention should be given to examining whether citation cartels have an impact on RAs when comparing them.^[32]

1.5. Study aims

To compare RAs and SCAs among countries, 3 hypotheses need to be confirmed: the SI scores are unidimensional in measurement to measure the domain in CS; China has dominated the CS; and a dashboard of online KIDMAP is possible to reveal the SCA for each country/region.

2. Methods

2.1. Data sources

The profile data with a 199 × 254 matrix, containing the SI values for a set of 199 countries in each of the 254 WoS SCs (over the period 2010 to 2019 and the final dataset consists of 2090,429 clusters, accounting for 122,241,092 authorships, related to 16,679,223 unique publications^[7]), were downloaded from the study. For simplification, only 45 countries by research size (e.g., those in G20; BRIC, i.e., Brazil, Russia, India, China, and South Africa; Hong Kong, Taiwan, Singapore, Iran, Vietnam, Malaysia, Indonesia, New Zealand, Indonesia, etc) and 96 SCs related to biomedicine were included in this study (see dataset in Supplemental Digital Content 1, https://links.lww.com/MD/J15). Due to the CS focus in this study, 7 SCs were finally analyzed, including CS_Artificial Intelligence, CS_Interdisciplinary Applications, CS_Cybernetics, CS_Hardware & Architecture, CS_Information Systems, CS_Software Engineering, and CS_Theory & Methods.

As this study did not involve the examination or treatment of patients or the review of patient records, it was exempt from review and approval by our research ethics committee.

2.2. Major indicators used in this study

There are 3 major indicators that are involved in this study:

1. Total fractional impact (TFI):

The TFI of country k in SC j is defined as Eq 1^[7]:

${\displaystyle \begin{array}{l}{\displaystyle TF{I}_{jk}={\displaystyle \underset{i=1}{{\displaystyle \sum ^{N_{jk}}}}}{f}_{ik}\frac{c_i}{c_j},}\end{array}}$

$N_{jk}$ = number of publications of country k, in $S{C}_j$.

$f_{ik}=$ fractional contribution of coauthors of country k to article i. For an article with n coauthors, m of which is affiliated with country k, $f_{ik}$ is equal to m/n (i.e., equal credits to each article).

In international economics, revealed comparative advantage (RCA) (or Balassa Index^[32]) is used to determine a country’s relative advantages or disadvantages in a specific category of goods or services based on trade flows. Thus, TFI is a type of RCA that compares relative citations within a country’s SC.

1. The SI:

Applying TFI, the $S{I}_{jk}$ of country k in SC j is given by Eq 2^[7]:

${\displaystyle \begin{array}{l}{\displaystyle S{I}_{jk}=\frac{\frac{TF{I}_{jk}}{{\displaystyle \sum _{j=1}^L}TF{I}_{jk}}}{\frac{{\displaystyle \sum _{k=1}^n}TF{I}_{jk}}{{\displaystyle \sum _{j=1}^L}{\displaystyle \sum _{k=1}^n}TF{I}_{jk}}},}\end{array}}$

where L is the number of SCs (i.e., 254 in the WoS) within an SC. n is the number of countries. If $S{I}_{jk}$ is >1, then country k specializes in research in SC j; there is no specialization in j if $S{I}_{jk}$ is <1. Based on the definition in Eq. 2, the SI is a normalized indicator of RCA that can be used to measure the SCA of nations.^[7]

1. SCA:

Based on SI, SCA can be measured individually for an SC across countries or for a country across SCs. To determine whether the composite SCA scores (shown in Eq 3) can be used to compare RAs between countries, it is necessary to confirm that the scale is unidimensional.^[9–11]

$RS{I}_{jk}=\mathrm{R}\mathrm{O}\mathrm{U}\mathrm{N}\mathrm{D}(S{I}_{jk}/3,0)$ as an integer in ordinal category format ranging from 0 to 5, (3)

${\displaystyle \begin{array}{l}{\displaystyle SC{A}_k={\displaystyle \underset{j=1}{{\displaystyle \sum ^L}}}RS{I}_{jk}÷L,}\end{array}}$

L is the number of SI in a country. Complete data of all 45 countries in all 7 SC-related CS are reported in Supplemental Digital Content 1, https://links.lww.com/MD/J15.

2.3. KIDMAP in Rasch analysis

In the KIDMAP, item difficulties and person measurements are vertically on the y-axis, and Z scores are horizontally on the x-axis. Unexpected items are beyond the upper and lower limits (i.e., >2.0 and <−2.0). The expected responses are vertically along the middle line against the observed scores in comparison to the expected responses of the items horizontally. Person measures are distributed and shown on the left-hand side. Five strata are divided and symbolized by alphabet from A to E, based on person measures: >3.5, >1.0, >−1.5, >−4.0, and ≤−4.0. An online KIDMAP is illustrated in Figure 1, based on 26 $persons\times 20$ items20 with polytomous responses.^[33]

Figure 1.: Interpretation of KIMDAP used in this study (note. person measure locates at the middle and other relevant statistical values are also shown on the KIDPAP).

2.4. Three parts divided in this study

2.4.1. Whether the composited SI scores as a unidimensional scale.

Whether the mean SI score (in Eq 3) is unidimensional^[9–11] should be taken into account ahead of this study. In general, EFA or Rasch analysis^[12,13] is used to inspect unidimensionality.^[9] The way to examine unidimensional data in this study is to perform Raschonline at the link^[34] and to detect the infit MNSQ (<1.5) across items^[16,17]; see details at the link.^[35]

2.4.2. Whether China dominates the CS field.

When comparing person measures in Rasch analysis, KIDMAP for China was displayed. A scatter plot was provided if 2 dimensions were found in the CS field. China can only be considered to dominate the field of CS if it ranks first in composited scores (or person measures in Rasch analysis due to the objectively sufficient feature in the Rasch model^[16,17]).

2.4.3. Whether a dashboard of online KIDMAP is possible to reveal the SCA for each country.

We utilized Raschonline,^[34] which allows readers to draw online KIDMAP to examine the impact of knowledge produced in each country/region within a specific field (e.g., CS) as measured by the RSI in Eq 3.

2.5. Creating dashboards on Google Maps

The way to draw the online KIDMAP is presented at the link.^[35] All graphs, including the scatter plot with 95% control lines,^[36] were drawn by author-made modules in Excel (Microsoft Corp, Redmond, WA). We created pages of HTML used for Google Maps (Google LLC, CA). All visual representations on Google Maps can be linked to dashboards on Google Maps.

3. Results

3.1. Whether the composited SCA score is unidimensional

Rasch analysis for RSIs was performed and is shown in Figure 2, based on 95 SCs and 45 countries/regions. We can see that 3 items with red bubbles on the right side are misfitted to the Rasch model (i.e., infit MNSQ > 1.5). The 3 misfit SCs, most difficult (i.e., with low raw scores) and easy (i.e., with frequent counts in raw scores) SCs, are shown at the bottom of Figure 2.

Figure 2.: A total of 96 items related to medical research shown on Wright Map for identifying misfit items (note. Wright map is also known as variable map or person-item map, which is unique in Rasch measurement in comparison of item difficulties and person measure on an identical continuum interval scale).

Two dimensions (i.e., factors) were found in CS using EFA in Table 1. The 2 factors are labeled by traditional and advanced CS (=computer science). Accordingly, 2 dimensions exist in CS.

Table 1 - Factor analysis of research areas in computer science.

Research area	Factor 1	Factor 2	Commun	Specific
CS_Artificial Intelligence		-0.93	0.89	0.11
CS_Interdisciplinary Applications		-0.78	0.64	0.36
CS_Science, Cybernetics		-0.72	0.51	0.49
CS_Hardware & Architecture	-0.82		0.8	0.2
CS_Information Systems	-0.77		0.92	0.08
CS_Software Engineering	-0.94		0.9	0.1
CS_Theory & Methods	-0.9		0.83	0.17
Sumsquared eigenvalues	3.05	2.45	5.5	1.5

Factors 1 and 2 are labeled as traditional computer science and advanced computer science, respectively.

3.2. Whether China dominates the CS field

Figures 3 and 4 illustrate 2 pieces of KIDMAP with China receiving grades D and C, respectively. In each KIDMAP, China did not rank first. Figure 5 illustrates 2 domains of CS for 45 countries/regions, indicating that China did not dominate the CS field between 2010 and 2019, based on SI indicators. China was ranked third with −2.62 and 0.79 logits after Taiwan and Slovenia (−2.62 and 9.24 logits in Factors 1 and 2) in the period from 2010 to 2019.

Figure 3.: KIDMAP for China in factor 1 of the research domain related to computer science (note. KIDMAP is a type of person performance sheet used for interpretation of scoring results for students and parents in school).

Figure 4.: KIDMAP for China in factor 2 of the research domain related to computer science (note. the second KIDMAP based on factor 2 for China is displayed and compared on herself based on her measurements and statistics).

Figure 5.: Two domains of computer science for 45 countries/regions shown on a scatter plot (note. the 95% confidence lines on a scatter plot is used to compare performance among entities on a sheet).

3.3. Whether a dashboard of online KIDMAP is possible to reveal the SCA for each country

In the KIDMAP, each nation’s RA is calculated using the Z score ((observed $-$ expected) $÷$ SD) across items. Figures 3 and 4 illustrate the KIDMAP when a member is selected (e.g., China#37). In Figures 3 and 4, no item had an absolute Z score >2.0 (i.e., outside the limits of > 2.0 and −2.0) based on the individual’s ability at $-2$.62 and 0.79 logits in Figures 3 and 4, respectively.

How to draw the online KIDMAP is presented at the link^[35] and Supplemental Digital Content 2, https://links.lww.com/MD/J16 Readers are encouraged to replicate the results based on data in Supplemental Digital Content 1, https://links.lww.com/MD/J15.

3.4. Online dashboards shown on Google Maps

All the QR codes in graphs are linked to the dashboards. Readers are suggested to scan QR code in Figures to examine the displayed dashboards on Google Maps.

4. Discussion

4.1. Principal findings

We observed that CS domains are divided into 2 groups (traditional and advanced domains); no evidence has been found that China dominates CS; based on SI indicators, China was ranked third with −2.62 and 0.79 logits after Taiwan and Slovenia (−2.62 and 9.24 logits in Factors 1 and 2) in the period from 2010 to 2019.

Accordingly, the 3 hypotheses that the SI scores are unidimensional in measurement to measure the domain in CS, China has not dominated the CS, and a dashboard of online KIDMAP is possible to reveal the SCA for each country/region are confirmed.

4.2. Additional information

The US and China are the top 2 largest economies in the world.^[37] The 2 countries have the most publications in the last decade.^[38] Therefore, it is meaningful to conduct a quantitative analysis of the research publications from the 2 countries to compare their RAs based on SCA using SIs in countries across SCs (note. SI is based on article citations in Eq 2).

The current state of MI in China has been described as “hot in industry application, and cold in academic research.”^[3] The most distinctive SCAs of CS were found at the edge over the US. The SCAs of individual counties are easy to compare RA with other counterparts using KIDMAP, as shown in Figures 3 and 4. The concept of SCA is based on international economics using the Balassa Index^[32] to determine a country’s relative advantages or disadvantages in a specific category of goods or services based on trade flows,

As a result of describing the scientific profile of each country in comparative terms, we are able to draw KIDMAP in comparison of measures with other counterparts and items with Z scores based on the Rasch model.

China’s dominance of some fields is still increasing; however, when compared to other counterparts based on SI in SCA, no such substantially comprehensive RAs were observed in China till 2019 (see Figures 3 and 4). It is worth investigating further what the future will hold for China in CS (i.e., after 2019).

4.3. Implications and changes

The proposed indicator and methodology have a wide range of potential applications in policy-making. Typically, governments seeking to shape their national scientific profiles are faced with scarce resources, which requires selective allocation, sometimes in conjunction with incentives. Despite their best efforts, their results are likely to be affected by uncontrollable forces, both domestic and international, including sudden changes in research directions along the intersections of disciplines^[39,40] and author collaboration in affiliation research institutes in countries/regions.

The study has several distinctive features. First, Online KIDMAP has been demonstrated to present the dominant role in a specific research area (e.g., CS, SC), which is unique and has not previously been demonstrated in the literature.

Second, the Rasch multidimensional model using Raschonline^[34] provided us with a visual representation of the person-item map (i.e., Wright Map), which transformed raw responses into a logit interval score. The Rasch rating scale model^[41] was applied to the website of Raschonline^[34] developed by the authors in this study.

Third, a detailed illustration of the online KIDMAP^[42] was provided, and it was displayed on Google Maps. Readers who are capable of drawing the online KIDMAP on their own in the future have been introduced and demonstrated in this study.

4.4. Limitations and suggestions

Several issues need to be thoroughly examined in further research. First, the method used to draw visualizations shown in Supplemental Digital Content 2, https://links.lww.com/MD/J16 is not unique and not inevitable. The way to draw these graphs may use any method that provides similar or additional features to the visual results.

Second, the dashboards in this study are displayed on Google Maps. Because Google Maps requires a paid project key for the use of the cloud platform, these installments are not free of charge. This makes it difficult for other authors to replicate the use in a short period of time.

Third, Raschonline^[34] can produce KIDMAP and Wright Map by copy & paste approaches. It is recommended that Rasch analysis be performed to produce KIDMAP in a more easily friendly manner in the future.

Fourth, we acknowledge that the proposed approach has limitations, in particular, due to the typical biases associated with raw data of SI from a previous study,^[7] which only considered scientific publications published in international scientific journals as research output and indexed in certain commercial bibliographic collections. Future research could focus on repeating the proposed analysis using input-based indicators. We would be interested in exploring the similarities and differences between the analytical outcomes of the 2 approaches, as well as the factors responsible for these differences, particularly for those related to intercountry differences in scientific productivity.

Fifth, over the past decade or more, the total output of Chinese research has grown rapidly. There have been many reports that China is still lagging behind in terms of quality.^[43] It is possible to take another measure that ranks in the top 1% of scientific studies^[43] or article citations (e.g., SI in Eq 2), as did use the SI in future relevant studies.

Finally, it should be noted that the SCAs disclosed with KIDMAP are only for China. Other countries with a higher SI than China might lead to drawing KIDMAP as well, as shown in Figures 3 and 4, to provide policy suggestions to their government.

5. Conclusion

The study objectives were achieved by verifying 3 hypotheses: the SI scale is not unidimensional to directly compare RAs in countries; AAC can distinguish the distinctive features of China and the US; and a dashboard is possible to reveal the SCA for each country (or each SC) that has been confirmed in this study.

A breakthrough was achieved by using 6 visualizations to reveal the SCA of nations, particularly with a person-item map in the Rasch multidimensional model to confirm the $SC{A}_k$ that should be replaced with the $SC{A}_R$ or $SC{A}_S$ in Eqs. 8 and 9. Otherwise, the composite SI score used to differentiate RAs in countries/regions would be problematic due to several domains in the data. Specifically, the use of visualizations to identify study results at a glance is recommended in future relevant studies.

Acknowledgments

We thank Enago (www.enago.tw) for the English language review of this manuscript.

Author contributions

Conceptualization: Po-Tsung Yen.

Formal analysis: Willy Chou.

Investigation: Wei-Chih Kan.

Methodology: Tsair-Wei Chien.