The clinical outcome of acute ischemic stroke (AIS) patients is associated with the grade of collateral filling.1 It is hypothesized that the ischemic process can be slowed down by collateral vessels perfusing the tissue that is still salvageable, which is termed the penumbra.2 In patients who undergo endovascular treatment (EVT), good collateral filling is associated with smaller final infarct volumes and better clinical outcomes than when the collateral filling is poor or absent.2–4
In the past decade, collateral evaluation has gained much interest and multiple grading systems for different imaging modalities have been developed.5,6 Computed tomography angiography is widely used for the evaluation of patients with suspected AIS. It is a fast, relatively inexpensive and noninvasive method for assessing occlusions and potential causes of stroke. A single time frame snapshot (single-phase CTA) of the collateral status can be assessed and used for treatment guidance and predicting patient outcomes. However, a single time frame snapshot may make it difficult to grade collateral circulation in case of delayed contrast arrival.2,7 Multiphase CTA may solve the issues that come with single-phase CTA.8,9 In addition, multiphase CTA has proven to be superior to grading collaterals on single-phase CTA in terms of interobserver agreement, predicting final infarct core volume and predicting functional outcome.8,9 Patients with suspected stroke often undergo both CTA and CT perfusions (CTP). Computed tomography perfusion is the ultimate multiphase CTA study since it typically includes 40 to 50 time frames obtained at a 1- to 3-second temporal resolution and monitors the contrast agent from wash-in to wash-out. Therefore, CTP source images can be used to reconstruct 3-phase CTA, multiphase CTA and temporal maximum-intensity projection (tMIP) images. Digital subtraction angiography (DSA) is a 2D study with high spatial and temporal resolution and high contrast making it the ultimate dynamic study of the cerebral vessels and the reference standard for assessing collateral circulation in AIS patients.10 However, DSA is invasive, relatively costly and time consuming, which is undesirable in the acute stroke setting. Digital subtraction angiography is, therefore, only routinely performed in patients who subsequently undergo EVT.
With regard to collateral assessments, single-phase CTA, CTP-derived 3-phase CTA, multiphase CTA, tMIP images, and DSA have not been compared before.
In this study, we aimed to compare assessment of collateral filling on single-phase CTA and CTP-derived 3-phase CTA, multiphase CTA, and tMIP images in terms of interrater reliability and agreement between imaging modalities. In a secondary analysis, we aimed to evaluate the relation between collateral assessments and 90-day clinical outcome.
MATERIALS AND METHODS
Consecutive adult patients with AIS from Stanford Medical Center, who were considered for EVT between 2010 and 2018, were selected for this study. Inclusion criteria were (1) presence of CTP and CTA images that were acquired as part of the acute stroke protocol and (2) occlusion of the internal carotid artery (ICA) and/or occlusion of the M1 and/or M2 segment of the middle cerebral artery. Patients were excluded in case of bilateral stroke. Digital subtraction angiography cases were excluded if the filming did not extend into the late venous phase, patient motion precluded adequate interpretation, or if only anteroposterior images were available for analysis. The need for informed consent was waived by the local institutional research board for this retrospective analysis of data, which was collected as part of clinical practice. Of the 530 patients who were considered for EVT 101 were included for the interobserver analysis. Eighty patients had interpretable DSA and 74 had clinical outcome assessed at 90 days (Fig. 1).
Baseline data included demographics and cardiovascular risk factors, such as hypertension, diabetes mellitus, hyperlipidemia, smoking, atrial fibrillation, prior cerebrovascular accident, prior coronary artery disease, and use of antiplatelet medication or anticoagulants. Furthermore, we collected admission National Institutes of Health Stroke Scale and data on treatment including administration of intravenous tissue-type plasminogen activator, EVT, and whether reperfusion was achieved, defined as a Thrombolysis in cerebral infarction Scale of IIB or III. We also recorded time from symptom onset to treatment.
Different CT scanners were used from Siemens Healthcare (Erlangen, Germany) and General Electric Healthcare (Milwaukee, WI). Computed tomography perfusion was performed with 80 kV and 100 mAs. Thirty-seven phases at 1-second time interval followed by 33 phases at 3-second time interval were acquired. Either one run with 10-mm slices or 2 runs with 5-mm slices were performed covering at least ASPECTS levels 1 and 2 of the brain.
The CTA studies of the carotid arteries were obtained on 64-slice CT scanners. The image acquisition protocol was as follows: spiral mode, a 0.6- to.8-second gantry rotation; collimation, 64 × 0.5–1 mm; pitch, around 1; slice thickness, 1 to 1.25 mm; reconstruction interval, 0.75 to 0.1 mm, and acquisition parameters, 120 kVp/240 mA. A caudocranial scanning direction was selected, covering the midchest to the vertex of the brain. Seventy to 80 mL of Isovue 300 or 370 (lopamidol; Bracco Diagnostics Inc, Monroe Township, NJ) was injected into an antecubital vein with a power injector at a rate of 4 to 5 mL per second. Optimal timing of the CTA acquisition was achieved using a test bolus technique.
A Siemens Artis Zee biplane neuroangiography unit was used for EVT. The DSA images were acquired during EVT with standard anteroposterior and lateral views following cervical ICA contrast injection of the afflicted side.
Single-phase CTA images were created from thin-slice data and saved as 20-mm maximum intensity projection (MIP) images. The images were captured on ASPECTS levels 1 (basal ganglia) and 2 (at the top of the lateral ventricles), respectively.11
The CTP source data were used to create 3-phase CTA, multiphase CTA and tMIP images. Three-phase CTA images were prepared using the CTP images with the thinnest slice thickness available (ranging from 1.5 to 10 mm). The first phase was selected based on the peak of arterial inflow, which was automatically calculated and checked visually. The second phase was timed 8 seconds after the first phase and the third phase was timed 8 seconds after the second phase.8 Similar to single-phase CTA, 20-mm MIP images were captured on ASPECTS levels 1 and 2, respectively.
Multiphase CTA was created from all the CTP time-phases containing thick slice images (ranging from 5 to 10 mm) on the 2 ASPECTS levels. To assess the collateral score dynamically, the series were saved as video files.
All the CTP time-phases were reconstructed into tMIP images, where the highest Hounsfield unit was taken for every voxel with perfusion analysis software (Intellispace Portal, version 6.0; Philips Healthcare, Best, the Netherlands). The MIP thickness was set to 20 mm and the images were captured on ASPECTS levels 1 and 2. The reconstructions were created automatically by the software.
Regarding the reference standard, all the available DSA images, which were acquired after CT, were used for determining the grade of collateral circulation before EVT took place.
The single-phase CTA, 3-phase CTA, multiphase CTA, and tMIP images were presented separately to 2 separate neuroradiologists (J.D.—19 years of experience and M.W.—21 years of experience). The images were anonymized and placed in a randomized order for the reviews. The observers were blinded to clinical information and other imaging data except for the occlusion site. On single-phase CTA, 3-phase CTA, multiphase CTA, and tMIP images, the collaterals in the affected territory were graded using a static collateral score: 0, absent collaterals; 1, collaterals filling 50% or less of the occluded territory; 2, collaterals filling greater than 50%, but less than 100% of the occluded territory; 3, collaterals filling 100% of the occluded territory.12–14 For 3-phase CTA, multiphase CTA and DSA. The modified American Society of Interventional and Therapeutic Neuroradiology (ASITN) score15 was used, which enables dynamic grading of collaterals: 0, nonexistent or barely visible pial collaterals on the ischemic site during any point of time; 1, partial collateralization of the ischemic site until the late venous phase; 2, partial collateralization of the ischemic site before the venous phase; 3, complete collateralization of the ischemic site by the late venous phase; 4, complete collateralization of the ischemic site before the venous phase.15,16 Collateral assessment on DSA images was done by a third observer (J.H.—6 years of experience) and was used as the reference standard. The decision for choosing one DSA observer was based on previous studies.17,18 No consensus meetings were arranged.
Favorable clinical outcome was defined as a modified Rankin Scale (mRS) 0–2, 90 days after the index event.
We reported frequencies and percentages for categorical variables and medians with first (Q1) and third (Q3) quartiles for continuous variables. Since collateral scores are an ordinal variable, the agreement between the 2 observers was quantified by calculating Cohen κ, with disagreements weighted according to their squared distance from perfect agreement.19 The level of agreement was categorized and based on the κ values: poor, <0.20; fair, 0.21 to 0.40; moderate, 0.41 to 0.60; good, 0.61 to 0.80; very good, 0.81 to 1.00. Confidence intervals for κ statistics were based on 5000 bootstrap resamples. Concordance between the pooled CT measurements and DSA measurements was quantified using percentages of agreement and concordance correlation coefficients (CCCs).20,21 Because clinical decision making is often based on the dichotomized (either poor or good collateral supply) collateral score, we also evaluated the dichotomized (0–1 vs 2–3) static collateral score and modified (m)ASITN (0–2 vs 3–4) score with respect to the comparison between CT and DSA.13,22,23 Lastly, we quantified the relation between the imaging assessments on the original and binary scales and the primary outcome (90-day mRS, ≤ 2) by using binary logistic regression, and we adjusted for recanalization status.24 Association measures were reported as odds ratios (OR) and 95% confidence intervals (CI). The statistical analyses were performed in R (version 3.5.0).
Patients with (n = 80) and without (n = 21) interpretable DSA acquisitions were compared (Table 1). Median age of the total group (n = 101) was 75 (Q1–Q3, 67–82), and 38 (38%) patients were men.
TABLE 1 -
Patient Characteristics, Overall and Stratified by Availability of DSA
||Total (N = 101)
||DSA (n = 80)
||No DSA (n = 21)
|Age: median (Q1–Q3), y
|Admission NIHSS, median (Q1–Q3)
|Time from symptom onset to CTP
| Time to tPA (min.), median (Q1–Q3)
| Reperfusion (TICI IIB-III)
|Medical history, n (%)
|Prior antiplatelet or anticoagulant therapy
|Imaging findings, n (%)
|Left side occluded on CTA
|Occlusion site on CTA
| Multiple sites
|Follow-up (n = 76), n (%)
|Favorable clinical outcome at 90 days*
Categorical characteristics are summarized as counts and percentages (calculated out of nonmissing values), and continuous characteristics are summarized as medians with first (Q1) and third (Q3) quartiles.
* Defined as mRS ≤ 2.
NIHSS indicates National Institutes of Health Stroke Scale; TICI, thrombolysis in cerebral infarction; CVA, cerebrovascular accident; CAD, coronary artery disease; M1, M1 segment of middle cerebral artery; M2, M2 segment of middle cerebral artery.
Observed static collateral scores (Supplemental Table I, http://links.lww.com/RCT/A105) were generally high in this population: the number of observations ranged from 0 to 3 (0–3%) for collateral score 0, from 8 to 23 (8–23%) for collateral score 1, from 22 to 44 (22–44%) for collateral score 2 and from 42 to 68 (42–68%) for collateral score 3. Similarly, the number of mASITN scores ranged from 1 to 2 (1–2%) for mASITN 1, from 4 to 19 (4–19%) for mASITN 2, from 17 to 45 (17–45%) for mASITN 3, and from 7 to 47 (7–47%) for mASITN 4.
Disagreements between observers did not differ more than 1 point on the collateral scales, resulting in disagreements between the dichotomized scales in 10 cases.
CTA and DSA images from an example case with good collateral circulation are shown in Figure 2 and Figure 3, respectively. The CTA and DSA images from an example case with poor collateral circulation are shown in Figure 4 and Figure 5, respectively.
Interobserver agreement was 0.46 (tMIP), 0.58 (3-phase CTA), 0.67 (multiphase CTA), and 0.69 (single-phase CTA) for static assessments and 0.52 (3-phase CTA) and 0.54 (multiphase CTA) for dynamic assessments (Fig. 2).
Concordance between the CT observations and the reference standard (DSA) is summarized in Table 2 and displayed in Figure 6. Agreement (range, 29–53%) and concordance (CCC range, 0.08–0.27) on the ordinal scale were poor. Agreement improved after dichotomization of the collateral scores (range, 54–81%), but concordance remained poor (CCC range, −0.02 to 0.24).
TABLE 2 -
Measures of Concordance Between Collateral Scores Assessed on CT and DSA (Reference Standard)
||CCC (95% CI)
||CCC (95% CI)
| CS single-phase
| CS 3-phase
| CS multiphase
| CS tMIP
| mASITN 3-phase
| mASITN multiphase
CS indicates collateral score;
Clinical outcome was successfully collected in 76 (75%) patients. Patient characteristics did not differ significantly between patients who had follow-up and patients who did not have follow-up. Adjusted for recanalization, higher static collateral scores on multiphase CTA (OR, 1.7; 95% CI, 1.1–2.7) and tMIP (OR, 2.0; 95% CI, 1.1–3.4) were associated with favorable clinical outcome (Table 3). Similarly, higher mASITN scores on 3-phase CTA (OR, 1.5; 95% CI, 1.1–2.2) and multiphase CTA (OR, 1.7; 95% CI, 1.1–2.6) were associated with favorable clinical outcome. For single-phase CTA (OR, 1.1; 95% CI, 0.7–1.8) and 3-phase CTA (OR, 1.6; 95% CI, 1.0–2.6), a positive trend was observed for the association with favorable clinical outcome. No significant associations were found between DSA assessments and clinical outcome.
TABLE 3 -
Associations Between Imaging Assessments and 90-Day Clinical Outcome (Modified Rankin Scale <3)
||Crude OR (95% CI)
||Adjusted* OR (95% CI)
| 3 Phase
|Collateral score DSA
| Original scale
| Binary scale
| Original scale
| Binary scale
* Adjusted for recanalization status.
† P < 0.05.
We compared single-phase CTA and CTP-derived 3-phase CTA, multiphase CTA and tMIP with DSA with respect to their ability to reliably grade collateral circulation in AIS patients. We evaluated 2 scoring systems: a static collateral score entailing 4 categories and a dynamic collateral score (mASITN) entailing 5 categories. Agreement between the 2 observers was moderate to good. Collateral assessment achieved similar interobserver agreement for all imaging modalities and concordance with DSA was in general poor. Associations with favorable outcome were significant for CTP-derived 3-phase CTA, multiphase CTA and tMIP assessments, but not for single-phase CTA or DSA assessments.
The observed interobserver agreement in this study was comparable to previously reported measures of agreement. Previous studies found κ values for single-phase CTA of 0.49,13 0.8714 and 0.68,17 which is consistent with the results of our study (κ, 0.69). Multiphase CTA has been proposed as a reliable tool for assessing collateral grade in patients with AIS.8,9 Interobserver agreement was found to be very good (κ, 0.81) in one small study, while we found a κ of 0.58.8
The overall concordance between CT assessments and DSA was poor, despite the reasonable agreement between the observations on CT and DSA. Studies evaluating the concordance between CT and DSA are scarce. One study found poor agreement between single-phase CTA and DSA (κ, 0.24), which is in line with the observations in this study.17 Another study found a modest correlation between anterior circulation collaterals on CTA and DSA, but different collateral scores were used for each imaging modality.18 Another study found good agreement (82%) and a correlation (Spearman correlation coefficient 0.83) between multiphase CTA and DSA.25 Three-point collateral scales were used, but no concordance measures were given. As a result, the percentage of agreement corresponds with our study, but we were not able to compare concordance measures between multiphase CTA and DSA. An explanation for the observed concordance may be the fact that neurointerventionalists do not usually perform DSA of either the contralateral ICA or the posterior circulation in anterior circulation stroke. As a result, the DSA assessment of collateral filling is likely to be incomplete relative to information from CTA and CTP. Also, the collateral scales on each modality and technique may be sufficiently different that they do not correlate. Computed tomography (CT) angiography lacks the temporal resolution of DSA, and CTA scales cannot capture the time resolved blood flow into the collateral circulation. For instance, if one looks at time point 1 and time point 2, which are separated by 20 seconds on a multiphase CTA, the collateral score might be good on the CTA scale. However, the DSA scale might show that it takes a very long time for the collateral vessels to fill, which might translate to a poor collateral score. Perfusion maps generated from CTP may offer more information than CTA because they allow evaluation of blood flow on a tissue level. However, evaluation of perfusion maps was beyond the scope of this study.
Collateral grading has been established as a predictor of clinical outcome independent of the number of phases acquired with CT.4,26–28 In our study, single-phase assessments were not significantly related to favorable clinical outcome, although a positive trend was observed: the more acquired CT phases, the stronger the associations between the collateral grading and clinical outcome. This is in line with a previous study, which reported that the association with clinical outcome was stronger for multiphase assessments than for single-phase assessments.9 Similarly, one study found that time-invariant CTA was superior to single-phase CTA in terms of predicting clinical outcome with the collateral score.7 These results confirm the added value of multiple-phase CTA, either created with CTA or with CTP, when it comes to prediction of clinical outcomes.
The association between DSA assessments and clinical outcome has been investigated in one study, but no association was observed.17 This is in line with our study in which we did not observe an association between DSA assessments and clinical outcome. However, the results from this particular analysis must be interpreted with caution because selection bias cannot be ruled out completely. Still, the selection process in this study reflects routine clinical care and therefore we believe the risk of selection bias is low. Future studies could elucidate whether prediction of clinical outcome is indeed more feasible with CTA than with DSA.
We calculated measures of concordance (CCCs) enabling us to evaluate accuracy and consistency between the CT assessments from 2 observers and the DSA-assessments from one observer. However, CCC and other measures of agreement are sensitive to imbalanced data as was the case in our study. To solve this issue, a case control design may be used, but we did not have enough cases to use such a design. Patients with poor collaterals are unlikely to be selected for EVT, because they likely had larger infarctions that would exclude them from thrombectomy eligibility. As a result, the predominant number of patients with good collateral supply in our study leads to an increased risk of agreement by chance only, resulting in poor CCCs. In our clinic, all patients with suspected AIS either undergo CT or MRI or both. Selection bias may have been an issue as many patients were excluded from this study because no CT data were available. The majority of these patients was imaged with MRI instead of CT. Evaluation of collateral grading on MRI was beyond the scope of this study. We do not believe that collateral filling is associated with the choice of imaging modality, because it is expected that patients with poor collateral filling preferably undergo CT as their clinical condition is probably worse than patients with good collateral filling. Moreover, poor agreement between assessments on CTA and DSA was also observed in a more balanced population before.17
Still, the imbalanced distribution of collateral filling is a limitation of our study, and future studies could, therefore, reassess measures of correlation between CT and DSA in more balanced populations. Patients who did not undergo DSA after all were excluded from the concordance analyses. We do not believe that this elevates the risk of selection bias as the selection process in this study reflects routine clinical care. Another limitation of our study is that we captured and assessed images on the 2 ASPECTS levels only. This was done to achieve a standardized assessment. This could have influenced the collateral assessments as the observers were not able to look at the whole vasculature of the brain. Lastly, we did not evaluate commercially available CT grading software for assessing collateral filling in patients with AIS. In our opinion the method of assessment of collateral filling should also be possible without specialized software since it is not standardly available. Future studies could, however, evaluate how collateral grading done by observers compares to automated grading tools.
In conclusion, collateral assessment achieved similar interobserver agreement for all imaging modalities and concordance with DSA was in general poor. Collateral status evaluated on multiphase CTA was associated with clinical outcome, which was not the case for DSA.
1. Ribo M, Flores A, Rubiera M, et al. Extending the time window for endovascular procedures according to collateral pial circulation. Stroke
2. Vagal A, Aviv R, Sucharew H, et al. Collateral Clock Is More Important Than Time Clock for Tissue Fate: A Natural History Study of Acute Ischemic Strokes
. 2018. Epub ahead of print 1 September 2018. DOI: 10.1161/STROKEAHA.118.021484.
3. Elijovich L, Goyal N, Mainali S, et al. CTA collateral score predicts infarct volume and clinical outcome after endovascular therapy for acute ischemic stroke: a retrospective chart review. J Neurointerv Surg
4. Wufuer A, Wubuli A, Mijiti P, et al. Impact of collateral circulation status on favorable outcomes in thrombolysis treatment: a systematic review and meta-analysis. Exp Ther Med
5. McVerry F, Liebeskind DS, Muir KW. Systematic review of methods for assessing leptomeningeal collateral flow. AJNR Am J Neuroradiol
6. Martinon E, Lefevre PH, Thouant P, et al. Collateral circulation in acute stroke: assessing methods and impact: a literature review. J Neuroradiol
7. Smit EJ, Vonken E, van Seeters T, et al. Timing-invariant imaging of collateral vessels in acute ischemic stroke. Stroke
8. Menon BK, d'Esterre CD, Qazi EM, et al. Multiphase CT angiography: a new tool for the imaging triage of patients with acute ischemic stroke. Radiology
9. Garcia-Tornel A, Carvalho V, Boned S, et al. Improving the evaluation of collateral circulation by multiphase computed tomography angiography in acute stroke patients treated with endovascular reperfusion therapies. Interv Neurol
10. Liu L, Ding J, Leng X, et al. Guidelines for evaluation and management of cerebral collateral circulation in ischaemic stroke 2017. Stroke Vasc Neurol
11. Pexman JH, Barber PA, Hill MD, et al. Use of the Alberta Stroke Program Early CT Score (ASPECTS) for assessing CT scans in patients with acute stroke. AJNR Am J Neuroradiol
12. Kim JJ, Fischbein NJ, Lu Y, et al. Regional angiographic grading system for collateral flow: correlation with cerebral infarction in patients with middle cerebral artery occlusion. Stroke
13. Tan JC, Dillon WP, Liu S, et al. Systematic comparison of perfusion-CT and CT-angiography in acute stroke patients. Ann Neurol
14. Tan IY, Demchuk AM, Hopyan J, et al. CT angiography clot burden score and collateral score: correlation with clinical and radiologic outcomes in acute middle cerebral artery infarct. AJNR Am J Neuroradiol
15. Seker F, Potreck A, Mohlenbruch M, et al. Comparison of four different collateral scores in acute ischemic stroke by CT angiography. J Neurointerv Surg
16. Higashida RT, Furlan AJ, Roberts H, et al. Trial design and reporting standards for intra-arterial cerebral thrombolysis for acute ischemic stroke. Stroke
17. Jansen IGH, Berkhemer OA, Yoo AJ, et al. Comparison of CTA- and DSA-based collateral flow assessment in patients with anterior circulation stroke. AJNR Am J Neuroradiol
18. Casault C, Al Sultan AS, Trivedi A, et al. Collateral scoring on CT angiogram must evaluate phase and regional pattern. Can J Neurol Sci
19. Cohen J. Weighted kappa: nominal scale agreement with provision for scaled disagreement or partial credit. Psychol Bull
20. Lin LI. A concordance correlation coefficient to evaluate reproducibility. Biometrics
21. Morgan CJ, Aban I. Methods for evaluating the agreement between diagnostic tests. J Nucl Cardiol
22. Soares BP, Tong E, Hom J, et al. Reperfusion is a more accurate predictor of follow-up infarct volume than recanalization: a proof of concept using CT in acute ischemic stroke patients. Stroke
23. Hernandez-Perez M, Puig J, Blasco G, et al. Dynamic magnetic resonance angiography provides collateral circulation and hemodynamic information in acute ischemic stroke. Stroke
24. Demchuk AM, Goyal M, Yeatts SD, et al. Recanalization and clinical outcome of occlusion sites at baseline CT angiography in the Interventional Management of Stroke III trial. Radiology
25. Kim SJ, Noh HJ, Yoon CW, et al. Multiphasic perfusion computed tomography as a predictor of collateral flow in acute ischemic stroke: comparison with digital subtraction angiography. Eur Neurol
26. Schregel K, Tsogkas I, Peter C, et al. Outcome prediction using perfusion parameters and collateral scores of multi-phase and single-phase CT angiography in acute stroke: need for one, two, three, or thirty scans?J Stroke
27. Maier IL, Scalzo F, Leyhe JR, et al. Validation of collateral scoring on flat-detector multiphase CT angiography in patients with acute ischemic stroke. PLoS One
28. Shin N-Y, Kim K, Park M, et al. Dual-phase CT collateral score: a predictor of clinical outcome in patients with acute ischemic stroke. PLoS One