Validation of the predictive accuracy of “clinical + morphology nomogram” for the rebleeding risk of ruptured intracranial aneurysms after admission
Chinese Neurosurgical Journal volume 8, Article number: 5 (2022)
Rebleeding can cause a catastrophic outcome after aneurysmal subarachnoid hemorrhage. A clinical + morphology nomogram was promoted in our previous study to assist in discriminating the rupture intracranial aneurysms (RIAs) with a high risk of rebleeding. The aim of this study was to validate the predictive accuracy of this nomogram model.
The patients with RIAs in two medical centers from December 2020 to September 2021 were retrospectively reviewed, whose clinical and morphological parameters were collected. The Cox regression model was employed to identify the risk factors related to rebleeding after their admission. The predicting accuracy of clinical + morphological nomogram, ELAPSS score and PHASES score was compared based on the area under the curves (AUCs).
One hundred thirty-eight patients with RIAs were finally included in this study, 20 of whom suffering from rebleeding after admission. Hypertension (hazard ratio (HR), 2.54; a confidence interval of 95% (CI), 1.01–6.40; P = 0.047), bifurcation (HR, 3.88; 95% CI, 1.29–11.66; P = 0.016), and AR (HR, 2.68; 95% CI, 1.63–4.41; P < 0.001) were demonstrated through Cox regression analysis as the independent risk factors for rebleeding after admission. The clinical + morphological nomogram had the highest predicting accuracy (AUC, 0.939, P < 0.01), followed by the bifurcation (AUC, 0.735, P = 0.001), AR (AUC, 0.666, P = 0.018), and ELAPSS score (AUC, 0.682, P = 0.009). Hypertension (AUC, 0.693, P = 0.080) or PHASES score (AUC, 0.577, P = 0.244) could not be used to predict the risk of rebleeding after admission. The calibration curve for the probability of rebleeding showed a good agreement between the prediction through clinical + morphological nomogram and actual observation.
Hypertension, bifurcation site, and AR were independent risk factors related to the rebleeding of RIAs after admission. The clinical + morphological nomogram could help doctors to identify the high-risk RIAs with a high predictive accuracy.
Rebleeding is a catastrophic event with a high mortality after aneurysmal subarachnoid hemorrhage [1,2,3]. Patients can be protected from poor outcomes though appropriate surgical intervention [4,5,6,7,8], for the limitation of medical sources, a notable part of patients cannot receive treatment as soon as they are sent to hospital. Considering that rebleeding occurs most within 6 h after the initial hemorrhage [2, 3, 9], it is meaningful to establish a model, through which the ruptured intracranial aneurysms (RIAs) with a high risk of rebleeding could be quickly identified.
There are two main aspects related to the stability of intracranial aneurysms, known as the pathological characteristics of aneurysm wall  and the hemodynamic characteristics of aneurysms [11,12,13]. Considering the strong correlation between pathological characteristics of aneurysm wall and morphological features , we established a nomogram model based on clinical and morphological factors (hypertension, bifurcation, irregular shape, and aspect ratio) to identify the RIAs with a high risk of rebleeding after admission . However, the predictive accuracy was not validated by other independent cohorts.
In this study, we retrospectively reviewed the clinical characteristics of patients with RIA from two medical centers and the morphology of RIAs. The aim of this study was to validate the predictive accuracy of “clinical + morphological nomogram.”
Study group and screening criteria
We retrospectively reviewed the patients with RIAs in Beijing Tiantan Hospital and Beijing and Peking University International Hospital from December 2020 to September 2021. As is presented in Fig. 1, patients were selected according to the following standards: (1) a CTA (computational tomography angiogram) was performed after initial hemorrhage; (2) the patients were sent to our institution within 12 h as soon as the occurrence of symptoms, e.g., when an acute headache or a sudden coma occurred; (3) clinical records were complete, or clinical history could be traced. We further excluded the patients who (1) had other intracranial tumors, angiostenosis or angio-malformation, e.g., meningioma or arteriovenous malformation; (2) had a family history of intracranial aneurysm; (3) had multiple intracranial aneurysms, because it would make it difficult to identify the source of bleeding or rebleeding; (4) had dissection, thrombus or traumatic aneurysms; (5) received special treatment for RIA in other medical institutions before admitted to our institution.
Once patients were admitted to the medical institutions, they would receive standard medical care, including acute pressure lowering and intensive medical monitoring, followed by the guideline and recommendation [16, 17]. The target of blood pressure was to lower their systolic pressure to 120–140 mmHg.
After initial hemorrhage, patients with Hunt-Hess I–II would receive a surgical intervention within 72 h, who could not benefit from immediate surgical treatment; therefore, we would just give these patients conservative treatment until the Hunt-Hess grade “degraded”; however, once patients’ condition progressively deteriorated, an emergency surgical intervention would be performed. In addition, for patients with Hunt-Hess V, surgical treatment would not be recommended.
Clinical information was collected from electronical medical records regarding to age, gender, comorbidities (including hypertension, dyslipidemia, diabetes mellitus, coronary heart disease, and ischemic stroke), aneurysmal subarachnoid hemorrhage history, Hunt-Hess grade at admission, time from admission to rebleeding (time from admission to neurological symptoms) or surgery as well as blood pressure at admission and before rebleeding/surgery. The time interval from admission to rebleeding or surgery was recorded. Modified Fisher scale (mFS) and IA site were collected based on radiological data.
The digital imaging and communications on medicine data of CTA performed after admission were collected and converted into reordered slices (about 0.5 mm per slice). The DICOM data was introduced into Mimics 17.0 (Mimics Research 17.0, Materialize, Belgium) and reconstructed for further studies.
The measurement of morphological parameters was performed by the same neurosurgeons (PJ.J. and J.W.) based on the vascular model. The discrepancies were solved by consulting a senior neuro-interventionist (HW.H.). Aneurysm size (S), diameter of dome (D), perpendicular height (H), diameter of parent artery, vessel angle (VA), aneurysm inclination angle (AA), volume, and surface area were measured here . The parameters mentioned were measured twice by two investigators, the average of whose measurements was taken for further analyses. Aspect ratio (AR), size ratio (SR), undulation index (UI), and nonsphericity index (NSI) were calculated . An irregular shape was identified according to our previous study .
The nomogram points were calculated based on hypertension, AR and bifurcation site, which were then transferred into risk probability or rebleeding according to our previous study . The risk probability of rebleeding was used for further analyses.
Identification of rebleeding after admission
In this study, rebleeding was diagnosed based on two aspects: (1) the patients had a sudden disorder of consciousness, a gradually worsening neurological state or convulsion after admission, and (2) the magnitude of subarachnoid, intracerebral or intraventricular blood significantly increased in immediate CTs compared with that at admission, and the magnitude of bleeding did not increase or kept stable at/before admission. Rebleeding was confirmed by experienced neurosurgeons (S.W.) according to the bleeding presentation on medical record and CT after admission.
The patients suffering from rebleeding were categorized as the rebleeding group, otherwise the stable group.
Statistical analysis and model establishment
Categorical variables were presented as numbers (n) and percentage (%). Continuous variables with a normal distribution were presented as means and standard deviation, and medians (m) as well as inter-quartile range (IQR) if possible. We compared the differences between continuous variables of the two groups based on Student’s t tests or Wilcoxon rank sum tests as well as the differences in categorical variables based on chi-square tests or Fisher’s exact tests. The clinical + morphological nomogram was given in our previous study ; the risk of rebleeding after admission was calculated, meanwhile PHASES score and ELAPSS score were calculated according to previous protocols [20, 21]. The parameters with significance in univariable analysis were input into Cox regression model to identify the independent risk factors, whose result was presented as hazard ratio (HR) and a confidence interval of 95% (CI). A two-tailed P < 0.05 was considered statistically significant, the SPSS 24.0 (SPSS, Chicago, IL) was adopted for statistical analyses, and a two-sided P < 0.05 was considered as statistical significance.
A calibration curve was derived to assess the calibration of the actual rebleeding percentage through the model. The nomogram was subjected to bootstrapping validation (1000 bootstrap resamples). The predictive accuracy of the model while predicting rebleeding was measured by the area under the curve (AUCs) through receiver operating characteristic curve (ROC) analyses. The models with AUC > 0.7 were considered as useful models for clinical work.
The differences between rebleeding group and stable group
This study finally included 138 patients with RIAs, 20 (14.5%) of whom suffered from rebleeding after admission (Fig. 2). The information of patients and RIAs was summarized in Table 1. For all patients, within those whose age ranged from 21 to 77 years, 57 (41.3%) were male and 47 (34.1%) had a hypertension history. Sixty-five (47.1%) RIAs sited in the internal carotid artery, 49 were (25.5%) in the middle cerebral artery, and 24 (17.4%) sited in other sites.
The significance was found in gender (P = 0.002), hypertension (P = 0.033), irregular shape (P = 0.009), bifurcation (P < 0.001), aneurysm size (P = 0.005), AA (P = 0.014), AR (P < 0.001), SR (P = 0.018), bottleneck factor (P = 0.014), and height-to-width ratio (P = 0.002). There was no significance in dyslipidemia, diabetes mellitus, mFS scale, Hunt-Hess grade, locations, dome diameter, height, VA, volume, surface area, or NSI (all P > 0.05).
The ELAPSS score was significantly different between rebleeding group and stable group (14 vs. 9, P = 0.009), whereas the PHASES score failed to be significant (2 vs.2, P = 0.237).
Risk factors related to the rebleeding after admission
The result of Cox regression analysis was summarized in Table 2. We performed univariate Cox analyses based on the parameters with significance in univariate analysis. The result showed that hypertension, bifurcation, AR, irregular shape, aneurysm size, AR, SR, bottleneck factor, and height-to-width ratio were risk factors for rebleeding after admission, which were then input into a multivariate model through the backward method. The parameters, including hypertension (HR, 2.54; 95% CI, 1.01–6.40; P = 0.047), bifurcation (HR, 3.88; 95% CI, 1.29–11.66; P = 0.016), and AR (HR, 2.68; 95% CI, 1.63–4.41; P < 0.001), were demonstrated as independent risk factors related to rebleeding after admission.
The predictive accuracy for the rebleeding after admission
We further compared the predictive accuracy of independent risk factors, clinical + morphological nomogram, ELAPSS, and PHASES of rebleeding after admission. According to the result of ROC analyses (Fig. 3A–D and Table 3), the nomogram had the highest predictive accuracy (AUC, 0.939, P < 0.001), followed by bifurcation (AUC, 0.735, P = 0.001) and AR (AUC, 0.666, P = 0.018). However, hypertension (AUC, 0.693, P = 0.080) and PHASES score (AUC, 0.577, P = 0.244) could not be used to predict the risk of rebleeding after admission. For the risk of rebleeding after admission, the calibration plot showed a substantial agreement between the prediction by nomogram and the actual observation (Fig. 4). To avoid the effect of Hunt-Hess grade and time from admission to rebleeding or surgery, we performed a subgroup analysis, whose result showed that rebleeding RIAs had a higher risk probability in both I–II-grade and III–IV-grade patients (Supplementary Figure 1) on each day (Supplementary Figure 2).
Rebleeding is a catastrophic event after aneurysmal subarachnoid hemorrhage. How to identify RIAs with a high risk of rebleeding is helpful to make treatment strategies. This study demonstrated hypertension, bifurcation and AR as independent risk factors related to rebleeding after admission. We validated the predictive accuracy of the clinical + morphological nomogram for the risk of rebleeding.
Hypertension can aggravate the unstable hemodynamic condition of intracranial aneurysms [20,21,22]. In the current study, we found that hypertension could increase the risk of rebleeding after admission, which was consistent with our previous study . The potential mechanism is that a systematic hypertension can damage the vascular structure and make the vessels fragile [22,23,24]. Such biological effect makes the vessels prone to rupture once an accidental stress occurs.
It was also confirmed in this study that the aneurysms sited in bifurcation or with a large AR had a high risk of rebleeding after admission, and that the aneurysms sited in bifurcation had a higher possibility to suffer from the impact of blood flow [13, 25]. The dynamic change from direct-impact area to surrounding area could cause physical injuries to the aneurysm wall . The aneurysms with a large AR usually are in an unstable hemodynamic condition with severe damage in aneurysm wall . In our preliminary studies, we also found that AR was also a predictor for rebleeding of RIAs . Based on these facts, bifurcation site and AR could serve as the parameters to identify the RIAs with a high risk of rebleeding after admission.
In previous study, we established a nomogram based on clinical and morphological characteristics of RIAs, including hypertension, bifurcation, irregular shape and aspect ratio, which were also confirmed as the independent risk factors for rebleeding in this study. It was revealed in our further analyses that the clinical + morphological nomogram reached a high predictive accuracy, which outperformed independent risk factors, ELAPSS and PHASES model. This model could help neurosurgeons and neuro-interventionists quickly identify the rebleeding risk of RIAs based on CTA, which was also easy to follow and understand. In China, because of a large patient population with limitations of medical resources , it is meaningful to give priority to RIAs with a high risk of rebleeding. Therefore, the clinical + morphological nomogram could assist doctors to make treatment strategies.
There were two limitations in the current study. First of all, because of the change in morphology after indiscoverable rebleeding  and the effect of hemorrhage on the quality of radiological images, our conclusion may be limited. Secondly, not so many potential factors were considered this study, which might be also related to the risk of rebleeding. Though there are some limitations, this study could also provide the evidence that clinical + morphological nomogram could help doctors to identify the RIAs with a high risk of rebleeding.
Hypertension, bifurcation site, and AR were independent risk factors related to rebleeding of RIAs after admission. The clinical + morphological nomogram could help doctors to identify the high-risk RIAs with a high predictive accuracy.
Availability of data and materials
Data sharing not applicable to this article as no datasets were generated or analyzed during the current study.
Ruptured intracranial aneurysm
Computational tomography angiography
Modified Fisher scale
Diameter of dome
Aneurysm inclination angle
Kienzler J, Marbacher S, Remonda L, Soleman J, Ai Schlaeppi J, Leupold U, et al. Outcome after in-hospital rebleeding of rupture of intracranial aneurysms. J Neurol Surg A Cent Eur Neurosurg. 2016;77(3):207–21. https://doi.org/10.1055/s-0035-1570007.
Jaechan P, Hyunjin W, Dong-Hun K, Yong-Sun K, Young KM, Im Hee S, et al. Formal protocol for emergency treatment of ruptured intracranial aneurysms to reduce in-hospital rebleeding and improve clinical outcomes. J Neurosurg. 2015;122(2):383–91. https://doi.org/10.3171/2014.9.JNS131784.
Rosenørn J, Eskesen V, Schmidt K, Rønde F. The risk of rebleeding from ruptured intracranial aneurysms. J Neurosurg. 1987;67(3):329–32.
Darkwah Oppong M, Skowronek V, Pierscianek D, Gembruch O, Herten A, Saban DV, et al. Aneurysmal intracerebral hematoma: risk factors and surgical treatment decisions. Clin Neurol Neurosurg. 2018;173:1–7. https://doi.org/10.1016/j.clineuro.2018.07.014.
Cordonnier C, Demchuk A, Ziai W, Anderson CS. Intracerebral haemorrhage: current approaches to acute management. Lancet. 2018;392(10154):1257–68. https://doi.org/10.1016/S0140-6736(18)31878-6.
Fisher CM, Kistler JP, Davis JM. Relation of cerebral vasospasm to subarachnoid hemorrhage visualized by computerized tomographic scanning. Neurosurgery. 1980;6(1):1–9. https://doi.org/10.1227/00006123-198001000-00001.
Frontera JA, Claassen J, Schmidt JM, Wartenberg KE, Temes R, Jr Connolly SE, et al. Prediction of symptomatic vasospasm after subarachnoid hemorrhage: the modified fisher scale. Neurosurgery. 2006;59(1):21–7. https://doi.org/10.1227/01.NEU.0000218821.34014.1B.
Ko SB, Choi HA, Carpenter AM, Helbok R, Schmidt JM, Badjatia N, et al. Quantitative analysis of hemorrhage volume for predicting delayed cerebral ischemia after subarachnoid hemorrhage. Stroke. 2011;42(3):669–74. https://doi.org/10.1161/STROKEAHA.110.600775.
Hijdra A, van Gijn J, Nagelkerke NJ, Vermeulen M, van Crevel H. Prediction of delayed cerebral ischemia, rebleeding, and outcome after aneurysmal subarachnoid hemorrhage. Stroke. 1988;19(10):1250–6. https://doi.org/10.1161/01.STR.19.10.1250.
Frosen J, Tulamo R, Paetau A, Laaksamo E, Korja M, Laakso A, et al. Saccular intracranial aneurysm: pathology and mechanisms. Acta Neuropathol. 2012;123(6):773–86. https://doi.org/10.1007/s00401-011-0939-3.
Dolan JM, Kolega J, Meng H. High wall shear stress and spatial gradients in vascular pathology: a review. Ann Biomed Eng. 2013;41(7):1411–27. https://doi.org/10.1007/s10439-012-0695-0.
Meng H, Xiang J, Liaw N. The role of hemodynamics in intracranial aneurysm initiation. Int Rev Thromb. 2012;7(1):40–57.
Meng H, Tutino VM, Xiang J, Siddiqui A. High WSS or low WSS? Complex interactions of hemodynamics with intracranial aneurysm initiation, growth, and rupture: toward a unifying hypothesis. AJNR Am J Neuroradiol. 2014;35(7):1254–62. https://doi.org/10.3174/ajnr.A3558.
Liu Q, Jiang P, Wu J, Li M, Gao B, Zhang Y, et al. Intracranial aneurysm rupture score may correlate to the risk of rebleeding before treatment of ruptured intracranial aneurysms. Neurol Sci. 2019;40(8):1683–93. https://doi.org/10.1007/s10072-019-03916-1.
Liu Q, Yang Y, Yang J, Li M, Yang S, Wang N, et al. Rebleeding of ruptured intracranial aneurysm after admission: a multidimensional nomogram model to risk assessment. Front Aging Neurosci. 2021;13:692615. https://doi.org/10.3389/fnagi.2021.692615.
American Society of Anesthesiologists Task Force on Perioperative Blood M. Practice guidelines for perioperative blood management: an updated report by the American Society of Anesthesiologists Task Force on Perioperative Blood Management*. Anesthesiology. 2015;122(2):241–75. https://doi.org/10.1097/ALN.0000000000000463.
Connolly ES Jr, Rabinstein AA, Carhuapoma JR, Derdeyn CP, Dion J, Higashida RT, et al. Guidelines for the management of aneurysmal subarachnoid hemorrhage: a guideline for healthcare professionals from the American Heart Association/american Stroke Association. Stroke. 2012;43(6):1711–37. https://doi.org/10.1161/STR.0b013e3182587839.
Jiang P, Liu Q, Wu J, Chen X, Li M, Li Z, et al. A novel scoring system for rupture risk stratification of intracranial aneurysms: a hemodynamic and morphological study. Front Neurosci. 2018:12(596). https://doi.org/10.3389/fnins.2018.00596.
Dhar S, Tremmel M, Mocco J, Kim M, Yamamoto J, Siddiqui AH, et al. Morphology parameters for intracranial aneurysm rupture risk assessment. Neurosurgery. 2008;63(2):185–97. https://doi.org/10.1227/01.NEU.0000316847.64140.81.
Backes D, Rinkel GJE, Greving JP, Velthuis BK, Murayama Y, Takao H, et al. ELAPSS score for prediction of risk of growth of unruptured intracranial aneurysms. Neurology. 2017;88(17):1600–6. https://doi.org/10.1212/WNL.0000000000003865.
Greving JP, Wermer MJ, Brown RD Jr, Morita A, Juvela S, Yonekura M, et al. Development of the PHASES score for prediction of risk of rupture of intracranial aneurysms: a pooled analysis of six prospective cohort studies. Lancet Neurol. 2014;13(1):59–66. https://doi.org/10.1016/S1474-4422(13)70263-1.
Tada Y, Wada K, Shimada K, Makino H, Liang EI, Murakami S, et al. Roles of hypertension in the rupture of intracranial aneurysms. Stroke. 2014;45(2):579–86. https://doi.org/10.1161/STROKEAHA.113.003072.
Lindgren AE, Kurki MI, Riihinen A, Koivisto T, Ronkainen A, Rinne J, et al. Jaaskelainen JE, von und zu Fraunberg M: Hypertension predisposes to the formation of saccular intracranial aneurysms in 467 unruptured and 1053 ruptured patients in Eastern Finland. Ann Med. 2014;46(3):169–76. https://doi.org/10.3109/07853890.2014.883168.
Taylor CL, Yuan Z, Selman WR, Ratcheson RA, Rimm AA. Cerebral arterial aneurysm formation and rupture in 20,767 elderly patients: hypertension and other risk factors. J Neurosurg. 1995;83(5):812–9. https://doi.org/10.3171/jns.1995.83.5.0812.
Xiang J, Natarajan S, Tremmel M, Ma D, Mocco J, Hopkins L, et al. Hemodynamic-morphologic discriminants for intracranial aneurysm rupture. Stroke. 2011;42(1):144–52. https://doi.org/10.1161/STROKEAHA.110.592923.
Dhar S, Tremmel M, Mocco J, Kim M, Yamamoto J, Siddiqui AH, et al. Morphology parameters for intracranial aneurysm rupture risk assessment. Neurosurgery. 2008;63(2):185–96; discussion 196-187. https://doi.org/10.1227/01.NEU.0000316847.64140.81.
Bian LH, Liu YF, Nichols LT, Wang CX, Wang YL, Liu GF, et al. Epidemiology of subarachnoid hemorrhage, patterns of management, and outcomes in China: a hospital-based multicenter prospective study. CNS Neurosci Ther. 2012;18(11):895–902. https://doi.org/10.1111/cns.12001.
Skodvin TO, Johnsen LH, Gjertsen O, Isaksen JG, Sorteberg A. Cerebral aneurysm morphology before and after rupture: nationwide case series of 29 aneurysms. Stroke. 2017;48(4):880–6. https://doi.org/10.1161/STROKEAHA.116.015288.
We thank Qingyuan Liu and Shuo Wang provided technical support in analyzing the predictive accuracy of each model.
This work was supported by the projects of National Natural Science Foundation of China (Grant Nos. 82071296, 81471210, and 81671129), “Major special projects in the 13th five-year plan” (Grant No. 2016YFC1301800) and “Major special projects in the 14th five-year plan” (Grant No. 2021YFC2501100). The sponsors had no role in the design or conduct of this research.
Ethics approval and consent to participate
The work was approved by the Institutional Review Board of Tiantan Hospital (KY2017-076-01).
Consent for publication
Written informed consents will be obtained from their legally authorized representatives. Privacy of patient will be effectively protected.
All authors certify that we have no affiliations with or involvement in any organization or entity with any financial interest (such as honoraria; educational grants; participation in speakers’ bureaus; membership, employment, consultancies, stock ownership, or other equity interest; and expert testimony or patent-licensing arrangements), or non-financial interest (such as personal or professional relationships, affiliations, knowledge or beliefs) in the subject matter or materials discussed in this manuscript.
Supplementary Figure 1. Subgroup analysis based on Hunt-Hess grade. We performed subgroup analysis based on Hunt-Hess grade, and found that the rebleeding RIAs had higher risk probability in both I-II grade and III-IV grade patients.
Supplementary Figure 2. Subgroup analysis based on operation time. We performed subgroup analysis based on operation time. 96 patients received surgery at the first day, 37 received surgery at the second day, and 5 received surgery at the third day. Comparing with the stable RIAs, the rebleeding RIAs had higher risk probability in each day.
About this article
Cite this article
Sui, J., Wang, N., Jiang, P. et al. Validation of the predictive accuracy of “clinical + morphology nomogram” for the rebleeding risk of ruptured intracranial aneurysms after admission. Chin Neurosurg Jl 8, 5 (2022). https://doi.org/10.1186/s41016-022-00274-4