Cumulative receiver operating characteristics for analyzing interaction between tissue visfatin and clinicopathologic factors in breast cancer progression

Background Visfatin has been reported to be associated with breast cancer progression, but the interaction between the visfatin and clinicopathologic factors in breast cancer progression status requires further investigation. To address this problem, it is better to simultaneously consider multiple factors in sensitivity and specificity assays. Methods In this study, a dataset for 105 breast cancer patients (84 disease-free and 21 progressing) were chosen. Individual and cumulative receiver operating characteristics (ROC) were used to analyze the impact of each factor along with interaction effects. Results In individual ROC analysis, only 3 of 13 factors showed better performance for area under curve (AUC), i.e., AUC > 7 for hormone therapy (HT), tissue visfatin, and lymph node (LN) metastasis. Under our proposed scoring system, the cumulative ROC analysis provides higher AUC performance (0.746–0.886) than individual ROC analysis in predicting breast cancer progression. Considering the interaction between these factors, a minimum of six factors, including HT, tissue visfatin, LN metastasis, tumor stage, age, and tumor size, were identified as being highly interactive and associated with breast cancer progression, providing potential and optimal discriminators for predicting breast cancer progression. Conclusion Taken together, the cumulative ROC analysis provides better prediction for breast cancer progression than individual ROC analysis. Electronic supplementary material The online version of this article (10.1186/s12935-018-0517-z) contains supplementary material, which is available to authorized users.

However, these factors are generally taken individually without considering possible mutual interactions.
Visfatin (nicotinamide phosphoribosyltransferase or NAMPT) [9] plays an important role in regulating metabolism, inflammation, and carcinogenesis [10,11]. Visfatin was also reported to overexpress in many types of cancer, such as colorectal [12], prostatic [13], gastric [14], esophageal [15], and breast cancer [16][17][18][19][20]. Therefore, visfatin is considered to be an oncoprotein. Our previous study [21] reported that visfatin was highly expressed in breast cancer tissues and is associated with tumor size, ER negativity (−), PR (−), and poor diseasefree/overall survival as well as low recurrence rates for hormone therapy. The correlation between visfatin and several clinicopathologic factors were analyzed individually [21], but possible interaction between visfatin and breast cancer progression-associated clinicopathologic factors has not been investigated. Moreover, the specificity and sensitivity of visfatin as an independent prognosis predictor for breast cancer has not been addressed.
Previously statistical analysis approaches for breast cancer prognosis typically depend on regression models, including logistic regression and the Cox proportional regression method. ROC analysis could provide another strategy for breast cancer prognosis based on sensitivity and specificity. In addition, possible interaction between visfatin and clinicopathologic factors in breast cancer progression has not been investigated. Thus, this study aims to evaluate the interaction effects of clinicopathologic factors in breast cancer progression and identifies high risk factors through cumulative ROC analysis.

Dataset information
The dataset was derived from our previous work [21] with Institutional Review Board (IRB) approval (KMUH-IRB-980567) and informed consent. A total of 105 female breast cancer patients treated with surgery but without radiotherapy or chemotherapy were enrolled at the Department of Surgery, Kaohsiung Medical University Hospital during 2003-2008. Data including visfatin levels and clinicopathological factors is downloadable at https://wp.kmu.edu.tw/changhw/files/2017/05/ROC_tissue-visfatin_dataset.xlsx.

Grouping for breast cancer progression
Breast cancer patients were classified into progressing and disease-free groups. The progressing group included patients with local/regional recurrence or distant metastasis of breast cancer along with those did not survive the follow-up period. The disease-free group included patients who remained disease-free for 60 months following initial diagnosis and who survived the follow-up period.

Individual ROC analysis
Clinicopathologic factors were dichotomized by ROC curve analysis. The area under the ROC curve (AUC) is used to calculate the accuracy of dichotomous results for predicting the risk of breast cancer progression. A higher AUC value represents a better prognosis prediction for breast cancer progression.

Cumulative ROC analysis
To score cumulative ROC analysis, the top-ranked factors are ranked according to the AUC of individual ROC analysis. This system transforms individual ROC analysis data into dichotomous data for cumulative ROC analysis. In brief, the cutoff point for each clinicopathologic factor was determined from its individual ROC curve. Values of each factor either above or below its the cutoff point were respectively assigned scores of 1 and 0. All data were transformed into this scoring system for further cumulative ROC analysis.
Different factors were chosen to calculate their cumulative AUC values. This cumulative analysis began with selecting the first two factors with the top AUC values for an individual ROC. Other factors were then tested as the third one and the cumulative AUC value was calculated. Only the factor that provided the highest cumulative AUC value was kept. This process was repeated for other factor combinations. Accordingly, different numbers of clinicopathologic factors that generated the largest cumulative AUC value were selected to investigate the joint effects of clinicopathologic factors associated with breast cancer progression. Table 1 summarizes the distribution of clinicopathologic factors between the disease-free and the progressing groups. The difference between both groups was estimated using the Fisher's exact test or χ 2 test. The dichotomous analysis for single and combinational clinicopathologic factors were determined by individual and cumulative ROC as mentioned above (2.3 and 2.4). Combining different factors, the likelihood ratio (LR) [23] was chosen to evaluate disease-free status in patients. LR+, i.e., sensitivity/(1 − specificity) [23], is the ratio of the probability of a positive test in the progressing patients to that of disease-free patients. In contrast, LR−, i.e., 1 − sensitivity)/specificity [23], is the ratio of the probability of a negative test in the progressing patients to that of disease-free patients. All statistical calculations were analyzed by STATA version 11.0.

Clinicopathologic characteristics and progression of breast cancer
Disease progression developed in 21 (20.0%) of the 105 breast cancer patients tracked within 5-year follow-up. Table 1 summarizes the clinicopathologic characteristics of breast cancer patients in disease-free and progressing statuses. Compared to the disease-free patients, the progressing patients had a significantly higher proportion of tissue visfatin > 50% (81.0%), stage III/IV (52.4%), grade 3 (33.3%), age ≥ 52 years (76.2%), tumor size ≥ 2 cm (81.0%), LN metastasis (71.4%), triple negative subtype (38.1%), and hormone therapy (71.4%). Table 2 presents the dichotomous results for the various single clinicopathologic factors listed in Table 1. Hormone therapy (AUC = 0.7143), tissue visfatin (AUC = 0.7083), and LN metastasis (AUC = 0.7083) show acceptable performance in the AUC results (≥ 0.7) under single factor analysis, indicating these clinicopathologic factors are acceptable prognosis factors for breast cancer progression. The AUC values for other factors were less than 0.7.

Cumulative ROC analysis of breast cancer progression
Our proposed cumulative ROC analysis scoring system was used to determine the combined effects of tissue visfatin and these clinicopathologic factors. As shown in Table 3, the cumulative AUC value for the combinations of cumulative top-ranked clinicopathologic factors was calculated using the AUC value for different factor combinations, where the factors were added individually in descending order of individual AUC value ( Table 2). The scoring system obtained the highest AUC value (0.886) for the 6 cumulative top-ranked factors combination, including HT, tissue visfatin, LN metastasis, stage, age, and tumor size. AUC values increased from 0.764 to 0.886 for the cumulative top-ranked clinicopathologic factors, but decreased slightly after combining 7 or more cumulative top-ranked factors. Therefore, 6 dichotomized clinicopathologic factors yielded the best prognosis prediction for breast cancer progression.

Cumulative ROC analysis performance for predicting breast cancer progression
According to the cumulative top-ranked clinicopathologic factors scoring results, 6 selected dichotomized clinicopathologic factors provided scores ranging from 0 to 6 (Table 4). Both scores 1 and 2 have 100% sensitivity (i.e., all progressing patients are correctly classified).
However, score 2 has a higher specificity compared to score 1 (45.24% vs 14.29%), indicating that score 2 can predict disease-free patients more precisely. Score 4 has the best result, with a correctly classification of 83.81% for disease progression with the best combination of sensitivity (76.19%) and specificity (85.71%) as well as good LR performance (LR+ 5.33 and LR− 0.28).

Discussion
We previously reported that the correlation between tissue visfatin and several individual clinicopathologic factors [21], but the specificity and sensitivity, along with mutual interactions, were not investigated for breast cancer progression. In the current study, individual ROC analysis was used to evaluate the specificity and sensitivity performance of these clinicopathologic factors associated with breast cancer progression. However, as shown in Table 2, aside from HT, tissue visfatin and LN metastasis, the other clinicopathologic factors displayed weak ROC performance (individual AUC < 0.7), and contributed less impact to breast cancer progression. These results suggest that many single factors may have a differential impact on breast cancer progression. Moreover, the possible interactions between tissue visfatin and clinicopathologic factors were not examined by individual ROC analysis.
A similar effect (Additional file 1) was found in univariate logistic regression results, but multivariate analysis Table 4 Performance of scores from cumulative ROC analysis in predicting breast cancer progression. Dataset was retrieved from our previous study [21] LR+ Likelihood ratio for a positive test result, LR− Likelihood ratio for a negative test result a The number of dichotomized factors was the cumulative effects of the various clinicopathologic factors from  Table 5 Distribution for scores and selected factors associated with breast cancer progression. Dataset was retrieved from our previous study [21] a Cumulative score (Table 4)   produced inconsistent results, except for LN metastasis. According to the individual ROC results (Table 2), the individual AUC of LN metastasis is 0.7083, with a cumulative AUC combining HT, tissue visfatin, LN metastasis, stage, and age, and the tumor size is 0.886. Although multivariate logistic regression analysis presents a novel approach to breast cancer diagnosis, the ROC analysis approach based on sensitivity and specificity also provides acceptable results.
Cumulative ROC analysis has recently been applied in many research fields [32][33][34][35][36][37]. Six of 25 cephalometric measurements were selected by cumulative ROC analysis to optimize the determination of surgical or nonsurgical treatment for skeletal Class III malocclusions [32]. Cumulative ROC analysis has also been applied in cancer association studies, identifying cumulative risks for five SNPs for association with papillary thyroid carcinoma [33]. Thus, cumulative ROC can effectively identify joint effects of multiple disease-or gene-associated factors.
Similarly, our study developed a scoring system for multiple factors-based cumulative ROC analysis and identified a minimum of 6 clinicopathologic factors for predicting breast cancer progression: HT, tissue visfatin, LN metastasis, tumor stage, age, and tumor size ( Table 3). Only 4 of these 6 factors provided acceptable sensitivity, specificity, and classification accuracy for predicting breast cancer progression through cumulative ROC analysis with a high AUC score of 0.886 (Table 4). These results suggest that the cumulative ROC analysis strategy can be effectively used to identify the joint effect of multiple factors for breast cancer progression.
Recently, interactions between genes and physical activities has been emphasized [38][39][40], with a particular focus on genetic polymorphisms. In the current study, we focus on the genetic expression of tissue visfatin in terms of protein level in tissue histochemistry. Cumulative ROC analysis shows tissue visfatin levels can combine with common clinicopathologic factors to provide an analysis of the interaction between breast tumor biomarkers and clinicopathologic factors.
Among these interactions, HT is the best single predictor for breast cancer progression (Table 2). Clinical studies have suggested that breast cancer patients may benefit from adjuvant HT [41][42][43]. Tissue visfatin has also been reported to be an important biomarker for breast cancer progression [21]. In the current study, HT and tissue visfatin were shown to have a strong joint association with breast cancer progression. In addition, tumor size and lymph node metastasis are common prognosis factors for both breast cancer progression [44,45] and Nottingham prognostic index [45]. Advanced stage (III/ IV) breast cancer is typically characterized by greater tumor size, lymph node invasion, pathological change in the surrounding tissue, or metastases in distant organs. Advanced stage tumor burden is usually associated with a poor disease prognosis in breast cancer [46][47][48]. Furthermore, age has been suggested to be an important risk factor for breast cancer, as inferred from the association with age-related tissue microenvironment alteration [49]. Moreover, therapeutic impact differs with age [50,51]. Hence, breast cancer progression is significantly associated with tumor characteristics and aging [48], but how this age-dependency increases in disease progression remains unclear. Moreover, the proportion of progressing patients with 6 selected high-risk clinicopathologic factors (No-HT, tissue visfatin > 50%, LN metastasis, tumor stage III/IV, age > 50 years, and tumor size ≥ 2 cm) increased with cumulative score, indicating that these selected high-risk clinicopathologic factors were highly associated with breast cancer progression. Accordingly, these factors exhibit a high degree of mutual interactivity in cumulative ROC analysis.
Although our study did not consider the follow-up interval for breast cancer progression, our proposed strategy for cumulative ROC analysis shows the joint effect of clinicopathologic factors for predicting breast cancer progression in terms of sensitivity and specificity for disease prediction. Furthermore, the proposed scoring system was based on the highest AUC performance from cumulative ROC analysis and did not account for the association of low AUC clinicopathologic factors effects on breast cancer progression.
The present study is subject to several limitations. No follow-up intervals are included and the cumulated ROC approach is mainly affected by the AUC ranking results, which might restrict exhaustive search for all possible combinations of complex interactions. For example, although ER, PR and Her2 status were reported to be highly associated with breast cancer progression, the cumulated AUC ranking results for the combined interactions still excluded these 3 factors. In addition, we did not include molecular subtype (TNBC, luminal A, luminal B and Her 2 enriched). Moreover, the visfatin level was examined from the tumor specimens of breast cancer patients, indicating the visfatin level required invasive procedures. Further examination of visfatin level through the secretary samples such as blood should be considered.

Conclusions
The current study examined the impact of 13 clinicopathologic factors for predicting breast cancer progression. Under our proposed scoring system, the cumulative ROC analysis provides better AUC performance than individual ROC analysis in predicting breast cancer progression. Analysis of factor interaction identified a minimum of 6 factors which display a high degree of mutual interaction and association with breast cancer progression: HT, tissue visfatin, LN metastasis, tumor stage, age, and tumor size.