A. Microscopy Imaging Data

Digital microscopy is rapidly emerging as a tool for establishing pathologic diagnosis, evaluating treatment efficacies and performing morphologic research. In distinction to traditional visual review of histological sections, which introduces human bias and remains largely qualitative [4], a computer-based analysis of virtual microscopic slides can be systematic, objective, efficient, and complete [5][6][7]. Moreover, many features in a microscopic image can be identified and analyzed by computer algorithms but not by human observers. Thus, imaging data from histologic slides contains rich phenotypic information that can potentially be exploited to yield clinically meaningful results.

In our research, we have used the microscopic images from TCGA project on glioblastomas (GBMs), which are WHO grade IV astrocytic neoplasms that are rapidly progressive and ultimately fatal. All digitized slides are Haematoxylin and Eosin (H&E) stained permanent sections that were formalin-fixed and paraffin-embedded. In aggregate, 428 whole slides associated with 162 patients are included. All were scanned at 20x magnification with a high-resolution, high-throughput digitized scanner. The overall storage size of the complete image data set for study is about 175Gbytes with JPEG compression ratio of 5.11. The image resolution is up to 63922 × 45753 pixels.

B. “Omics” Data

Phenotypic data derived from digitized images was correlated with TCGA molecular data, providing insight to underlying biological mechanisms and potentially uncovering therapeutic targets within a morphologic class. Each TCGA sample was characterized by multiple molecular platforms including gene (mRNA) and microRNA expression, DNA copy number variation, DNA sequence and DNA methylation.

A recent study of TCGA GBMs defined four transcriptional subtypes: proneural (PN), neural (NR), classical (CL) and mesenchymal (MS) [8]. Each subtype is defined by a characteristic gene expression profile and genetic alterations, including mutations and chromosomal changes (amplification/deletion). For our study, transcriptional subtypes were either obtained from the supplementary information in an earlier work [8] or determined with Prediction Analysis of Microarray (PAM) software version 2.21 using RMA normalized Affymetrix HT-HGU133 mRNA expression platform data. A sample expression average was computed for samples with multiple corresponding arrays. Unlogged expression was filtered to remove probes with a fold change less than 1.5 or an expression range less than 20.

Somatic mutations and chromosome alterations (amplification or deletion) for genes CDKN2A, EGFR, IDH1, NF1, PDGFRA, TP53, and PTEN, have been provided by the Memorial Sloan-Kettering Cancer Center (MSKCC)². Mutational status from 205 samples was available. Copy number variation data from the same set consists of a consensus derived from a combination of platforms (Agilent, Affymetrix SNP 6, Illumina) together with methods (RAE[9], GISTIC[10], GTS[11]) for identifying regions of genomic aberration likely to drive cancer pathogenesis. Copy number alterations are represented by homozygous deletion, hemizygous deletion, neutral change, gain, and high-level amplification.

C. Clinical Outcomes

Clinical data on patient age, chemo- and radiotherapies, and survival was downloaded from the TCGA portal³.

D. Computational Infrastructure

High resolution digitized pathology images are extremely large, with some occupying several gigabytes even in compressed form. The TCGA dataset include hundreds of pathology images and presents a significant computational challenge for analysis. To expedite processing, we partitioned each whole slide image into non-overlapping regions of 4096 × 4096 pixels to permit parallel analysis. This choice balances between memory requirements and the loss of microanatomy due to tiling. Larger regions have physical memory constraints. Smaller ones place a greater fraction of nuclei on region boundaries resulting in their loss during analysis. To scale up the analysis component of the architecture, we process images with a large-scale, high-performance computation infrastructure where a cluster of computer nodes executes jobs simultaneously. This configuration currently consists of seven Dell 1950 1U rack mount units. Each unit is configured with Dual Xeon E5420 CPUs running with four cores at 2.5Ghz for a total of eight cores per node.

A. Microscopy Imaging Analysis

We developed a suite of image analysis tools for segmenting and characterizing nuclei. To reliably identify nuclei, we applied the fast hybrid grayscale reconstruction algorithm to images for normalizing background regions degraded by artifacts arising from tissue preparation and scanning [18]. This operation substantially separates the foreground from the normalized background and allows recognition of nuclei by simple thresholding. Overlapped nuclei were subsequently separated with the watershed method.

We then extracted a complementary set of features for each identified nucleus to obtain phenotypic signatures of GBMs. These features fall under four primary headings: nuclear morphometry, region texture, intensity and gradient statistics, as summarized in Fig. 2 (a) [14]. Since specific nuclear features have traditionally been used to distinguish types of gliomas, morphometric features (such as the degree of elongation, and size) are included. Nuclear texture information is captured by multiple descriptors, as it varies across nuclei due to the content and clumping of chromatin. Features relevant to nuclear intensity and intensity gradient are included as well. All nuclear features are computed with the grayscale image channel converted from the original color image. Additionally, we applied the same set of texture and gradient features to “cytoplasm” regions surrounding nuclei. Since the true cellular borders of glioma cells cannot be resolved on H&E stained images, cytoplasm refers to a fixed-distance radius surrounding a nucleus. In practice, we dilated the nuclear regions with an eight-pixel margin to identify this space. Fig. 2 (b) presents a small image region where glioma nuclei and cytoplasm regions are depicted. Features derived from cytoplasm are computed with the grayscale image channel as well as the isolated channels for H&E stain signals separated by a color deconvolution algorithm [15]. As the cytoplasm space is obtained by dilating the nuclear regions, its morphologic features are not calculated. Cytoplasm features are then combined with nuclear features for better representation. In aggregate, 74 features extracted from nuclei and proximal cytoplasm describe the morphology and texture characteristics of each nucleus and its neighboring area.

Open in a separate window

Fig. 2

(a) Features computed from nuclear and “cytoplasm” regions are summarized; (b) A typical image region is overlaid with nuclear (red) and “cytoplasm” (green) boundaries identified by computer algorithms.

All nuclear and cytoplasmic features associated with a GBM were then summarized into a single vector to represent each patient. To this end, we calculated the first moment of each feature and the second moments of all possible pairs of features [16]. The first moment represents the average value for a specific feature, whereas the second order statistics define relationships between features regarding 1) nuclear morphology, 2) nuclear morphology and nuclear staining, or 3) nuclear morphology/staining and cytoplasmic staining for each patient. The summarization step produces an N(N+3)/2-dimensional feature vector to represent the morphology of each patient in a high dimensional space, where N is equal to 74 in our case. Thus, each patient is represented by a 2849-dimensional imaging signature vector derived by aggregating the features of machine-identified nuclei in the associated microscopic whole-slide images.

This is followed by a consensus clustering procedure to compute the probabilities that signatures of pairwise cases are grouped in the same cluster over 100 independent trials of K-means experiments. This analysis is aimed at uncovering the existence of intrinsic morphological clusters defined by nuclear feature signatures. We set the number of clusters as K=3.

B. PAIS Query Support

Segmentation results and features are stored in the PAIS database. To correlate micro-anatomic morphometry with molecular profiles and clinical outcome, summary statistics on image features need to be computed for each patient. This process involves calculating the mean feature vectors and the feature covariance values of all possible feature pairs over all nuclei in images of each patient. The PAIS database is queried to search for feature pairs and retrieve corresponding feature values. The summary statistics for each image are combined in a separate program to create a single-feature vector for a patient. Queries for the mean, standard deviation, and covariance of feature calculations are supported through IBM DB2 Structured Query Language (SQL) queries with DB2’s built-in aggregation functions: the AVG, STDDEV, and COVARIANCE functions, respectively. An example of PAIS database query for the mean and covariance of three morphometry features, i.e. area, perimeter, and eccentricity, is shown in Fig. 3 where calculation_flat, and patient are two tables storing nuclear morphometry features and patient-slide relationships; pais_uid is the primary key that joins these two tables.

Open in a separate window

Fig. 3

A SQL example is presented to query for mean and covariance nuclear feature vector for each patient in PAIS database.

With the efficient and expressive database query support on morphological signature computation, we are able to correlate nuclear morphometry with clinical outcomes and molecular characterizations and to produce results suggesting a possible relationship across nuclear morphometry, patient survival, and molecular data.

A. Response to Therapy and Survival Analysis

Summary nuclear feature vectors are computed with slides grouped by patients. These patients are further grouped into three consensus clusters based on nuclear morphometry. In Table 1, we present the p-values of the Log-Rank test [17] comparing patient survival to the three consensus clusters. The Log-Rank test between cluster two and three yields statistically significant difference in survival, with longer survivals for patients in cluster three. Additionally, the Kaplan-Meier plot for the three clusters of patients is shown in Fig. 4, where Area Under Curve (AUC) for cluster two (AUC = 296.06) is much smaller than that for clusters one (AUC = 2441.81), and three (AUC = 1302.79). This suggests that patients in cluster two have worse prognosis than those in cluster one and three, although this observation needs to be further validated with a larger number of samples. In Fig. 5, we present the Kaplan-Meier plots of three clusters of patients showing surivals of those treated with either standard and aggressive therapy. The resulting p-values of the Log-Rank tests with patient survivals with regard to response to therapy from cluster one, two and three are 0.00705, 0.158, and 0.000640, respectively. The results suggest that patients in cluster one and three show significantly favorable response to aggressive therapy compared to standard therapy. Cluster two contains a small number of patients and conclusions regarding response to therapy are limited.

Open in a separate window

Fig. 4

Kaplan-Meier plot for patients of three consensus clusters is presented with AUC values and 95% lower and upper confidence bounds.

Open in a separate window

Fig. 5

Kaplan-Meier plots for patients treated with aggressive (blue) and standard (red) therapy from (a) cluster 1, (b) cluster 2, and (c) cluster 3, are presented with 95% lower and upper confidence bounds.

This work was supported by the NCI Contract HHSN261200800001E; TCGA Contract 29 x 55193; NIH 5R01LM009239-04; NHLBI R24 HL085343; and by the Clinical and Translational Science Awards program under PHS Grant UL1RR025008.