Assessment of object-based indices to identify convective organization

Mandorli, Giulio; Stubenrauch, Claudia J.

doi:https://doi.org/10.5194/gmd-17-7795-2024

Articles | Volume 17, issue 21

https://doi.org/10.5194/gmd-17-7795-2024

Articles | Volume 17, issue 21

Methods for assessment of models

06 Nov 2024

Methods for assessment of models |

| 06 Nov 2024

Assessment of object-based indices to identify convective organization

Giulio Mandorli and Claudia J. Stubenrauch

Abstract

The emerging field of convective organization has attracted significant attention due to its potential implications for weather and climate. Numerous indices have been developed to identify organization of convection, serving as essential tools for advancing our understanding in this area. Because of the large number of these indices, many results on convective organization are still uncertain, and different studies have shown diverging results. The present analysis studies and compares nine object-based indices in order to evaluate their ability to quantify organization. The analysis begins by establishing a set of criteria expected for convective organization and subsequently subjecting the indices to assessment against these benchmarks. The criteria are grouped into three categories. The first category tests the robustness of the indices against noise. The second category evaluates their sensitivity to the size and position of the convective objects. The third category assesses their dependency on the specific characteristics of the dataset in use. Among the indices scrutinized, none fulfill all the desired conditions, and some conditions are only marginally satisfied. Therefore, we developed a new index, called the Organization Index based on Distance and Relative Area (OIDRA), as an example of a well-behaving index. The unmet conditions and differences between indices can explain the discord between different organization studies. The results come down to a guideline that will help to advance our description of deep convective organization.

Download & links

Article (PDF, 5857 KB)

Supplement (1041 KB)

Download & links

How to cite.

Received: 30 Aug 2023 – Discussion started: 28 Nov 2023 – Revised: 13 Feb 2024 – Accepted: 15 Sep 2024 – Published: 06 Nov 2024

1 Introduction

Atmospheric convection is a fundamental process characterized by the vertical movement of air masses within the Earth's atmosphere. As the sun heats the Earth's surface, warm air rises, transporting heat and moisture through the atmosphere. This upward motion triggers the formation of clouds and weather phenomena, playing a crucial role in shaping our planet's weather and climate. In radiative–convective equilibrium simulations, convection shows a tendency to cluster horizontally as time passes. This behavior was firstly pointed out by Held et al. (1993), and then it was confirmed in several other studies (e.g., Tompkins, 2001; Bretherton et al., 2005; Wing and Emanuel, 2014). Because of this feature, clustered convection is referred to as aggregated or organized convection, or convective organization. In recent years, because of the great importance of convection on climate, many studies have focused their attention on convective organization, either looking for an explanation of such a phenomenon with simulation (e.g., Wing and Emanuel, 2014; Tompkins and Semie, 2017; Cronin and Wing, 2017; Muller and Romps, 2018; Muller et al., 2022) or trying to measure convective organization in observations and relate it to known quantities (Wing et al., 2017, 2020; Bony et al., 2020; Bläckberg and Singh, 2022; Stubenrauch et al., 2023). Both types of analysis need a method to quantify convective organization. However, quantifying the degree of convective organization is challenging. There is still no consensus on the best method to use, and various methods have been proposed in recent years, reviewed by Biagioli and Tompkins (2023). These organization metrics are often given by a single real number, called an organization index. The recently emerging large number of convective indices and diverging results (Wing et al., 2017; Stubenrauch et al., 2023) ask for a systematic assessment, i.e., statistical studies that verify the robustness of these indices. This is challenging because convective organization does not currently have a rigorous definition. Nevertheless, it is still possible to verify if these indices satisfy certain conditions expected from the metrics of organization. Such studies have so far only been performed for example cases (Retsch et al., 2020; Jin et al., 2022).

Assessing metrics by verifying expected conditions has already been undertaken in other fields, for example, in high-energy physics by Cacciari et al. (2008). This approach is reproduced here to refine our definition of convective organization. Hence, this statistical study assesses nine object-based indices by verifying if they satisfy seven conditions. Only object-based indices are studied in this work, while non-object-based indices are not included. The convective objects have been identified by images of continuous areas of cold infrared (IR) brightness temperature measurements. This assessment is complementary and independent of the assessment of convection tracking methods (e.g., Prein et al., 2023), which have been developed to identify the convective objects.

This article is outlined as follows. Section 2 describes how we reconstruct the convective objects within a selected region from an existing deep convective cloud tracking database. Moreover, Sect. 2 briefly recalls the convective organization indices under study. Section 3 describes the procedure of this study, and Sect. 4 shows the results. Finally, Sect. 5 summarizes the results and discusses a potential outlook.

2 Data and methods

2.1 Datasets of convective objects

The statistical comparison between indices needs a dataset of horizontal binary fields that mimic deep convective clouds for which it is possible to compute the convective organization indices. Since the goal is not to study physical processes but the behavior of the indices, any dataset can be used. However, in order to represent well the typical size, occurrence, and disposition of deep convection in the tropics, we have chosen a real satellite dataset with a good spatial and temporal resolution.

Fiolleau et al. (2020) provide such a dataset with calibrated infrared (IR) brightness temperatures (T_B) by combining different geostationary satellites to span the entire band from 40° N to 40° S. The spatial resolution is 0.04°, and the temporal frequency is 30 min. For this study, we reconstruct convective objects from cold brightness temperatures with a cold core (T_B<190 K) surrounded by T_B<235 K by grouping all eight connected grid boxes. Holes in each object are filled to avoid degenerate dispositions. This procedure is implemented with the Python framework developed by van der Walt et al. (2014). We selected the oceanic Tropical Warm Pool region expanding over 0–9.6° N and 140.4–150° E. The original resolution is downscaled to 0.08° to analyze images with a size of 120×120 grid boxes. Then, images with fewer than two objects are rejected (4.1 % and 2.9 % of the events with no object and one object, respectively). Finally, a total of 76 462 images in the period 2012–2016 are considered for this study.

The following analysis aims to study the behavior of the organization indices, and the results will not be dependent on the dataset used. The robustness of this analysis against the dataset can be proven by comparing the results obtained using different datasets. When comparing datasets, several differences may emerge. Some can be caused by the inherent nature of the datasets, including the shape and spatial distribution of objects, while others arise just from different distributions of object numbers and sizes. The primary focus of this work is to address the former, as they bear a more direct relevance to the intrinsic concept of convective organization. To prove the reliability of the results presented here, we have simulated a dataset to compare with the convective object dataset obtained from cold brightness temperatures. Therefore, we have built images of randomly placed circular objects of different sizes. We used a Monte Carlo simulation technique which follows distributions of object sizes and the number of objects with the same shape as the ones of the convective object dataset from cold T_B. Examples of images generated with this method are given in the Supplement. Despite the large differences in shape and spatial distribution of the objects in the two datasets, the final results are similar, meaning that they do not depend on the nature of the objects. The results obtained with the brightness temperature database are shown in the following, while the ones obtained with the newly simulated dataset are shown in the Supplement.

In order to provide a context for this study, the frequency distribution of the number of objects and their total area is shown in Fig. 1. For both datasets, the frequency of the number of objects decreases, while the total area of the convective systems increases with an increasing number of objects. Examples of the analyzed images are given in the following sections.

https://gmd.copernicus.org/articles/17/7795/2024/gmd-17-7795-2024-f01

Figure 1Distributions of events and total area of convection as a function of the number of objects for both the brightness temperature dataset and the simulated dataset. The horizontal size of the domain is 0.92 Mm².

Download

2.2 Indices of convective organization

The indices of organization try to measure the degree of organization in a determined domain. These indices are computed by using the size and position of the objects described in the previous section. Using the same data is crucial for performing a comparison because any differences in the organization values come entirely from differences in the indices' behavior.

The index I_org was conceptually developed by Weger et al. (1992) for the study of cumulus cloud fields. They compared the cumulative distribution of the object's nearest neighbor distance (NNCDF) of the cloud disposition with the one of a random disposition. Later, Tompkins and Semie (2017) exploited the idea of comparing the two NNCDFs to extract one single value that discriminates between random and clustered clouds. The I_org is a real number between 0 and 1. Values larger than 0.5 are associated with organized convection, while values of I_org≲0.5 indicate disorganized convection. The comparison between nearest neighbors makes I_org insensitive to organization beyond the β-mesoscale (∼ 100 km). Therefore, Biagioli and Tompkins (2023) developed a new index, called L_org, that is sensitive to all scale organization. L_org is computed by comparing the theoretical and observed distributions of all paired convective object distances. The index L_org is dimensionless: it is zero for random disposition, and it is positive for organized convection. Both I_org and L_org do not take into account the size of the objects.

As a consequence, White et al. (2018) developed the Convective Organization Potential (COP) by assuming that 2D objects that are larger and closer together are more likely to interact with each other on the horizontal plane. Its concept reproduces the gravitational potential, and it is determined from the distance between the centers of the objects and the radii of equal-area circles A scalar value is associated with each unique connection between pairs of objects, and the arithmetic average is computed over all the pairs. Recently, a modification of COP, called area-based COP (ABCOP), was proposed by Jin et al. (2022) in order to improve the dependency on the object's area and number. Both COP and ABCOP are positive and dimensionless, and higher values represent higher degrees of organization.

The Radar Organization Metric (ROME) (Retsch et al., 2020) was built, similarly to COP and ABCOP, on a sum of scalar values associated with all unique object pairs. Originally, ROME was developed to quantify organization on small domains with high resolution. However, it has also been applied to larger domains (Bläckberg and Singh, 2022; Stubenrauch et al., 2023). The value of ROME is always positive and between 1 and 2 times the object's mean area. ROME is measured in square kilometers (or, equivalently, in number of grid boxes), and its value is large for a high degree of organization.

The Simple Convective Aggregation Index (SCAI) was introduced by Tobin et al. (2012) and is based on the number of and distance between convective objects within a domain. This index was used in studies by Tobin et al. (2012, 2013) and Stein et al. (2017). Since SCAI does not consider the object sizes, the Modified Convective Aggregation Index (MCAI) was proposed by Xu et al. (2019) to correct this feature. These indices are unitless, and they are inversely proportional to the total number of grid boxes in the domain. Thus, their values scale with the size of the image under consideration, as for ROME. SCAI and MCAI are the only indices that identify organized convection with low values; therefore, in our study, we have negated SCAI and MCAI for an easier comparison with the other indices. Consequently, both negated SCAI and MCAI are always negative, and their values are high (close to zero) for organized objects and low for disorganized ones.

The Morphological Index Of Convective Aggregation (MICA) (Kadoya and Masunaga, 2018) is the only index that does not consider the relative object's disposition, but it considers the amount of space on the domain where no convection occurs.

Along with all the abovementioned organization indices, the degree of organization has also been estimated using just the total area of convection (Tan et al., 2015; Bao et al., 2017), and they are included in this comparison. The number of convective objects within the domain has also been used to quantify convective organization (Tobin et al., 2013; Bläckberg and Singh, 2022). However, in this paper, it is not explicitly presented, since the associated results closely resemble those obtained with SCAI and MCAI.

The study of the above indices led us to the development of a new index of organization. This index is called the Organization Index based on Distance and Relative Area (OIDRA), and it is studied in the next section together with all the other indices.

The exact definitions of all indices are given in Appendix A1. All the introduced indices have the same goal: to identify organized convection and discriminate it from unorganized convection. However, as shown by the correlations in Table 1, the indices do not give a coherent answer.

The correlation coefficients between these indices are often smaller than 0.5, and sometimes they are even negative. This means that the estimated strength of organization depends on the index. Hence, the choice of the organization index may strongly affect the conclusions of climate studies of deep convective organization, in particular, when the indices used are anti-correlated.

Table 1 also presents the correlations between the above indices with three important variables that characterize the convective systems within the studied domain: the mean size of the convective objects ( $\sum_{i} A_{i} / N$ ), their number (N), and the total convective area covering the domain (∑_iA_i). Before proceeding with the analysis, we highlight the following correlations:

ROME is completely correlated to the mean size of the objects. Thus, ROME reflects the mean object size entirely.
SCAI and MCAI are strongly correlated to the number of objects.
ABCOP strongly correlates to the total area of convection covering the studied domain.

All the other indices show weaker relationships with these variables. The dissimilarities between the correlations presented in Table 1 motivate this work. They highlight the challenge of studying convective organization, and they show that different indices may consider different aspects of convective organization.

Table 1Correlation coefficients, multiplied by 100, of the indices with each other and with the number, total area, and mean size of convective objects. Bold numbers highlight correlations with coefficients larger than 0.5.

Download Print Version | Download XLSX

3 Analysis strategy

3.1 Conditions to be satisfied

The studied indices were designed with the purpose of quantifying the degree of deep convective organization. Estimating their skill is challenging because organization has no formal definition. However, we can assess if the indices satisfy some expected conditions of convective organization. We have chosen seven conditions, which can be grouped into three categories and will be evaluated via sensitivity studies.

The first category concerns the behavior of the indices towards small perturbations. The indices should not significantly change

1.
when one random grid box in the domain is set to convective,
2.
when two objects merge into one by adding one single grid box as convective.

Both conditions are meant to study the effect of noise. To simulate the difference in behavior, in the first case, we randomly add one grid box in a non-convective region of the image; thus the number of objects increases by one. In the second case, we add the convective grid box between two close objects in such a way that the objects merge and the number of objects decreases by one. The indices that satisfy the criteria above are called noise-safe.

The second category evaluates the intrinsic behavior of the indices. The indices should

3.
decrease when objects are moving apart,
4.
increase when one object's size is increasing.

Condition (3) is the most relevant one of all because it defines the relationship between the proximity of the objects and the organization. The indices that do not satisfy it should not be used to quantify organization.

Condition (4) states that the organization gets stronger when any object area is increasing. This condition is not explicitly taken into account in all convective indices. ROME, COP, ABCOP, and MICA are built with this assumption, while I_org, L_org, SCAI, and MCAI do not consider the area of convective objects in their formulation. If this assumption is not satisfied, one potential approach to study organization is to stratify the images by the total area of convection. This approach is marginally performed by Tobin et al. (2012), where the values of organization are compared within similar intervals of total precipitation. Either way, if condition (4) will prove to be important or if it will be considered to be irrelevant, we evaluate it for all the indices in order to provide awareness of the relationship between the indices and the area of convection.

The third category evaluates the dependency on the resolution and on the domain limits. This set of conditions is crucial to compare conclusions from analyses of organization using different data. The indices should not significantly change

5.
when using a slightly different spatial resolution,
6.
when using data taken at a slightly different time,
7.
when choosing a slightly different spatial domain.

3.2 Comparison strategy

In order to quantify the dependence of the indices on the specific perturbance under consideration, we modify the images accordingly: for each image, we compute the value i of each index (reference), then we modify the image as required by the condition under study, and we compute the new value of the index i^′ (perturbation). The difference $i^{'} - i$ shows how much the value of the index has changed. However, the organization strength is given by ranking the value i: for example, extremes are defined by percentiles. Therefore, instead of comparing the absolute change in the values of the indices i, we compare their percentiles p(i). Let f(x) be the measured distribution function of a certain index; the percentile of the value i is defined by

\begin{matrix} (1) & p (i) = 100 \int_{- \infty}^{i} f (x) d x, \end{matrix}

where the factor 100 is set to have values from 0 to 100. A value of p(i)=90 means that the convection is in the 10 % most organized disposition.

Using percentiles is particularly advantageous because the difference $Δ p = p (i^{'}) - p (i)$ really quantifies the amount of change in organization estimated by the indices. For example, a difference Δp=10 means that the perturbation results in a jump in the organization of 10 % of the events. Since Δp is dimensionless, it can be compared between different indices. Consequently, we quantify changes in each index with Δp. In the following, an average value of Δp<3 is considered small, while an average value of Δp>10 is considered large.

We have found that it is very difficult for a single index to satisfy all the presented conditions. In particular, as shown in the next section, none of the considered indices does that. Hence, we developed a new index, called OIDRA.

4 Results

4.1 First category: noise-safeness

In this section, we study conditions (1) and (2). Condition (1) states that the consequences on the indices must be small when one grid box is added as a new convective object. The consequences of adding a single convective grid box are also shown by Retsch et al. (2020) and Jin et al. (2022). However, only a few cases have been examined. Instead, this work provides a statistical picture that validates previous results, proving that those cases were not cherry-picked.

https://gmd.copernicus.org/articles/17/7795/2024/gmd-17-7795-2024-f02

Figure 2(a) Convective objects in the domain of interest are shown in gray for 4 January 2012 at 12:30 UTC; the additional convective grid box is in red, and it shows the perturbation associated with condition (1). (b) Convective objects in the domain of interest for 1 January 2012 at 21:00 UTC; the additional grid box is in red, and it shows the perturbation associated with condition (2).

Condition (2) states that, if there are two objects as close as one grid box, merging them does not produce significant changes in the value of the indices. The validity of this condition stems from the fact that the presence of an additional grid box does not substantially alter the disposition of convection. Consequently, it is expected that the inclusion of this grid box would not significantly affect the values of the organization indices. Condition (2) has never been discussed in studies of organization. Still, it is of great importance because, when two objects merge, the sizes, number, and distances between objects change, thus changing the value of the indices. Therefore, both conditions (1) and (2) contribute to comprehending the potential impacts of any source of noise on the organization indices.

Figure 2 illustrates the perturbations implemented for the two conditions. For the sensitivity study (1), the additional grid box is always placed randomly and it is never in contact with other objects. For study (2), the additional grid box always merges two objects. The perturbation shown in Fig. 2b is only possible when there are two objects as close as one grid box. Therefore, only those events are used to evaluate condition (2) (26 255 events out of 76 462).

https://gmd.copernicus.org/articles/17/7795/2024/gmd-17-7795-2024-f03

Figure 3Distribution of I_org (a) and COP (b) for both the reference and the perturbed dataset. Percentile distribution of I_org (c) and COP (d). Bidimensional distribution of p(I_org) (e) and p(COP) (f) of the reference and the perturbed dataset. The red and gray arrows indicate values and percentiles of I_org and COP, respectively, for the configuration in Fig. 2a with and without the additional grid box.

Download

The procedure is similar for all sensitivity studies: for each image, the indices of organization are computed (labeled as “reference”), then the associated perturbation is applied and the indices are computed on the new image. The set of images after the perturbation is labeled as “perturbed”. As an illustration, Fig. 3a and b compare the distributions of I_org and COP before and after the perturbation of condition (1) using the complete statistics. As expected, the random noise produces on average a decrease in both I_org and COP. However, for I_org, a non-negligible fraction of events close to zero is redistributed at higher values. Those events are mainly images with only two objects that are placed at opposite sides of the image (not shown). Therefore, the additional grid box reduces the nearest neighbor distances and increases the value of I_org. Figure 3c and d show the distributions of the percentiles for I_org and for COP. The percentiles are always computed with respect to the reference distribution; thus the distributions of the reference dataset are flat, while the ones of the perturbed dataset are not.

The gray and red arrows indicate the values of I_org and COP for the event shown in Fig. 2a. The values of I_org and COP move from 0.80 to 0.70 and from 0.32 to 0.28, respectively. The corresponding percentiles move from 0.91 to 0.78 and from 0.14 to 0.06, leading to differences of Δp(I_org)=13 and Δp(COP)=8. This means that one additional grid box, even if it is just 1 out of 120², changed the rank of the value of I_org by 13 % and the rank of COP by 8 %. It is also worth noting that the case depicted in Fig. 2a is classified as highly organized according to I_org, whereas it is classified as highly disorganized according to COP.

The distributions in Fig. 3e and f present the relationship between the percentiles of the reference and of the perturbed dataset for I_org and for COP. The two figures are similar, and they show that both indices decrease on average. Thus, the new convection dispositions are correctly identified as less organized. Moreover, the vertical spread of the distribution comes from the randomness of the additional grid box position. Figure 3 proves that the behaviors of I_org and COP under noise are similar even if their reference distributions are very different.

The results of the sensitivity study for all the indices and for both conditions are shown in Fig. 4 as the distributions of Δp. The difference Δp produced by the perturbation of condition (1) has a long tail for I_org, L_org, COP, and MICA. A long tail means that one single grid box can heavily modify the index value. The other indices exhibit a narrower peak around zero. For SCAI and MCAI, the Δp distributions derive mostly from the average number of objects. In the other cases, the enhanced peak at zero occurs because the indices are closely related to the total area of the objects in the domain; thus adding one single grid box does not affect their value significantly. Under perturbation (2), all the indices peak at zero, proving that merging two close objects does not affect their value much. However, I_org, L_org, COP, and ABCOP show a significant tail at negative Δp.

https://gmd.copernicus.org/articles/17/7795/2024/gmd-17-7795-2024-f04

Figure 4Distribution of Δp for all the indices (a) for condition (1) and (b) for condition (2).

Download

The most frequent sign of Δp is very important because it indicates if the indices generally classify the perturbed dataset as more or less organized. In most cases, the sign is negative for perturbation (1), meaning that the organization is reduced. The only index that classifies the perturbed dataset as more organized is ABCOP, which is in contrast with the other organization indices. In particular, adding one convective grid box always produces an increase in the value of ABCOP. For perturbation (2), I_org, L_org, and ABCOP predict a decrease in organization, while SCAI, MCAI, COP, ROME, MICA, and OIDRA predict an increase after the perturbation.

In order to quantify the average change in the values of the indices, we report in Table 2 the average of the absolute percentile difference $〈 | Δ p | 〉$ for each index under study. Large values reveal a higher sensitivity to noise, while low values indicate noise-safe indices. The indices ABCOP and OIDRA do not change significantly under perturbation (1). Under perturbation (2), the indices MICA and OIDRA do not change significantly, while I_org exhibits large changes.

Table 2Average of the absolute change in percentile $〈 | Δ p | 〉$ after the perturbations of condition (1) and condition (2).

Download Print Version | Download XLSX

For all indices and for both conditions, Δp is influenced by the average and distribution of the number of objects of the reference dataset: images with a larger number of objects show less sensitivity to noise. Therefore, we report the values of $〈 | Δ p | 〉$ as a function of the number of objects in the reference dataset in Fig. 5. For both perturbations, the sensitivity of all indices decreases as the number of objects increases. As a result, each index becomes noise-safe when a sufficient number of objects is present. The index I_org is very sensitive to noise at a low number of objects, and it becomes noise-safe when more than 35 objects are present. This result is in agreement with Semie and Bony (2020), who raised a similar statement of robustness for I_org. The index L_org has a similar behavior to I_org for a low number of objects, but it becomes more robust to noise as the number of objects increases. The sensitivities to noise of SCAI and MCAI follow the distribution of the number of events shown in Fig. 5 because of the high correlation between them. Generally, the indices that take into account the area of the objects become noise-safe more rapidly than the ones that do not. Among the various methods available to quantify convective organization, ROME, OIDRA, and the total convective area are demonstrated to be the most robust against noise.

https://gmd.copernicus.org/articles/17/7795/2024/gmd-17-7795-2024-f05

Figure 5Sensitivity of each index as a function of the number of objects in the reference dataset. Perturbation (1) is shown in panel (a), and perturbation (2) is shown in panel (b).

Download

4.2 Second category: intrinsic behavior

In this section, we study conditions (3) and (4). Condition (3) states that the degree of organization must increase with the proximity of the objects. This condition is the most important one, and no index of organization can miss it because it encapsulates the essence of convective organization. Condition (4) states that the degree of organization must increase when the object sizes are increasing. The sensitivity study associated with this condition distinctly presents the role of the object sizes for each index.

The procedure is similar for conditions (3) and (4). Firstly, every image in the dataset has been extended rightward by 50 grid boxes with an empty space. Then, an object called the “test object” is placed in the extension. For condition (3), the test object is as big as 10×10, it is placed in the leftmost part of the extension, and the perturbation consists of moving it rightward up to the end, as shown in Fig. 6a. The perturbation is parametrized by the magnitude of the displacement. For condition (4), the test object has a square shape of 2×2, it is placed in the center of the extension, and the perturbation consists of increasing its size up to 48×48, as shown in Fig. 6b. The perturbation is parametrized by half the length of the object side. The reference configuration is the one where the test object is present and is not perturbed, and all the other configurations are compared to that one.

https://gmd.copernicus.org/articles/17/7795/2024/gmd-17-7795-2024-f06

Figure 6Convective objects in the domain of interest, similar to Fig. 2. Perturbations of condition (3) and condition (4) are shown in panels (a) and (b), respectively, for 1 January 2012 at 13:30 UTC. The leftmost part shows the initial image. The rightmost part shows the extension of 50 grid boxes, where a test object is added. In panel (a), the extra object has a size of 10×10 and it is shifted rightward. In panel (b), the extra object is centered and its size is increased.

Figure 7a and b show the distribution of Δp(COP) as a function of the perturbation of both conditions. The two figures show that COP has the correct trends for both perturbations: COP decreases when the test object is moving away, and it increases when the test object size is increasing. In addition, the high correlation with the reference dataset proves that the value of COP is still linked to the reference value after the perturbations. The mean 〈Δp〉 is reported in Fig. 7c and d for all the indices as a function of the perturbation of both conditions. In Fig. 7c, all the trends are negative; thus all indices correctly increase with the proximity of the objects. However, the trends have different magnitudes: ROME and ABCOP have very little dependency on the test object position. Conversely, MICA shows a very large dependency on the object position. Such a large value is due to the specific definition of MICA (see Appendix A8) and to the strategy used to evaluate condition (3): the perturbation affects A_cld; thus it proportionally affects the value of MICA. The index I_org exhibits a rapid decrease initially, followed by stabilization, because it is not sensitive to organization beyond the β-mesoscale. This behavior has been corrected in L_org, which looks to be sensitive to the object's position across all distances. Even if this condition should be satisfied by construction, it has never been validated before this work, and it is always taken for granted.

Figure 7d displays the behavior of the indices when the test object's size is increasing. The left part of Fig. 7d corresponds to the increase in a small object, whereas the right part corresponds to the increase in a large object. We observe different behaviors. Firstly, SCAI, I_org, and L_org are constantly zero because their values do not depend at all on the object sizes. Then, ROME and COP strongly increase, the latter linearly, while ABCOP, MICA, and MCAI only have a small increase. Lastly, OIDRA first decreases and then increases with respect to the object’s side. Therefore, OIDRA shows a larger organization when a big object increases, while it indicates a smaller organization when a small object increases.

https://gmd.copernicus.org/articles/17/7795/2024/gmd-17-7795-2024-f07

Figure 7Distribution of Δp(COP) as a function of perturbation (3) (a) and perturbation (4) (b). The gray boxes indicate the percentile range from 30 % to 70 %, the colored boxes indicate the percentile range from 20 % to 80 %, and the colors display the correlation between the reference and the modified dataset. The whiskers cover 10 % to 90 % of the distributions. The means and the medians of Δp(COP) are shown by the rhombuses and the black lines, respectively. The means of Δp are displayed for all the indices as a function of the perturbations of condition (3) (c) and condition (4) (d).

Download

4.3 Third category: capacity to compare across diverse datasets

Having multiple independent studies that agree with each other is crucial for advancing scientific understanding. It shows that the results are reliable and can be trusted. In the specific case of convective organization, those independent studies can be performed by analyses that use different datasets. Therefore, it is crucial to ensure that the indices of organization do not depend significantly on the dataset characteristics. Otherwise, achieving consensus among different studies becomes exceedingly challenging. Conditions (5), (6), and (7) are meant to evaluate the possible dependency of the indices on the resolution and domain limits. In the following, each of them is discussed separately.

4.3.1 Condition (5): sensitivity to the horizontal spatial resolution

The datasets that are used to study convective organization may be different; in particular, the horizontal resolution used in the various analyses may vary by up to 1 order of magnitude. It is thus of great importance to understand the role of horizontal resolution on the organization indices. In this section, we evaluate condition (5), which states that the horizontal resolution of the dataset should not significantly influence the value of the organization indices. The role of the resolution is studied by downscaling each image of factors 2, 3, 4, 5, and 6. In other words, the images, which are initially 120×120, are reduced to 60×60, 40×40, 30×30, 25×25, and 20×20 grid boxes. The reduction is performed by grouping the initial images' grid boxes in blocks of n×n, then the mean values over the blocks are computed where the values 1 and 0 are associated with convective and non-convective regions. If the mean value is larger than 0.5, the block is set to 1 (i.e., it is associated with convection); otherwise, if the mean value is smaller than or equal to 0.5, the block is set to zero (it is not associated with convection). An example is given in Fig. 8, comparing an image with the original resolution of 120×120 grid boxes and an image with the resolution downscaled by a factor 3 so that it consists of 40×40 grid boxes. The set of images with the original resolution is considered to be the reference sample, and the images with different resolutions are the perturbed datasets.

https://gmd.copernicus.org/articles/17/7795/2024/gmd-17-7795-2024-f08

Figure 8Convection in the domain of interest on 1 January 2012 at 02:00 UTC with the original resolution (a) and with the coarse-grained resolution reduced by a factor of 3 (b).

https://gmd.copernicus.org/articles/17/7795/2024/gmd-17-7795-2024-f09

Figure 9Distribution of Δp for all the indices after downscaling the resolution by a factor of 3.

Download

When the grid box size is rescaled, the value of each index may change correspondingly, depending on the index units. ROME and the total area of convection have a dimension of square kilometers; therefore the area per grid box has to be corrected when the resolution is changed. SCAI and MCAI are inversely proportional to the number of grid boxes; thus they quadratically increase with the resolution downscaling. All the other indices are dimensionless, and they do not have any explicit dependency on the resolution. These different scaling factors are removed by multiplying the indices by the inverse factor so that, in principle, the average value of the indices should not change. Such an operation is crucial to studying differences between the reference and the modified dataset.

Figure 9 shows the distribution of Δp for all the indices under a change in resolution of a factor of 3. It is noteworthy that ROME, SCAI, and MCAI have a clear peak at zero, proving that the artificial scaling described above is correct and that we understand the resolution's dependency on the indices well. The distributions of Δp exhibit a generally broad range, indicating the significant impact of the resolution on the values of the indices. The most important effect of downscaling the resolution is due to the smaller objects in the domain (the ones with sizes of one or two grid boxes). The smaller objects can be lost (if they are isolated) or can be merged with bigger objects nearby, and, in both cases, the consequences can be significant. These effects are particularly pronounced for I_org: the disappearance of the small objects produces an increase in the nearest neighbor distances and consequently a decrease in the value of I_org. Conversely, COP is influenced by the size of the objects; thus its value increases with lower resolutions. The distributions of I_org and COP are shown in Fig. 10 as a function of the resolution downscaling factor as described in Sect. 4.2. When the resolution reduction does not change the reconstructed number of objects, the value of Δp is close to zero for all the indices.

https://gmd.copernicus.org/articles/17/7795/2024/gmd-17-7795-2024-f10

Figure 10Distribution of Δp(I_org) (a) and Δp(COP) (b) as a function of the resolution scale factor. The description is similar to Fig. 7.

Download

The mean of Δp is shown in Fig. 11a for different resolutions of the indices. When the resolution scale factor is small, the values of 〈Δp〉 are close to zero, and they move away from zero when the scale factor increases. The sign of 〈Δp〉 is different for different indices: it is negative for ABCOP and I_org, and it is positive for all other indices. The index which shows less sensitivity to different resolutions on both spread (not shown) and average of Δp is OIDRA.

The most important information for comparing the datasets with different spatial resolutions is given in Fig. 11b. The figure shows the correlations between the reference dataset and the datasets with lower resolutions. They vary with respect to the originally observed values, decreasing monotonically with increasing scaling factor. The total area is not much affected by the change in resolution, and, along with it, the indices more related to the total area are also the ones less sensitive to changes in resolution. The indices which are more affected are I_org, L_org, and COP. The correlation of I_org drops to 0.5 when the resolution is downscaled by a factor of 3. The index which has a higher correlation between different resolutions is OIDRA.

https://gmd.copernicus.org/articles/17/7795/2024/gmd-17-7795-2024-f11

Figure 11Means of Δp (a) and correlations with the reference (b) are displayed for each index as a function of the resolution scale factor.

Download

The presented results are of great importance for comparing the results of different studies. Different analyses often have different resolutions; thus, they can have discrepancies even if they make use of the same organization index. This effect has to be summed up with other sources of uncertainties, like noise or different thresholds as remarked upon in Sect. 4.1, and the resulting discrepancies can be considerable. In particular, they can be very important when I_org is used, and they can lead to different results even in similar analyses.

4.3.2 Condition (6): sensitivity to observation time

The time variability of the organization index has already been shown (Tompkins and Semie, 2017; Cronin and Wing, 2017; Muller and Romps, 2018; Muller et al., 2022), and seasonal and diurnal cycles of organization have been studied using satellite observations (White et al., 2018; Stubenrauch et al., 2023). In most analyses, convective organization indices are computed per snapshot and then averaged over long periods, while small variability is not examined. In this section, we analyze the variability in the indices on a small timescale, evaluating condition (6). In principle, the daily cycle can affect all the results of this analysis because we are comparing the indices at different times. Therefore, the effect of the daily cycle has been calculated. Since it is found to be negligible for the considered region, its effect is not shown.

https://gmd.copernicus.org/articles/17/7795/2024/gmd-17-7795-2024-f12

Figure 12Autocorrelation of each index between 30 min and 12 h.

Download

The variability with time for each index is summarized in Fig. 12, which shows the autocorrelation of the indices as a function of the time shifts from 30 min to 12 h. The autocorrelations have large values for small differences in time, while they decrease with increasing shifts. There are large differences between indices: COP, I_org, and L_org have a very rapid decrease, with correlations already lower than 0.5 for COP and L_org and 0.4 for I_org after 1 h. This means that these indices cannot be compared when computed with more than 1 h distance and that they cannot be related to any physical variable which has different timescales. The autocorrelation of ROME, MICA, and OIDRA is close to 1 for a small time, while it decreases toward zero with timescales of a few hours. Lastly, the autocorrelation of SCAI, MCAI, and ABCOP has a slower decrease, and it is still about 0.5 after 12 h. All the autocorrelations reach zero after a few days.

In order to further explore the fluctuations in the indices with time, the differences $〈 | Δ p | 〉$ are given in Table 3 for time shifts of 30 min, 90 min, 12 h, and 6 months. The table shows that $〈 | Δ p | 〉$ increases with time for all the indices and that the increase rate can differ depending on the specific index. The differences $〈 | Δ p | 〉$ also reflect the autocorrelation values and trend: where the autocorrelation is lower, $〈 | Δ p | 〉$ is larger. For very large time shifts, the autocorrelation is zero, and $〈 | Δ p | 〉$ moves towards the limit 33.3 (not shown); the exact value does not depend on the atmospheric dynamics, but it depends on the amplitude of the seasonal cycle, the daily cycle, and on whether there are any points of accumulation in the distribution of the indices. The values of $〈 | Δ p | 〉$ are very large for COP, I_org, and L_org, and they are already larger than 10 when the time shift is 90 min (except for MICA, which is 9.6). This means that the organization ranking given by indices is moving on average by more than 10 % every 90 min. After a time shift of 12 h, the differences $〈 | Δ p | 〉$ are larger than 20, and, in particular, they are close to 33.3 for COP, I_org, L_org, and OIDRA in agreement with the low correlation shown in Fig. 11.

Table 3Average of the absolute change in percentile $〈 | Δ p | 〉$ due to different shifts in time. The last row shows $〈 | Δ p | 〉$ for a random shuffle of the dataset.

Download Print Version | Download XLSX

This analysis can be of help in two cases. Firstly, when two studies employ the same organization index but compute it with a time lag, the two indices may exhibit a weak correlation, making direct comparisons challenging. Secondly, when organization is linked to other atmospheric variables that are measured with a time delay (e.g., from a different instrument), the rapid variability in organization may cause the relationship to weaken or disappear due to the time lag. By considering these cases, this analysis helps to account for temporal differences and enables a more accurate understanding of the relationship between organization and other variables in atmospheric studies.

4.3.3 Condition (7): sensitivity to the domain limits

Convective organization is computed from the disposition of convection in a specific domain. Different studies may not target exactly the same domain, leading to possible differences in their final results. In this section, we analyze condition (7), which states that organization indices must not be significantly different on similar domains. In particular, we analyze the changes in the indices of organization for domains that overlap from 80 % to 99 %. Such a study is performed by taking into consideration the region 0.8–8.8° N × 140.4–148.4° E made by 100×100 grid boxes with 0.08° resolution. This region is shifted eastward one grid box at a time, and, for each shift, the indices of organization are computed. The shift of one grid box eastward changes the domain by 1 %.

https://gmd.copernicus.org/articles/17/7795/2024/gmd-17-7795-2024-f13

Figure 13Distribution of Δp(I_org) (a) and Δp(COP) (b) as a function of the perturbations of condition (7). The description is similar to Fig. 7.

Download

Figure 13 shows the distributions of Δp for I_org and for COP as a function of the eastward shift in the domain. The distributions of Δp are very narrow for small shifts, and they get broader with larger shifts. The mean and the median are very close to zero with no trends, and the correlation decreases with the magnitude of the shift. The variability in both Δp(I_org) and Δp(COP) is similar, as is the correlation with the reference dataset.

The correlations of the indices between the shifted and the initial domains are shown in Fig. 14. They are large for small shifts, and they slowly decrease as the domain moves eastward. In particular, for small shifts, the trend is about 0.01 per grid box, and, since a one-grid box shift corresponds to 1 % of the image, the correlations decrease at a rate of approximately 0.01/%. For large shifts, the indices are divided into two groups: SCAI, MCAI, ROME, and ABCOP decrease slowly, while COP, OIDRA, MICA, and I_org show a more rapid decrease.

The results obtained with this analysis prove that the indices of organization have small differences when similar domains are considered. The larger the differences between domains, the larger the discrepancies of the indices are. Thus, all the indices show a small rate of change under the shifting of the domain, proving that all the indices satisfy condition (7).

https://gmd.copernicus.org/articles/17/7795/2024/gmd-17-7795-2024-f14

Figure 14Correlation of each index between the reference and the modified datasets as a function of the domain shift.

Download

5 Conclusions

Convective organization is an emerging topic that has received a lot of attention in recent years because of its potential implications for weather and climate. Several indices have been developed to identify organization, and they became an essential tool to deepen our knowledge on this topic. In this study, we assessed the reliability of each index by employing innovative methodology. Initially, we established a set of criteria that are anticipated for convective organization. Subsequently, we evaluated the indices to ascertain whether they met these established conditions.

In this article, we compare nine object-based indices. The conditions are assessed by applying different perturbations to the dataset and measuring the repercussions on the indices. This assessment considers objects of the size of mesoscale convective systems defined by cold brightness temperature.

In the previous sections, the indices are compared for each condition. In the following list, we summarize the results for each index, combining the different conditions to obtain the global picture that emerges from the results.

I_org: The index I_org does not fulfill several of the studied conditions. First of all, it is very sensitive to noise. Secondly, when two close objects are merged, I_org has large changes, and, on average, it predicts the new configuration to be less organized. In particular, the index I_org is not noise-safe when the number of objects in the domain is less than 35. Furthermore, I_org is not sensitive to the objects' sizes. Another weakness of I_org is its large sensitivity to the images' resolution. In fact, identical images with different resolutions produce significantly different values of I_org; the differences are in both the mean values and the distributions. Lastly, the time autocorrelation is decreasing rapidly; thus the relationship with other atmospheric variables may be lost if it is not searched exactly at the right time.

The abovementioned behaviors of I_org are obtained using convective object reconstruction. However, since I_org does not consider the size of the objects, some studies (e.g., Semie and Bony, 2020; Bony et al., 2020) applied it to local minima in brightness temperature, which may be seen as a proxy for the convective core positions. Such an approach may modify the behavior of this index. Identical considerations apply to L_org.
L_org: The behavior of L_org is similar to that of I_org, but its behavior improved for all the studied perturbations. Firstly, it is more robust to noise than I_org: the index L_org becomes noise-safe when there are more than 20 objects in the domain. Secondly, it is less sensitive than I_org to the spatial and temporal resolutions and to the domain limits. Moreover, L_org shows a dependency on the object's position across all distances, confirming its sensitivity to organization beyond the β-mesoscale. Finally, both I_org and L_org are not sensitive to the object's size.

Equivalently to I_org, the present results hold only when convective objects are reconstructed.
COP: The index COP, similarly to I_org, is sensitive to noise, and it becomes noise-safe when more than 25 objects are present in the domain. It increases with the proximity of objects, and it correctly increases with the objects' size. On the other hand, COP is sensitive to the images' resolution. In particular, the same image with the original resolution and a 3 times worse resolution gives values of COP that have a correlation of 0.65. For this reason, COP is not a good index for comparing different studies. Lastly, similarly to L_org, COP varies very rapidly with time.
ABCOP: The index ABCOP has a large correlation with the total objects' area ∑_iA_i. Therefore, it mostly reflects the total objects' area. This relationship uniquely comes from the larger objects in the images, while the smaller objects do not play a significant role in the value of ABCOP. This behavior comes from the ${max}_{j \neq i} ()$ function in the ABCOP definition (Eq. A8), which gives great importance to large objects. As a consequence, ABCOP is not very sensitive to noise. However, it strongly depends on the number of objects because it is defined as a sum over each object instead of as a mean like COP. This characteristic is visible in two of the studied conditions, and it negatively influences ABCOP response to noise. Firstly, when one convective grid box is added randomly in the domain, ABCOP incorrectly increases instead of decreasing because the number of objects increases. Secondly, when two close objects are merged, ABCOP incorrectly decreases instead of increasing because the number of objects decreases. In addition, ABCOP follows the behavior of the number of objects even under a change in horizontal resolution. ABCOP proves to be robust under changes in resolution, shifts in time, and shifts in the considered domain. Last but not least, the index ABCOP increases with the proximity of the objects but more slowly than the majority of the indices.
SCAI and MCAI: The indices SCAI and MCAI have similar behaviors. Their value is highly correlated with the number of objects ( $R = - 0.96$ ¹); thus they do not offer much more information than the number of objects itself. Hence, the behaviors of SCAI and MCAI reflect that of the number of objects. Upon adding a single random convective grid box to the images, both SCAI and MCAI show a decrease in their value. Furthermore, these indices indicate an increased level of organization when two nearby objects are merged. Both indices correctly exhibit an increase in value as the proximity of the object increases, with MCAI being slightly more sensitive than SCAI. The main distinction between these two indices lies in their dependence on the object's size. SCAI displays no sensitivity to object size, whereas MCAI takes into account the object's size, thereby rectifying SCAI's wrong behavior. Changes in resolution have an impact on the values of SCAI and MCAI, primarily due to variations in the number of objects. Nonetheless, their correlations with the original resolution decrease slowly, making them suitable indices for comparing different resolutions. Similarly, SCAI and MCAI are the indices that are less affected by shifts in time and space.
ROME: The index ROME satisfies most of the conditions presented here; however, its biggest problem is the very high correlation with the mean objects' size $\sum_{i} A_{i} / N$ as shown in Table 1. Thus, it is not convenient to quantify convection with ROME (Eq. A9) when a simpler formula can be used to obtain a very similar value. ROME is noise-safe enough, having values of Δp smaller than 3. Knowing the correlation with the mean area, we can state that those conditions are satisfied because of the dataset instead of the index behavior. Among all indices, ROME is the least sensitive to the proximity of the objects, with a variation of only Δp≲0.3 when a 10×10 test object is moving away from the other convective objects. Moreover, ROME is highly dependent on the size of the objects. The dataset's horizontal resolution does not strongly affect the ROME index, and the autocorrelation shows a slow decrease with time. The changes in ROME with the considered domain are about 0.5 % for a 1 % shift in the domain.
MICA: The index MICA is noteworthy because of its unique definition which has to be kept in mind when reading these results and when using MICA for an analysis. The obtained results are very much influenced by the type of data and the adopted strategy. For example, this work suggests that MICA is greatly influenced by the proximity of objects; however, this is not true in general, and it is only true when the test object changes A_cld (defined in Appendix A8). Similarly, the test object size studied in Sect. 4.2 is also influenced by the change in A_cld. One other example is the result given in Sect. 4.1, where MICA predicts a very small increase when two close objects are merged. The simulated noise produces a long tail of Δp, which means that a source of noise can produce a large change. The reason is that MICA does not take into account each object size, and it strongly depends on the disposition of the objects. Thus, if a simulated noise is placed far from the convective objects, the value of MICA decreases significantly. For the same reason, MICA increases when the resolution is reduced because small objects may be lost, reducing the area A_cld. The autocorrelation of MICA decreases rapidly with time, but it is large for small time lags. The changes in MICA with the considered domain are similar to those of the other indices.
OIDRA: The index OIDRA is the most noise-safe index, having percentile differences Δp smaller than 1 for conditions (1) and (2). It correctly depends on the proximity of the objects, and the change in OIDRA is significant for large shifts in the extra object. OIDRA is also the least sensitive to different resolutions. Specifically, even after a change in resolution of 6 times, its values remain correlated at 0.9 with those of the original resolution. The autocorrelation of OIDRA is close to 1 for small shifts in time, and it decreases rapidly, reaching less than 0.2 in 10 h. The correlation of OIDRA with itself during a domain shift is nearly 1 for small shifts, gradually decreasing initially and then exhibiting a more rapid decline for larger shifts. The index OIDRA is the best index according to the conditions studied in this work, except for condition (4). However, it increases when an object increases if the size of that object is larger than the mean size. Conversely, if the size of that object is smaller than the average, OIDRA decreases. This behavior can be avoided by comparing OIDRA within bins of total convective area.

Among the indices considered in this work, none satisfy all the desired conditions. Besides, some of the studied conditions are only poorly satisfied by the indices. Thus, two main conclusions can be extracted from these results. Firstly, the unsatisfied conditions and the different behavior that emerged from this study can explain the disagreement between different studies on organization. Secondly, the indices studied here may not be enough to completely characterize organization, and a more complete metric could be built by simultaneously using more indices, as suggested by Pscheidt et al. (2019) and performed by Janssens et al. (2021).

6 Outlook

The results of this assessment provide the first step to estimating uncertainties in quantifying convective organization. In the following, we discuss possible extensions of this study.

This article focuses on a domain of 10° × 10° because it is comparable to the domain sizes used in cloud-resolving model studies (Tompkins, 2001; Bretherton et al., 2005; Muller and Held, 2012; Wing and Emanuel, 2014; Holloway and Woolnough, 2016). In recent years, several observational studies tried to quantify the convective organization in the entire tropics (Xu et al., 2019; Bony et al., 2020; Jin et al., 2022; Bläckberg and Singh, 2022; Stubenrauch et al., 2023); hence a similar assessment should be performed using the tropics as a domain. Such a domain is far from being a square; thus attention should be given to the influence of domain shape and size. Furthermore, the tropical band is composed of regions with different degrees of convective homogeneity; therefore, it may be interesting to study the influence of the spatial distribution of convection on organization. Other effects to be studied include the smoothing of the data (Bony et al., 2020; Bläckberg and Singh, 2022) and the variables and thresholds used to identify convection (Stubenrauch et al., 2023) because these affect the shape and disposition of the objects.

Appendix A: Definitions of the organization indices

The indices under study are briefly described below. The specific case of a sole aggregate in the domain is not considered. In the following, the number of objects under study is indicated by N, the area of the ith object is A_i, and the distance between the centroids of the ith and the jth object is d_ij. The characteristic domain length is called L, and the total image size is L².

A1 Organization index

The organization index (I_org) (Weger et al., 1992; Tompkins and Semie, 2017) is derived from the comparison of two distributions. Let $\hat{F} (d_{nn})$ be the cumulative distribution of the nearest neighbor distance d_nn of the objects under study. The cumulative distribution of the nearest neighbor distance of point-like objects randomly displaced in an unbounded domain is a Weibull distribution, $F (d_{nn}) = 1 - \exp (- λ π d_{nn}^{2})$ , where λ is the mean number of objects per unit area. The organization index is then defined by

\begin{matrix} (A1) & I_{org} = \int_{0}^{1} \hat{F} (F^{- 1} (x)) d x; \end{matrix}

that is, the area under the curve (F(d_nn), $\hat{F} (d_{nn})$ ) between 0 and 1. The value of I_org is between 0 and 1, and it is close to 0.5 for random distributions.

A2 The index L_org

The index of organization L_org (Biagioli and Tompkins, 2023) is computed by comparing the distribution of all neighbor distances. The cumulative distribution of all neighbor distances is represented by Ripley's function (Ripley, 1976, 1977, 1981):

\begin{matrix} (A2) & K (r) = E (N (b (x, r) x), \end{matrix}

where 𝔼(N(b(x,r) x) is the expectation value of the number of objects in a dist b(x,r) centered in x and with radius r, excluding {x}. Besag's L-function (Besag, 1977) is defined as $L (r) = \sqrt{K (r) / π}$ , and its value for point-like objects randomly displaced in an unbounded domain is L(r)=r. Let $\hat{L} (r)$ be the observed Besag's L-function, and let L^th(r)=r be the theoretical one. The index L_org is

\begin{matrix} (A3) & L_{org} = \frac{1}{r_{\max}} \int_{0}^{r_{\max}} [\hat{L} (r) - L^{th} (r)] d r, \end{matrix}

where r_max is the integration limit, and it has to be equal to the greatest possible distance between objects' pairs.

A3 Simple Convective Aggregation Index

The Simple Convective Aggregation Index (SCAI) (Tobin et al., 2012) is based on the number of objects and the distances between objects. Let D₀ be the geometric averaged distance between each paired object's centroid $D_{0} = \sqrt[\frac{N (N - 1)}{2}]{Π_{i, j} d_{i j}}$ . SCAI is defined by

\begin{matrix} (A4) & SCAI = k \frac{N D_{0}}{L^{3}}, \end{matrix}

where k is a constant. It was originally set to 2000, but this does not affect the result.

A4 Modified Convective Aggregation Index

The Modified Convective Aggregation Index (MCAI) (Xu et al., 2019) is a modification of SCAI that also takes into account the areas of the objects under study. Similarly to SCAI, MCAI is defined by

\begin{matrix} (A5) & MCAI = k \frac{N D_{2}}{L^{3}}, \end{matrix}

where k is a constant and D₂ is the arithmetical average of size-corrected distance between objects

\begin{matrix} (A6) & \begin{aligned} D_{2} & = \frac{2}{N (N - 1)} \sum_{i = 1}^{N} \sum_{j = i + 1}^{N} max (0, d_{i j} \\ - \sqrt{A_{i} / π} - \sqrt{A_{j} / π}) . \end{aligned} \end{matrix}

A5 Convective organization potential

The Convective Organization Potential (COP) (White et al., 2018) is built on the concept of an interaction potential that tries to reproduce the gravitational force among cloud systems in a bidimensional space. The definition of COP is given as

\begin{matrix} (A7) & COP = \frac{2}{N (N - 1)} \sum_{i = 1}^{N} \sum_{j = i + 1}^{N} \frac{\sqrt{A_{i} / π} + \sqrt{A_{j} / π}}{d_{i j}}, \end{matrix}

which is the mean over all possible pairs of the interaction potential.

A6 Area-based Convective Organization Potential

The area-based Convective Organization Potential (ABCOP) (Jin et al., 2022) is a modification of COP. It is defined by

\begin{matrix} (A8) & ABCOP = \frac{1}{2 L} \sum_{i = 1}^{N} max_{j \neq i} (\frac{A_{i} + A_{j}}{d_{2} (i, j)}), \end{matrix}

where $d_{2} (i, j) = max (1, d_{i, j} - \sqrt{A_{i} / π} - \sqrt{A_{j} / π})$ is an estimate of the distance between the edges of the ith and the jth object.

A7 Radar Organization Metric

The Radar Organization Metric (ROME) (Retsch et al., 2020) was originally defined to measure the strength of organization within a radar scene. Let ${\tilde{d}}_{i j}$ be the smallest distance between the edges of the ith and the jth object in the domain. ROME is defined by

\begin{matrix} (A9) & \begin{aligned} ROME & = \frac{2}{N (N - 1)} \sum_{i = 1}^{N} \sum_{j = i + 1}^{N} \\ \cdot [A_{i j}^{(\max)} + A_{i j}^{(\min)} \cdot min (1, \frac{A_{i j}^{(\min)}}{{\tilde{d}}_{i j}^{2}})], \end{aligned} \end{matrix}

where $A_{i j}^{(\max)} = max (A_{i}, A_{j})$ and $A_{i j}^{(\min)} = min (A_{i}, A_{j})$ . The value of ROME is always in between $\bar{A} < ROME < 2 \bar{A}$ , where $\bar{A}$ is the object mean size $\sum_{i} A_{i} / N$ .

A8 Morphological index of convective aggregation

The morphological index of convective aggregation (MICA) (Kadoya and Masunaga, 2018) is the only index that takes into account both the cloud and the clear sky coverage. Let A_cld be the area of the smaller rectangle that encloses all the objects under study in the domain. MICA is defined by

\begin{matrix} (A10) & MICA = \frac{\sum_{i} A_{i}}{A_{cld}} \cdot \frac{| L^{2} - A_{cld} |}{L^{2}}, \end{matrix}

which is the multiplication of two terms: the first term quantifies the density of objects within a confined area, and the second term quantifies the amount of clear sky in the studied domain.

A9 Organization Index based on Distance and Relative Area

A new index of organization, called the Organization Index based on Distance and Relative Area (OIDRA), is defined and given as an example of a well-behaving index. Let ${\tilde{d}}_{i j}$ be the smallest distance between the edges of the ith and the jth object in the domain, and let $A_{T} = \sum_{i} A_{i}$ . OIDRA is defined by

\begin{matrix} (A11) & OIDRA = \frac{1}{A_{T}^{2}} \sum_{i = 1}^{N} A_{i}^{2} + \frac{2}{A_{T}^{2}} \sum_{i = 1}^{N} \sum_{j = i + 1}^{N} A_{i} A_{j} (1 - \sqrt{\frac{{\tilde{d}}_{i j}}{l}}), \end{matrix}

where l is the length scale at which we desire to study organization, and it is set to $L / \sqrt{2}$ in this study. The first part takes into account the different object sizes, while the second part also depends on the distances between them. OIDRA introduces two new concepts that have never been used before in other organization indices. Firstly, OIDRA does not depend explicitly on the size of the objects but only on their relative fraction. Secondly, the parameter l can be set to different values in order to calculate organization at different scales. When two objects are in close proximity ( ${\tilde{d}}_{i j} ≪ l$ ), their contribution to OIDRA is similar to what it would be if the objects were merged. When two objects are very far ( ${\tilde{d}}_{i j} ≫ l$ ), their contribution becomes negative. This index is always smaller than 1, and it is positive if $l > L / \sqrt{2}$ . In our case ( $l = L / \sqrt{2}$ ), OIDRA varies between 0 and 1.

Because of its specific formulation, OIDRA is different to all the other indices. The main reason can be attributed to its dependence on the object sizes, which are squared. This feature makes OIDRA very sensitive to object sizes, which makes it similar to ROME. ROME and OIDRA exhibit similar behaviors for conditions 1, 2, and 5, where the object size plays a crucial role. Moreover, ROME and OIDRA correlate higher than 0.5. Nevertheless, OIDRA's response to object proximity aligns more closely with L_org than with other indices.

Code and data availability

The TOOCAN dataset is available on the official website: https://doi.org/10.14768/1be7fd53-8b81-416e-90d5-002b36b30cf8 (Fiolleau and Roca, 2023). The code developed to perform this study is available at https://doi.org/10.5281/zenodo.8287752 (Mandorli, 2023). The simulated dataset can be obtained with the available code, and the entire analysis can be run using the simulated dataset. All the plots produced by the code are already available at https://web.lmd.jussieu.fr/~gmandorli/Assessment_of_the_object-based_indices_of_convective_organization (last access: 24 August 2023).

Supplement

The supplement related to this article is available online at: https://doi.org/10.5194/gmd-17-7795-2024-supplement.

Author contributions

GM developed the concept, undertook the analysis, and prepared the ﬁgures. CJS contributed to the discussion of the analysis. Both CJS and GM contributed to writing the paper.

Competing interests

The contact author has declared that neither of the authors has any competing interests.

Disclaimer

Publisher's note: Copernicus Publications remains neutral with regard to jurisdictional claims made in the text, published maps, institutional affiliations, or any other geographical representation in this paper. While Copernicus Publications makes every effort to include appropriate place names, the final responsibility lies with the authors.

Acknowledgements

The authors express their gratitude to Thomas Fiolleau for his invaluable assistance in comprehending and utilizing the TOOCAN dataset. The authors extend their appreciation to Giovanni Biagioli for engaging in constructive discussions regarding the behavior of the organization indices. Last, the authors thank Philip Blackberg and the anonymous reviewer for their valuable comments that significantly improved the quality of the paper.

Financial support

This research has been supported by the Centre National de la Recherche Scientifique (CNRS), the Centre National d'Etudes Spatiales (CNES), and the TTL-Xing (grant no. ANR-17-CE01-0015).

Review statement

This paper was edited by Richard Neale and reviewed by Philip Blackberg and one anonymous referee.

References

Bao, J., Sherwood, S. C., Colin, M., and Dixit, V.: The Robust Relationship Between Extreme Precipitation and Convective Organization in Idealized Numerical Modeling Simulations, J. Adv. Model. Earth Sy., 9, 2291–2303, https://doi.org/10.1002/2017MS001125, 2017. a

Besag, J.: Discussion on Dr Ripley's Paper, J. Roy. Stat. Soc. B, 39, 192–212, https://doi.org/10.1111/j.2517-6161.1977.tb01616.x, 1977. a

Biagioli, G. and Tompkins, A. M.: Measuring Convective Organization, J. Atmos. Sci., 80, 2769–2789, https://doi.org/10.1175/JAS-D-23-0103.1, 2023. a, b, c

Bläckberg, C. P. O. and Singh, M. S.: Increased Large-Scale Convective Aggregation in CMIP5 Projections: Implications for Tropical Precipitation Extremes, Geophys. Res. Lett., 49, e2021GL097295, https://doi.org/10.1029/2021GL097295, 2022. a, b, c, d, e

Bony, S., Semie, A., Kramer, R. J., Soden, B., Tompkins, A. M., and Emanuel, K. A.: Observed Modulation of the Tropical Radiation Budget by Deep Convective Organization and Lower-Tropospheric Stability, AGU Advances, 1, e2019AV000155, https://doi.org/10.1029/2019AV000155, 2020. a, b, c, d

Bretherton, C. S., Blossey, P. N., and Khairoutdinov, M.: An Energy-Balance Analysis of Deep Convective Self-Aggregation above Uniform SST, J. Atmos. Sci., 62, 4273–4292, https://doi.org/10.1175/JAS3614.1, 2005. a, b

Cacciari, M., Salam, G. P., and Soyez, G.: The anti-k_t jet clustering algorithm, JHEP, 04, 063, https://doi.org/10.1088/1126-6708/2008/04/063, 2008. a

Cronin, T. W. and Wing, A. A.: Clouds, Circulation, and Climate Sensitivity in a Radiative-Convective Equilibrium Channel Model, J. Adv. Model. Earth Sy., 9, 2883–2905, https://doi.org/10.1002/2017MS001111, 2017. a, b

Fiolleau, T. and Roca, R.: TOOCAN Database V2.08 – Tracking Of Organized Convection Algorithm using a 3-dimensional segmentation, IPSL Data Catalog [data set], https://doi.org/10.14768/1be7fd53-8b81-416e-90d5-002b36b30cf8, 2023. a

Fiolleau, T., Roca, R., Cloché, S., Bouniol, D., and Raberanto, P.: Homogenization of Geostationary Infrared Imager Channels for Cold Cloud Studies Using Megha-Tropiques/ScaRaB, IEEE T. Geosci. Remote, 58, 6609–6622, https://doi.org/10.1109/TGRS.2020.2978171, 2020. a

Held, I. M., Hemler, R. S., and Ramaswamy, V.: Radiative-Convective Equilibrium with Explicit Two-Dimensional Moist Convection, J. Atmos. Sci., 50, 3909–3927, https://doi.org/10.1175/1520-0469(1993)050<3909:RCEWET>2.0.CO;2, 1993. a

Holloway, C. E. and Woolnough, S. J.: The sensitivity of convective aggregation to diabatic processes in idealized radiative-convective equilibrium simulations, J. Adv. Model. Earth Sy., 8, 166–195, https://doi.org/10.1002/2015MS000511, 2016. a

Janssens, M., Vilà-Guerau de Arellano, J., Scheffer, M., Antonissen, C., Siebesma, A. P., and Glassmeier, F.: Cloud Patterns in the Trades Have Four Interpretable Dimensions, Geophys. Res. Lett., 48, e2020GL091001, https://doi.org/10.1029/2020GL091001, 2021. a

Jin, D., Oreopoulos, L., Lee, D., Tan, J., and Kim, K.-m.: A New Organization Metric for Synoptic Scale Tropical Convective Aggregation, J. Geophys. Res.-Atmos., 127, e2022JD036665, https://doi.org/10.1029/2022JD036665, 2022. a, b, c, d, e

Kadoya, T. and Masunaga, H.: New Observational Metrics of Convective Self-Aggregation: Methodology and a Case Study, J. Meteorol. Soc. Jpn. Ser. II, 96, 535–548, https://doi.org/10.2151/jmsj.2018-054, 2018. a, b

Mandorli, G.: gmandorl/Assessment_of_the_object-based_indices_to_identify_convective_organization: before_submission (Version v1), Zenodo [code], https://doi.org/10.5281/zenodo.8287752, 2023. a

Muller, C., Yang, D., Craig, G., Cronin, T., Fildier, B., Haerter, J. O., Hohenegger, C., Mapes, B., Randall, D., Shamekh, S., and Sherwood, S. C.: Spontaneous Aggregation of Convective Storms, Annu. Rev. Fluid Mech., 54, 133–157, https://doi.org/10.1146/annurev-fluid-022421-011319, 2022. a, b

Muller, C. J. and Held, I. M.: Detailed Investigation of the Self-Aggregation of Convection in Cloud-Resolving Simulations, J. Atmos. Sci., 69, 2551–2565, https://doi.org/10.1175/JAS-D-11-0257.1, 2012. a

Muller, C. J. and Romps, D. M.: Acceleration of tropical cyclogenesis by self-aggregation feedbacks, P. Natl. Acad. Sci. USA, 115, 2930–2935, https://doi.org/10.1073/pnas.1719967115, 2018. a, b

Prein, A., Feng, Z., Fiolleau, T., Moon, Z., Núñez Ocasio, K., Kukulies, J., Roca, R., Varble, A., Rehbein, A., Liu, C., Ikeda, K., Mu, Y., and Rasmussen, R.: Km-Scale Simulations of Mesoscale Convective Systems (MCSs) Over South America – A Feature Tracker Intercomparison, ESSOAR [data set], https://doi.org/10.22541/essoar.169841723.36785590/v1, 2023. a

Pscheidt, I., Senf, F., Heinze, R., Deneke, H., Trömel, S., and Hohenegger, C.: How organized is deep convection over Germany?, Q. J. Roy. Meteor. Soc., 145, 2366–2384, https://doi.org/10.1002/qj.3552, 2019. a

Retsch, M. H., Jakob, C., and Singh, M. S.: Assessing Convective Organization in Tropical Radar Observations, J. Geophys. Res.-Atmos., 125, e2019JD031801, https://doi.org/10.1029/2019JD031801, 2020. a, b, c, d

Ripley, B. D.: The second-order analysis of stationary point processes, J. Appl. Probab., 13, 255–266, https://doi.org/10.2307/3212829, 1976. a

Ripley, B. D.: Modelling Spatial Patterns, J. Roy. Stat. Soc. B, 39, 172–192, https://doi.org/10.1111/j.2517-6161.1977.tb01615.x, 1977. a

Ripley, B. D.: Wiley Series in Probability and Statistics, pp. 253–260, John Wiley & Sons, Ltd, ISBN 9780471725213, https://doi.org/10.1002/0471725218.scard, 1981. a

Semie, A. G. and Bony, S.: Relationship Between Precipitation Extremes and Convective Organization Inferred From Satellite Observations, Geophys. Res. Lett., 47, e2019GL086927, https://doi.org/10.1029/2019GL086927, 2020. a, b

Stein, T. H. M., Holloway, C. E., Tobin, I., and Bony, S.: Observed Relationships between Cloud Vertical Structure and Convective Aggregation over Tropical Ocean, J. Climate, 30, 2187–2207, https://doi.org/10.1175/JCLI-D-16-0125.1, 2017. a

Stubenrauch, C. J., Mandorli, G., and Lemaitre, E.: Convective organization and 3D structure of tropical cloud systems deduced from synergistic A-Train observations and machine learning, Atmos. Chem. Phys., 23, 5867–5884, https://doi.org/10.5194/acp-23-5867-2023, 2023. a, b, c, d, e, f

Tan, J., Jakob, C., Rossow, W. B., and Tselioudis, G.: Increases in tropical rainfall driven by changes in frequency of organized deep convection, Nature, 519, 451–454, https://doi.org/10.1038/nature14339, 2015. a

Tobin, I., Bony, S., and Roca, R.: Observational Evidence for Relationships between the Degree of Aggregation of Deep Convection, Water Vapor, Surface Fluxes, and Radiation, J. Climate, 25, 6885–6904, https://doi.org/10.1175/JCLI-D-11-00258.1, 2012. a, b, c, d

Tobin, I., Bony, S., Holloway, C. E., Grandpeix, J.-Y., Sèze, G., Coppin, D., Woolnough, S. J., and Roca, R.: Does convective aggregation need to be represented in cumulus parameterizations?, J. Adv. Model. Earth Sy., 5, 692–703, https://doi.org/10.1002/jame.20047, 2013. a, b

Tompkins, A. M.: Organization of Tropical Convection in Low Vertical Wind Shears: The Role of Water Vapor, J. Atmos. Sci., 58, 529–545, https://doi.org/10.1175/1520-0469(2001)058<0529:OOTCIL>2.0.CO;2, 2001. a, b

Tompkins, A. M. and Semie, A. G.: Organization of tropical convection in low vertical wind shears: Role of updraft entrainment, J. Adv. Model. Earth Sy., 9, 1046–1068, https://doi.org/10.1002/2016MS000802, 2017. a, b, c, d

van der Walt, S., Schönberger, J. L., Nunez-Iglesias, J., Boulogne, F., Warner, J. D., Yager, N., Gouillart, E., Yu, T., and scikit-image contributors: scikit-image: image processing in Python, 2, e453, https://doi.org/10.7717/peerj.453, 2014. a

Weger, R. C., Lee, J., Zhu, T., and Welch, R. M.: Clustering, randomness and regularity in cloud fields: 1. Theoretical considerations, J. Geophys. Res.-Atmos., 97, 20519–20536, https://doi.org/10.1029/92JD02038, 1992. a, b

White, B. A., Buchanan, A. M., Birch, C. E., Stier, P., and Pearson, K. J.: Quantifying the Effects of Horizontal Grid Length and Parameterized Convection on the Degree of Convective Organization Using a Metric of the Potential for Convective Interaction, J. Atmos. Sci., 75, 425–450, https://doi.org/10.1175/JAS-D-16-0307.1, 2018. a, b, c

Wing, A. A. and Emanuel, K. A.: Physical mechanisms controlling self-aggregation of convection in idealized numerical modeling simulations, J. Adv. Model. Earth Sy., 6, 59–74, https://doi.org/10.1002/2013MS000269, 2014. a, b, c

Wing, A. A., Emanuel, K., Holloway, C. E., and Muller, C.: Convective Self-Aggregation in Numerical Simulations: A Review, Surv. Geophys., 38, 1173–1197, https://doi.org/10.1007/s10712-017-9408-4, 2017. a, b

Wing, A. A., Stauffer, C. L., Becker, T., Reed, K. A., Ahn, M.-S., Arnold, N. P., Bony, S., Branson, M., Bryan, G. H., Chaboureau, J.-P., De Roode, S. R., Gayatri, K., Hohenegger, C., Hu, I.-K., Jansson, F., Jones, T. R., Khairoutdinov, M., Kim, D., Martin, Z. K., Matsugishi, S., Medeiros, B., Miura, H., Moon, Y., Müller, S. K., Ohno, T., Popp, M., Prabhakaran, T., Randall, D., Rios-Berrios, R., Rochetin, N., Roehrig, R., Romps, D. M., Ruppert Jr., J. H., Satoh, M., Silvers, L. G., Singh, M. S., Stevens, B., Tomassini, L., van Heerwaarden, C. C., Wang, S., and Zhao, M.: Clouds and Convective Self-Aggregation in a Multimodel Ensemble of Radiative-Convective Equilibrium Simulations, J. Adv. Model. Earth Sy., 12, e2020MS002138, https://doi.org/10.1029/2020MS002138, 2020. a

Xu, K.-M., Hu, Y., and Wong, T.: Convective Aggregation and Indices Examined from CERES Cloud Object Data, J. Geophys. Res.-Atmos., 124, 13604–13624, https://doi.org/10.1029/2019JD030816, 2019. a, b, c

We recall that SCAI and MCAI are negated in this work; hence the correlation with the number of objects is negative.

Articles

Short summary

In recent years, several studies focused their attention on the disposition of convection. Lots of methods, called indices, have been developed to quantify the amount of convection clustering. These indices are evaluated in this study by defining criteria that must be satisfied and then evaluating the indices against these standards. None of the indices meet all criteria, with some only partially meeting them.

Assessment of object-based indices to identify convective organization

2.1 Datasets of convective objects

2.2 Indices of convective organization

3.1 Conditions to be satisfied

3.2 Comparison strategy

4.1 First category: noise-safeness

4.2 Second category: intrinsic behavior

4.3 Third category: capacity to compare across diverse datasets

4.3.1 Condition (5): sensitivity to the horizontal spatial resolution

4.3.2 Condition (6): sensitivity to observation time

4.3.3 Condition (7): sensitivity to the domain limits

A1 Organization index

A2 The index Lorg

A3 Simple Convective Aggregation Index

A4 Modified Convective Aggregation Index

A5 Convective organization potential

A6 Area-based Convective Organization Potential

A7 Radar Organization Metric

A8 Morphological index of convective aggregation

A9 Organization Index based on Distance and Relative Area

A2 The index L_org