Evaluation of Rapeseed Leave Segmentation Accuracy Using Binocular Stereo Vision 3D Point Clouds

Abstract

Point cloud segmentation is necessary for obtaining highly precise morphological traits in plant phenotyping. Although a huge development has occurred in point cloud segmentation, the segmentation of point clouds from complex plant leaves still remains challenging. Rapeseed leaves are critical in cultivation and breeding, yet traditional two-dimensional imaging is susceptible to reduced segmentation accuracy due to occlusions between plants. The current study proposes the use of binocular stereo-vision technology to obtain three-dimensional (3D) point clouds of rapeseed leaves at the seedling and bolting stages. The point clouds were colorized based on elevation values in order to better process the 3D point cloud data and extract rapeseed phenotypic parameters. Denoising methods were selected based on the source and classification of point cloud noise. However, for ground point clouds, we combined plane fitting with pass-through filtering for denoising, while statistical filtering was used for denoising outliers generated during scanning. We found that, during the seedling stage of rapeseed, a region-growing segmentation method was helpful in finding suitable parameter thresholds for leaf segmentation, and the Locally Convex Connected Patches (LCCP) clustering method was used for leaf segmentation at the bolting stage. Furthermore, the study results show that combining plane fitting with pass-through filtering effectively removes the ground point cloud noise, while statistical filtering successfully denoises outlier noise points generated during scanning. Finally, using the region-growing algorithm during the seedling stage with a normal angle threshold set at 5.0/180.0* M_PI and a curvature threshold set at 1.5 helps to avoid the under-segmentation and over-segmentation issues, achieving complete segmentation of rapeseed seedling leaves, while the LCCP clustering method fully segments rapeseed leaves at the bolting stage. The proposed method provides insights to improve the accuracy of subsequent point cloud phenotypic parameter extraction, such as rapeseed leaf area, and is beneficial for the 3D reconstruction of rapeseed.

1. Introduction

Rapeseed is one of the most important oil crops globally, after oil palm and soybean, and is also a major source of biofuel [1,2]. During 2023, rapeseed occupied the largest planted area among oilseed crops in China, reaching 7253 thousand hectares while also achieving a huge production volume of 15.53 million tons [3]. After oil extraction, the resulting rapeseed cake is rich in amino acids and other substances, which can not only be used as animal feed [4,5] but also to improve the soil around the roots of plants [6]. Phenotypic traits of rapeseed reflect the quality and growth status of rapeseed [7] and also facilitate precision breeding [8,9]. Among these traits, leaf area and leaf area index reflect the canopy structure [10], monitor crop growth [11,12], and are closely related to yield [8]. In field production, obtaining leaf data often relies on visual estimation and destructive sampling methods. These methods are time-consuming, laborious, and can damage crops [13]. Therefore, non-destructive automated monitoring of rapeseed leaves is of great research significance.

In recent years, remote sensing technology has provided technical support for efficiently obtaining images on large-scale farmland [14]. Compared to 2D images, 3D images can capture the spatial morphology of crops, which can be used to analyze and study the morphological structure and growth process of crops [15,16,17]. Three-dimensional scanning technology is also of great significance for extracting crop phenotypic features and conducting 3D visualization. Furthermore, compared with active sensors such as ultrasonic and LiDAR, binocular stereo vision is a passive sensor that mainly uses the principle of binocular disparity to obtain three-dimensional spatial information with low cost and high resolution [18]. Previously, binocular stereo vision has been successfully applied to many plants [19,20,21,22,23]. Ge et al. [24] integrated binocular stereo vision technology with a Gaussian mixture model to successfully identify broccoli seedlings under various weed conditions. The instance segmentation model, YOLO-TomatoSeg, developed by Zheng et al. [25], in combination with the parameters of a binocular camera, attained precision in tomato localization that complied with the positioning demands of harvesting robots.

During the acquisition process, a large amount of noise is generated due to equipment and the collection environment, which has a detrimental effect on the subsequent point cloud stitching and analysis. At the same time, under the scenario of huge point cloud data, direct processing and feature extraction of the point cloud are challenging, requiring point cloud segmentation. Point cloud segmentation is the process of classifying point clouds based on local features, grouping points with similar attributes into regions, and dividing point clouds into blocks for further processing [26]. Point cloud segmentation can be categorized into the following types: region-growing-based [27,28], edge-based [29,30], model-based [31,32], point cloud clustering-based [33,34], and deep learning-based segmentation algorithm [35,36,37]. Accurate and effective segmentation of 3D point cloud data is a crucial step in point cloud processing, as it directly impacts the accuracy of subsequent processing tasks. Different factors, such as the quantity and structure of the point cloud, may pose challenges in segmentation. Thus, point cloud denoising and segmentation are important steps in point cloud processing. Liu et al. [38] removed background noise and outliers through pass-through filtering and statistical filtering, followed by reconstructing a 3D model of peanut plants using point cloud spatial coordinates. Wang et al. [39] acquired a potato 3D point cloud based on binocular stereo vision, removed outliers through filtering and k-means clustering, and calculated the crop water stress index (CWSI). Bao et al. [40] utilized steps, including voxel grid downsampling, ground plane fitting with the random sample consensus (RANSAC) algorithm, statistical filtering, and Euclidean clustering to eliminate noise points and obtain the sorghum plant height. The application of filtering algorithms to reduce the noise of point clouds has been widely used in many plant 3D reconstruction studies [41,42,43].

Achieving accurate and efficient segmentation remains a serious research hotspot in the field. Miao et al. [44] utilized the RANSAC algorithm and Euclidean clustering to remove ground points from point clouds and proposed a point cloud image conversion method to achieve stem and leaf segmentation. Zhu et al. [45] employed conditional filtering and statistical outlier removal filtering to remove the background point cloud based on a three-dimensional reconstruction of the tomato canopy and effectively obtained clear point clouds of the tomato canopy and leaves. Wang et al. [46] successfully achieved the extraction of organ-level parameters in maize without the need for organ-level segmentation by employing a distance-field-based segmentation pipeline. Hao et al. [47] developed a high-throughput method for cotton 3D phenotype acquisition and analysis using the PointSegAt deep learning network model. This method includes models for segmenting plant stems and leaves based on point cloud data, as well as an automatic segmentation algorithm. The study findings demonstrate the model’s strong performance in stem-leaf segmentation and single-leaf segmentation while also achieving automated measurements of plant height, leaf number, and wilting leaf area.

There have been significant achievements in 3D visualization of crops such as maize and cotton, but research on rapeseed is relatively scarce, and studies on rapeseed leaves are limited to a single stage [48]. Qiao et al. [49] used a DGCNN-sparse-dense point cloud mapping method to segment the crown siliques, which improved point cloud recognition and counting accuracy for rapeseed siliques. Xiong et al. [50] accurately quantified the 3D canopy structure and single-leaf traits in rapeseed seedlings, achieving a mean absolute percentage error of 3.68% for automated leaf area measurements; however, rapeseed plants in other growth stages have not yet been verified. Teng et al. [51] utilized an RGB-D camera for the 3D reconstruction of rapeseed and proposed an improved method based on the classical iterative closest point with a success rate of cloud registration of 92.5%. Therefore, we planned a study to obtain 3D point clouds of rapeseed leaves at the seedling and bolting stages. We removed background noise using pass-through and statistical filtering. Then, we extracted rapeseed leaves at different stages using region-growing and LCCP algorithms. The objectives of the study were as follows: (a) to obtain rapeseed point clouds using binocular stereo vision technology; (b) to combine different filtering methods to remove ground point clouds and outliers, filtering out most noise points; and (c) to segment the rapeseed leaves at seedling and bolting stages with the aim of improving the accuracy of subsequent point cloud phenotypic parameter extraction such as rapeseed leaf area.

2. Materials and Methods

2.1. Study Site and Experiment Design

The experiment was conducted in an experimental field of the College of Agriculture, Yangzhou University, Yangzhou City, Jiangsu Province, China (32°38′ N, 119°42′ E) during the 2021–2022 growing season. Three rapeseed varieties, namely Huyou 039, Qinyou 7, and Zheyouza 108, were selected for the experiment. The planting method involved strip sowing with a soil covering of 2–4 cm in depth. The experiment was replicated three times, with each plot area measuring 9 square meters, and a 0.5 m-wide isolation strip was set up between each plot. Standard irrigation and fertilization practices were followed throughout the experiment, and, due to the late sowing period for rapeseed, plastic film was used for protection during extremely low temperatures.

2.2. Point Cloud Data Acquisition

In this experiment, we utilized the Weijing Intelligent Stereo Vision Technology (Vizum, Beijing, China), which integrated various sensors, such as hyperspectral cameras, texture cameras, and infrared cameras, into a smart phenotyping platform. This equipment employed line laser binocular stereo vision technology with multiple high-precision wide-field stereo cameras. Four cameras were mounted on the top and two on the side.

Table 1. The specific parameters of the stereo camera.

Parameter Names	Detailed Descriptions	Parameter Names	Detailed Descriptions
Camera size (L × W × H)	Topside 400 × 66 × 75 mm Sidebar 260 × 66 × 75 mm	Output data	X/Y/Z depth point cloud data
Weight	0.75 kg	Baseline distance	Topside 320 mm Sidebar 160 mm
Resolution	1536 × 2048	Lens focus	6 mm
Detection accuracy	Spatial resolution ± 1 mm Positional repeatability ± 0.5 mm	Lens interface	M12
Maximum scanning frequency	2000 Hz	Exposure mode	Global shutter
External interface	Gigabit Ethernet port	Laser perspective	60°
communication method	Communication SDK	Laser wavelength	850 nm
Temperature	Working temperature: −10~50 °C Storage temperature: −20~70 °C	Laser power	1000 mW

presents the specific parameters of the stereo camera. During the camera installation process, the cameras were fixed in place while the laser emitter’s angle was kept perpendicular to the cameras. Data were output to a 5G transmission device via Ethernet cables, and the results were transmitted to a control room for overall management using 5G communication technology.

Data collection was conducted at the seedling and bolting stages of rapeseed. To minimize external noise interference, 3D scans were performed during favorable weather conditions with suitable lighting and no wind. The scanning mechanism of the smart phenotyping platform is shown in Figure 1. The specific steps were as follows: (1) Power on and access the main interface of the device. (2) Release the emergency stop mode to allow the vehicle to automatically leave the warehouse, then manually zero the X and Z axes of the scanning mechanism. (3) Perform region of interest (ROI) detection from each camera. (4) After setting the position of the plot, ensure the crop to be photographed is in the camera’s field of view by moving the vehicle to the plot and adjusting the position of the scanning mechanism with the height set to around one meter above the crop top. If the point cloud image quality is poor, check if the laser line is within the camera; if not, redefine the ROI. (5) Complete the scan and power off the device for storage.

2.3. Point Cloud Coloring

After acquiring the point cloud data, processing was performed using PCL1.11.1 software (https://pointclouds.org/downloads/, accessed on 16 January 2025) on the Windows 10 operating system. The point cloud data were saved in TXT format with six columns. The last three columns, representing color information, were all set to 0, indicating no color variation for visualizing the point cloud data. In this experiment, elevation-based rendering was used to color the point cloud, where the principle involves rendering the point cloud based on selected fields and establishing the relationship between coordinate changes and color changes, typically using the red, green, and blue colors. The specific steps for coloring the point cloud in this experiment were as follows: First, by calculating the maximum, median, and minimum elevation values of the point cloud in the z-axis direction, red, green, and blue colors were selected for rendering. The point with the minimum coordinate was set to blue, the middle coordinate was set to green, and the maximum coordinate was set to red. For the lower part of the point cloud, the ratio of each laser point elevation value in the interval between the minimum and median values was calculated, and then green was added to blue based on this ratio. Similarly, for the upper part of the point cloud, red was added to green based on the ratio of each laser point elevation value in the green zone. The effect of the colored point cloud is shown in Figure 2.

2.4. Point Cloud Denoising

2.4.1. Pass-Through Filtering to Filter Ground Point Cloud

It is challenging to manually remove the non-target point clouds, such as weeds, in the field, when the soil level is inconsistent. Therefore, before analyzing the rapeseed point cloud data, noise data needs to be processed. Visualizing the scanned rapeseed point cloud reveals that some ground points are similar in height to the plants. Thus, filtering the ground point cloud based on height would also result in filtering out parts of the rapeseed plants. By slicing the entire point cloud along the z-axis, there is always be a plane with only soil points. Therefore, the entire rapeseed plot point cloud was first subjected to plane fitting based on RANSAC. The main idea of this method is to select the minimum number of points from the point cloud data that can form a plane, calculate the parameters of the plane, and determine the distance ‘l’ from the remaining points to this plane. A threshold ‘L’ was set, and if ‘l < L’, then the point was considered to be on this plane. When the number of points on this plane reached ‘n’, the plane was saved and the points on the plane were considered matched. This process was repeated, and when ‘n’ was maximized, the plane with the best fitting effect was output as the main plane of the point cloud, as shown in Figure 3, where the shaded grid represents the fitted plane.

In this experiment, pass-through filtering was used to filter the noise of the ground point cloud. The principle of pass-through filtering is to first specify a dimension and set a threshold range. Then, iterate through each point in the point cloud and determine if the value of that point in the specified dimension is within the threshold. Points with values outside the threshold range are directly filtered out, and the remaining points after the iteration are considered the filtered point cloud. Since the coloring of the point cloud was performed based on elevation along the z-axis, the elevation values were used to determine the threshold range for denoising during pass-through filtering. The soil noise scanned in each plot was different, so the threshold range had to be selected accordingly.

2.4.2. Statistical Filtering for Denoising

During the scanning process, due to equipment issues and other factors, unevenly dense point cloud datasets are often generated. Within these datasets, there are outliers, which are points that deviate significantly from the majority of the data. These outliers are sparsely distributed in space and need to be filtered out promptly during point cloud preprocessing to reduce the difficulty of subsequent processing.

The main idea of statistical filtering is to statistically analyze the neighborhood of each point and calculate the average distance from each point to its k nearest neighbors. Assuming the distances of all points follow a Gaussian distribution, the mean and standard deviation become key factors. By setting the mean and variance in the algorithm, outliers can be filtered out. The main algorithm for statistical filtering is as follows:

(1) Set up the neighborhood as k and calculate the average distance (Si) from each point to all points within the k distance. For any two non-overlapping points P1 (${\mathrm{X}}_{\mathrm{i}}$, $Y_i$, $Z_I$) and P2 (${\mathrm{X}}_{\mathrm{j}}$, $Y_j$, $Z_j$) in the point cloud, Equation (1) shows the formula to iterate through each point to find the average distance to the k neighborhood.

$$\mathrm{S}\mathrm{i}={\displaystyle \frac{{\sum }_{j=1}^k\sqrt{{\left({\mathrm{X}}_{\mathrm{i}}-{\mathrm{X}}_{\mathrm{j}}\right)}^2+{\left(Y_i-Y_j\right)}^2+{\left(Z_I-Z_j\right)}^2}}{k}}$$

(1)

Here, (${\mathrm{X}}_{\mathrm{i}}$, $Y_i$, $Z_I$) represents the coordinates of point P1, while (${\mathrm{X}}_{\mathrm{j}}$, $Y_j$, $Z_j$) denotes the coordinates of point P2.

(2) Equations (2) and (3) show the formulas to calculate the mean distance (µ) and sample standard deviation (σ) of the entire point cloud dataset.

$$\mathsf{\mu }={\displaystyle \frac{1}{n}}{\sum }_{i=1}^n{S}_i$$

(2)

$$\sigma =\sqrt{{\displaystyle \frac{1}{n}}{\sum }_{i=1}^n{\left(S_i-\mu \right)}^2}$$

(3)

(3) Once the mean distance and standard deviation were calculated, the distance threshold ($d_{max}$) was determined using Equation (4).

$$d_{max}=\mu +\alpha \times \mu$$

(4)

Here, α is the standard deviation multiple used to control the effect of the distance standard deviation on the distance threshold. In the algorithm, by inputting the values of k and α, if the average distance of the points in the k-neighborhood of a particular point falls within the range of ($\mu -\alpha \times \mu ,\mu +\alpha \times \mu$), the point is retained.

2.5. Target Point Cloud Extraction

2.5.1. Segmentation of Rapeseed Leaves at Seedling Stage

The segmentation of the main stems and leaves of individual rapeseed plants was carried out using conditional filtering. During the seedling stage, a region-growing algorithm was applied to segment the leaves. The specific algorithmic process (Algorithm 1) is outlined as follows:

Algorithm 1. Pseudocode for Conditional Filtering

FUNCTION VisualizeCloud(cloud, filter_cloud)   create viewer for point cloud visualization   CREATE two viewports (v1, v2)   SET the background color for v1 to black   ADD text “point clouds” to v1   SET the background color for v2 to dark gray   ADD text “filtered point clouds” to v2   DEFINE color handler for cloud based on z field   ADD cloud to v1 with color handler   ADD filter_cloud to v2 with green color   WHILE viewer is not stopped     UPDATE viewer     SLEEP for a short duration FUNCTION main()   CREATE cloud_in and cloud_conditional point clouds   LOAD point cloud data from “HY-1p.pcd” into cloud_in   IF loading fails THEN     RETURN -1   PRINT the number of points in cloud_in   CREATE a condition filter (range_cond)   ADD conditions for filtering based on x, y, and z fields   CREATE conditional removal filter (condrem)   SET condition for condrem   SET input cloud for condrem   SET keep organized to true   APPLY filter to cloud_conditional   PRINT the number of points in cloud_conditional before removing NaNs   REMOVE NaN points from cloud_conditional   PRINT the number of points in cloud_conditional after removing NaNs   CALL VisualizeCloud with cloud_in and cloud_conditional   RETURN 0

In summary, region-growing segmentation involves the establishment of seed regions and region growth [52]. While establishing seed regions, we selected the point with the smallest curvature value as the initial seed point to ensure a smooth region, thereby reducing the number of segments during region growth. We utilized a method based on local surface fitting for estimating normal vectors and curvature values. Initially, each point in the point cloud was searched for its k nearest neighbors, and the least squares local plane was calculated for these points. By decomposing the covariance matrix, the smallest eigenvalue corresponds to the normal vector of the plane. When the eigenvalues were obtained, if they satisfy the condition λ1 ≤ λ2 ≤ λ3, Equation (5) shows the formula for curvature value.

$$\mathrm{H}={\displaystyle \frac{{\lambda }_1}{{\lambda }_1+{\lambda }_2+{\lambda }_3}}$$

(5)

2.5.2. Segmentation of Rapeseed Leaves at Bolting Stage

While using the region-growing algorithm for leaf segmentation during the bolting stage of rapeseed, it was discovered that certain leaves were overlapping due to the growth habit of the plant. Therefore, this study employed the LCCP algorithm for segmenting rapeseed leaves in the bolting stage. The LCCP algorithm is an image segmentation method based on the concavity and convexity of neighboring points in a point cloud. The algorithm consists of two main parts: dividing the point cloud into supervoxels based on the angle between normal vectors and spatial distance, determining the connectivity of adjacent supervoxels in the recorded adjacency graph of supervoxels, and finally merging all convexly connected supervoxels to form the segmentation result.

When over-segmenting the point cloud into supervoxels, the data was initially divided into voxels, and a voxel cloud with a specific resolution was created. The voxel cloud was then gridded, and the voxel closest to the center of each grid was selected as the initial seed voxel. After filtering out isolated seed points, a search region was established for the remaining seed voxels on the object surface. The number of voxels within the neighborhood radius of the seed point was calculated, and seed voxels with a voxel count below a fixed threshold in the intersecting region with the search range were removed. Following the initialization of the clustering algorithm, the edge properties, geometric features, and spatial distance of the point cloud were comprehensively considered to measure the similarity between voxels in the feature space. The voxel data were then clustered in the feature space, and an over-segmentation of the voxel data was performed using a flow-constrained clustering algorithm to obtain supervoxel data. After over-segmenting the point cloud, the Extended Convexity Criterion (CC) and Sanity Criterion (SC) were used to determine the concavity and convexity relationships between different blocks. The CC determines whether two adjacent supervoxels are concave or convex based on the vector and normal vector of the line connecting their centers. Stein et al. [53] provided a criterion graph for CC, as shown in Figure 4.

Figure 4 shows that when α₁ < α₂, the two supervoxels are connected convexly; otherwise, they are connected concavely. In the process of using the algorithm, Equations (6) and (7) were used for the CC criterion to reduce misjudgments of concave–convex relationships.

$${\mathrm{C}\mathrm{C}}_b=\left(\overrightarrow{P_i}-\overrightarrow{P_j}\right)=\left\{\begin{array}{c}true\left(\overrightarrow{n_1}-\overrightarrow{n_2}\right)·\overrightarrow{d}>0\vee \left(\beta <{\beta }_{Thresh}\right)\\ falseotherwise\end{array}\right.$$

(6)

$${\mathrm{C}\mathrm{C}}_e\left(\overrightarrow{P_i},\overrightarrow{P_j}\right)={\mathrm{C}\mathrm{C}}_b\left(\overrightarrow{P_i},\overrightarrow{P_j}\right)\wedge {\mathrm{C}\mathrm{C}}_b\left(\overrightarrow{P_i},\overrightarrow{P_c}\right)\wedge {\mathrm{C}\mathrm{C}}_b\left(\overrightarrow{P_j},\overrightarrow{P_c}\right)$$

(7)

where ${\mathrm{C}\mathrm{C}}_b$ is the basic convexity criterion, β is the spatial distance, β_Thresh is the concavity threshold, $\overrightarrow{d}$ indicates the vector from point $\overrightarrow{x_2}$ to point $\overrightarrow{x_1}$, CC_e is the extended convexity criterion, and $\overrightarrow{n_1},\overrightarrow{n_2}$ are the normals of two adjacent supervoxels [53].

If two adjacent supervoxels are not connected and have one side isolated, the SC is introduced with the following Equations (8)–(10).

$$\mathsf{\vartheta }\left(\overrightarrow{{\mathrm{P}}_{\mathrm{i}}},\overrightarrow{{\mathrm{P}}_{\mathrm{j}}}\right)=\min \left(\mathrm{\measuredangle }\left(\overrightarrow{\mathrm{d}},\overrightarrow{\mathrm{s}}\right),\mathrm{\measuredangle }\left(\overrightarrow{\mathrm{d}},-\overrightarrow{\mathrm{s}}\right)\right)=\min \left(\mathrm{\measuredangle }\left(\overrightarrow{\mathrm{d}},\overrightarrow{\mathrm{s}}\right),180°,\mathrm{\measuredangle }-\left(\overrightarrow{\mathrm{d}},\overrightarrow{\mathrm{s}}\right)\right)$$

(8)

$${\mathsf{\vartheta }}_{thresh}\left(\beta \right)={\vartheta }_{thresh}^{max}·{\left(1\right.\left.+\exp \left[-a·\left(\beta -{\beta }_{off}\right)\right]\right)}^{-1}$$

(9)

$$\mathrm{S}\mathrm{C}\left(\overrightarrow{P_i},\overrightarrow{P_j}\right)=\left\{\begin{array}{c}true\vartheta \left(\overrightarrow{{\mathrm{P}}_{\mathrm{i}}},\overrightarrow{{\mathrm{P}}_{\mathrm{j}}}\right){>\mathsf{\vartheta }}_{thresh}\left(\beta \left(\overrightarrow{n_1},\right.\left.\overrightarrow{n_2}\right)\right)\\ falseotherwise\end{array}\right.$$

(10)

where $\vartheta$ is the minimum angle between two directions, ${\vartheta }_{thresh}^{max}$ is 60°, $a$ is 0.25, and β_off is 25° [53].

Equation (11) shows the formula for the convex edge criterion between two adjacent supervoxels.

$$\mathrm{c}\mathrm{o}\mathrm{n}\mathrm{v}\left(\overrightarrow{{\mathrm{P}}_{\mathrm{i}}},\overrightarrow{{\mathrm{P}}_{\mathrm{j}}}\right)={\mathrm{C}\mathrm{C}}_{b,e}\left(\overrightarrow{P_i},\overrightarrow{P_j}\right)\wedge \mathrm{S}\mathrm{C}\left(\overrightarrow{P_i},\overrightarrow{P_j}\right)$$

(11)

where $\mathrm{c}\mathrm{o}\mathrm{n}\mathrm{v}\left(\overrightarrow{{\mathrm{P}}_{\mathrm{i}}},\overrightarrow{{\mathrm{P}}_{\mathrm{j}}}\right)$ refers to the concave–convex relationship between two supervoxels, ${\mathrm{C}\mathrm{C}}_{b,e}\left(\overrightarrow{P_i},\overrightarrow{P_j}\right)$ refers to the result of using the CC between two supervoxels, and $\mathrm{S}\mathrm{C}\left(\overrightarrow{P_i},\overrightarrow{P_j}\right)$ refers to the result of using the SC between two supervoxels.

After marking the concave–convex relationships of each small region, these regions were clustered into larger objects using a region-growing algorithm, completing the segmentation process.

3. Results

3.1. Evaluation of Point Cloud Denoising Accuracy

3.1.1. Evaluation of Pass-Through Filtering Denoising Accuracy

Taking the denoising effect of the first plot of rapeseed in Huyou 039 as an example, the rapeseed point cloud was filtered using pass-through filtering based on the elevation values. The same steps were followed for the subsequent plots. Figure 5a shows the original rapeseed point cloud, where the red box highlights the target point cloud. When coloring the point cloud, the z-axis direction was colored from blue to green to red, with the z-coordinate of a randomly selected point from the fitted plane serving as one limit of the pass-through filtering threshold. The blue value was used as the other limit, and denoising was performed using this threshold. The filtered point cloud is shown in Figure 5b. The z-axis direction of the filtered point cloud was colored from red to green to blue. The blank areas in the image indicate that the low points of the ground point cloud have been filtered out. If filtering continues upwards, the rapeseed point cloud will also be filtered out. Therefore, the appearance of blank areas in the denoised point cloud image serves as the criterion for the end of pass-through filtering.

The point cloud structure scanned in this experiment was complex, with a lot of noise. Point cloud filtering was performed to improve the quality and accuracy of point cloud data. The denoising ratio served as an evaluation metric to verify the effectiveness of pass-through filtering on denoising point cloud data. Equation (12) shows the formula for the denoising ratio.

$$\mathsf{\alpha }={\displaystyle \frac{\mathrm{p}-{\mathrm{p}}_i}{\mathrm{p}}}$$

(12)

where α represents the percentage of removed points compared to the total number of original points, p is the total number of original points, and p_i is the number of points after denoising.

Six parts of scanned point cloud data were denoised using pass-through filtering in this experiment. The number of denoised points and denoising ratios are shown in

Table 2. Denoising accuracy of the rapeseed point cloud using pass-through filtering method.

Point Number	1	2	3	4	5	6
Number of original points	18,386,979	20,343,619	20,404,340	20,356,679	20,226,673	20,337,138
Number of denoised points	9,526,272	10,274,000	10,428,856	11,763,012	10,844,220	10,093,445
Denoising ratio α (%)	48.2	49.5	48.9	42.2	46.4	50.4

.

A comparison between the denoised point cloud image and the original point cloud image reveals that ground noise was successfully removed through pass-through filtering. Results presented in Table 2 show that the original point cloud data were very large, and even after pass-through filtering, the lowest denoising ratio was only 42.2%. This result helps streamline the large point cloud data for further processing.

3.1.2. Evaluation of Statistical Filtering Denoising Accuracy

The previous denoising of the point cloud was only aimed at filtering the ground point cloud of the entire rapeseed plot. Noise generated by the rapeseed plants during scanning was not addressed. When segmenting the point cloud of individual rapeseed plants using conditional filtering, small-scale noise needed to be filtered out to reduce errors in subsequent point cloud processing. Therefore, statistical filtering was also required.

To prevent incomplete denoising or over-denoising, suitable threshold ranges needed to be selected for statistical filtering. The parameters involved were the number of neighboring points (k) and the standard deviation multiple (α). In this study, values of 5, 10, 20, 30, 50, and 100 were selected for k, and values of 0.01, 0.05, 0.1, 0.5, 1, 5, and 10 were selected for α. Combinations of k and α were tested for statistical filtering. A single rapeseed plant was selected as a demonstration, and line graphs of the standard deviation multiple, the number of neighboring points, and the filtered point cloud count were plotted, as shown in Figure 6.

From the graph, it can be observed that under constant neighboring points, the number of removed points decreases as the standard deviation multiple increases. However, when α reaches 5 or higher, the decreasing trend in the number of removed points tends to stabilize. When α is between 0.01 and 0.1, the number of removed points with a k value of 5 significantly exceeds the number removed than other values of k. However, when α is 0.1, the number of removed points under k = 100 significantly exceeds that for other values. When the α is between 0.1 and 0.5, the number of removed points with k = 5 decreases noticeably. Additionally, when the standard deviation multiple was between 0.5 and 5, the number of removed points increased as the k values increased. The line graph provides a rough understanding of the relationship between the number of removed points and the standard deviation multiple under different neighboring point counts. However, further precision is needed to determine the ranges of these two parameters. The comparison of the point cloud images after statistical filtering is shown in Figure 7.

We found that, when the standard deviation multiple is 0.01, as the number of neighboring points increases, not only were the noise point clouds removed, but valid point clouds were also significantly filtered out, as shown in Figure 7a,b. Therefore, it is important to determine the upper limit of k. Comparing Figure 7a,c,d, it is evident that when the number of neighboring points k remains constant, as α increases, the filtering effect on the noise point clouds (represented by the red portion) becomes less pronounced, while the filtering of valid points is not as significant. Additionally, it was observed that only a small number of valid points were removed under low α values (Figure 7a,c).

Further, the above results reveal that in statistical filtering, the number of neighboring points mainly affects the filtering of non-target noise. At the same time, the standard deviation multiple primarily influences the removal of target noise. Therefore, k values of 5 and 100 can be excluded, and α values of 0.01, 5, and 10 can also be excluded. Setting the k values to 10, 20, 30, and 50 and the α to 0.05, 0.1, 0.5, and 1.0 would be a suitable approach. Additionally, when α is set to 0.1, regardless of the different values of k, the four lines almost overlap, indicating minimal variation in the number of removed points. Hence, this study selects α as 0.1. Subsequently, a comparison of the number of points before and after statistical filtering for k values of 10, 20, 30, and 50 was conducted, and the experimental results are shown in

Table 3. Comparison of the total number of points before and after statistical filtering.

Typology	The Number of Neighboring Points k
	10	20	30	50
Total number of points in the point cloud before denoising	20,259	20,259	20,259	20,259
Total number of points in the point cloud after denoising	17,190	17,182	17,192	17,177
Number of points removed	3069	3077	3067	3082
Percentage of points removed (%)	15.15	15.19	15.14	15.21

.

We found that when k is set to 50, the proportion of removed points to the total points before filtering was the highest (15.21%), as presented in Table 3. Therefore, this study sets the number of neighboring points to 50 and the standard deviation multiple to 0.1 for statistical filtering processing.

3.2. Evaluation of Segmentation Accuracy in Target Point Clouds

3.2.1. Evaluation of Rapeseed Leaf Segmentation Accuracy at the Seedling Stage

Leaf-related parameters were extracted from the point cloud of a single rapeseed plant after segmentation algorithms were applied. Since the number of leaves in the rapeseed seedlings was small and the leaves did not overlap, the region-growing algorithm was used for segmenting individual rapeseed plant leaves. This method requires determining suitable normal angle thresholds and curvature thresholds. In this study, normal angle thresholds were taken as 3.0/180.0*M_PI, 5.0/180.0*M_PI, and 7.0/180.0*M_PI, and curvature thresholds were taken as 0.5, 1.0, and 1.5. The constant “M_PI” represents the numerical value of pi (approximately 3.1416) in programming languages.

Firstly, measurements were taken on different normal angle threshold values, and it was found that segmentation was not possible when the normal angle threshold was set to 7.0/180.0*M_PI. As the rapeseed plants at the seedling stage had four leaves, the normal angle threshold was set to 3.0/180.0*M_PI, and the segmented regions were fewer than four, leading to incorrect segmentation. However, when the normal angle threshold was set to 5.0/180.0*M_PI, it was observed that four regions could be accurately segmented; hence, this threshold was chosen.

During point cloud segmentation, the entire point cloud was divided into target point clouds and non-target point clouds, where non-target point clouds could also be considered as background points outside the target. Segmentation is often not perfect in a single step, and under-segmentation and over-segmentation are common issues. Generally, under-segmentation occurs when target point clouds are mistakenly segmented as background point clouds, leading to incomplete segmentation of the target point clouds. Over-segmentation occurs when background point clouds are mistakenly considered as target point clouds and segmented as such. After setting the normal angle threshold to 5.0/180.0*M_PI, tests were conducted with curvature thresholds of 0.5, 1.0, and 1.5. Different colors in the segmentation results represent different segmented regions, and the results are shown in Figure 8.

The results presented in Figure 8 reveal that under all three curvature values, most of the leaves could be segmented. However, when the curvature values were set to 0.5 and 1.0, the leaf edges and petiole parts were mistakenly identified as background point clouds, leading to under-segmentation. Setting the curvature value to 1.5 helped to alleviate this issue. Thus, based on the tuning results of this study, we set the normal angle threshold to 5.0/180.0*M_PI and the curvature threshold to 1.5.

Figure 9 shows the accuracy assessment of the Huyou039 leaf area, comparing the manually measured leaf area with the leaf area extracted from the point cloud. The R² and RMSE were 0.995 and 0.2589 cm², respectively.

3.2.2. Evaluation of Rapeseed Leaf Segmentation Accuracy at the Bolting Stage

Due to the unique characteristics of rapeseed leaves at the bolting stage, using the same segmentation method as in the seedling stage resulted in uneven segmentation of overlapping leaves, as shown in Figure 10. Different colors in the segmentation results represent different segmented regions. Therefore, this study employed the LCCP algorithm for leaf segmentation of rapeseed plants at the bolting stage. The main parameters involved in this algorithm are voxel resolution, seed resolution, concavity tolerance threshold, and smoothness threshold. Through experimentation with different parameter values, it was found that the best segmentation results were achieved when the values of these four parameters were set to 2.0, 100.0, 100, and 10, respectively. The results of the LCCP algorithm are shown in Figure 10.

After the segmentation using LCCP, rapeseed leaves were segmented and differentiated by different colors. By zooming in on a region where leaves overlap, it was observed that there is no over- or under-segmentation; rather, complete segmentation of the overlapping leaves occurs, as shown in Figure 11.

4. Discussion

4.1. Denoising of Rapeseed 3D Point Clouds

This study aimed to address the challenges of noise removal and leaf segmentation in rapeseed 3D reconstruction, providing methodologies for subsequent 3D reconstructions to obtain more comprehensive phenotypic parameters. The experimental design utilized rapeseed cultivated under field conditions as the data source. Previously, most research on rapeseed phenotypes relied on unmanned aerial vehicles equipped with an RGB camera [54], or LiDAR [55], which do not facilitate the convenient acquisition of 3D structural information about rapeseed. While utilizing 3D scanning technologies like LiDAR for object scanning, a substantial volume of point cloud data is acquired [56]. Furthermore, this process creates significant noise, leading to a considerable workload. Research on 3D crops has predominantly focused on species such as maize and cotton, with relatively few studies dedicated to the three-dimensional analysis of rapeseed leaves. The binocular stereo vision technology utilized in this study enabled the simultaneous acquisition of both color and structural information about the target while also being more cost-effective [34].

The point cloud acquired in this study contained a large number of non-target point clouds, such as ground point clouds, as well as outliers surrounding the primary point cloud. Since crops grew on the ground, in order to ensure the integrity of the crops, the ground point clouds were inevitably included in the scan. To address this noise, a combination of RANSAC and pass-through filtering algorithms was employed for denoising, while statistical filtering was applied to eliminate outliers. The RANSAC algorithm was used to fit a ground plane [57,58]. This approach of using pass-through filtering for setting z-axis thresholds effectively removed large areas of noisy point clouds, which was similar to the results of Teng, Zhou, Wu, Huang, Dong, and Xu [51]. In comparison to the research conducted by Hu, Lin, Peng, Wang, and Zhai [48], the primary distinction of this study lies in the adoption of the RANSAC algorithm for fitting the ground plane. This method demonstrates greater robustness compared to conventional techniques and further optimizes the z-axis threshold setting during the pass-through filtering process, leading to a more pronounced filtering effect. Additionally, the colorization of point clouds allowed for the visualization of noise removal effectiveness, providing a more intuitive and convenient reference for data analysis and subsequent processing. However, the number of points affected the speed of the denoising operation, and parameter thresholds needed to be fine-tuned. Therefore, future research needs to focus on improving the speed and other aspects of this method to achieve faster and more accurate denoising of point clouds.

4.2. Segmentation of Rapeseed Leaf

In the field of point cloud segmentation, the region-growing algorithm can successfully segment small plants such as green beans [59] and greenhouse ornamentals [60]. This study further supports the capability of the algorithm to efficiently maintain the integrity of rapeseed leaves at the seedling stage, and our results align with the successful application of this technique by Li, Liu, Zhang, Wang, Yao, Zhang, Fan, Li, Hai, and Fan [42] in segmenting the stalk of corn seedlings. However, rapeseed undergoes morphological changes as it grows, particularly at the bolting stage when the leaf surface greatly undulates, and there is significant leaf overlap. Thus, employing traditional region-growing segmentation to separate rapeseed leaves at the bolting stage into a whole single leaf is challenging.

In light of the aforementioned challenges, this study explores the application of the LCCP algorithm for leaf segmentation at the bolting stage. The research conducted by Wang and Chen [61] on the 3D reconstruction of early-stage green pepper seedlings inspired this investigation, as they observed that LCCP effectively leverages the natural concave–convex relationships between stems and leaves for segmentation. In this experiment, the LCCP algorithm was used to segment the leaves of rapeseed plants during the bolting stage. The results showed that the leaves were segmented accurately without any over-segmentation in the overlapping regions. This effectiveness is particularly notable in managing structural variations in complex plant morphologies. In contrast to the sophisticated methods that rely on deep learning approaches [62], the LCCP algorithm employed in this study does not require a large-scale training dataset, thereby reducing computational resource consumption and implementation complexity while significantly shortening processing times. This finding enriches the technical tools for plant point cloud segmentation. It also provides an efficient and cost-effective solution for monitoring and analyzing leaf conditions throughout the growth period of crops like rapeseed.

Zhang et al. [37] used the PointNet++ network to automatically segment the tip point cloud of the fringe branch. They achieved Intersection over Union (IoU), precision, and recall values of 96.29, 96.36, and 93.01, respectively. By combining PointNet++ to quantify canopy height and extracting dynamic numerical phenotypes, Chen et al. [63] investigated and analyzed a genome-wide association study of 160 wheat varieties to find reliable loci associated with height and NUE. The blade segmentation method proposed in this paper demonstrates significant effectiveness in addressing the issue of overlapping rapeseed leaves, thereby providing a technical foundation for the subsequent accurate extraction of leaf area and 3D morphological reconstruction. However, the current work has certain limitations, primarily reflected in its focus on processing point cloud data from only two growth stages of the rapeseed plants without a comprehensive analysis covering the entire growth period. Therefore, to enhance the universality and applicability of this segmentation algorithm, future research must urgently explore how to adjust and optimize algorithm parameters and strategies. This will ensure that the algorithm can effectively cope with and accurately segment the complex leaf shapes and dense overlapping characteristics exhibited by rapeseed plants in different late growth stages, thereby comprehensively advancing the development of rapeseed growth monitoring and precision agricultural management technologies.

5. Conclusions

The current study combined line laser binocular stereo vision technology, filtering algorithms, and segmentation algorithms to separate the point cloud of rapeseed leaves from the original rapeseed field and to segment rapeseed leaves at seedling and bolting stages. The study results demonstrated that integrating color filtering and filtering algorithms could effectively eliminate ground point clouds and outliers, resulting in the acquisition of relatively smooth point cloud data, thereby enhancing the quality and clarity of the point cloud. The region-growing algorithm performed well during the seedling stage but struggled to segment overlapping leaves during the bolting stage. The LCCP clustering method proved to be a more effective solution to address this issue. The leaf segmentation algorithms perform well at different growth stages, but a universal method for leaf segmentation has not been established. The method proposed in this study has produced satisfactory results in the early developmental stages of rapeseed, offering feasibility for extracting subsequent point cloud phenotypic parameters such as leaf area. Different segmentation algorithms are required for rapeseed leaves at different growth stages due to variations in plant morphology. Future research endeavors must aim to identify a universal algorithm for leaf segmentation, thereby streamlining the process of leaf segmentation.