2023 International Conference on Integrated Intelligence and Communication Systems (ICIICS)

Development of Intelligent Agricultural Automation

Based on Computer Vision
2023 International Conference on Integrated Intelligence and Communication Systems (ICIICS) | 979-8-3503-1545-5/23/$31.00 ©2023 IEEE | DOI: 10.1109/ICIICS59993.2023.10421284

1st Shuyun Lin 2nd Xingyuan Qi

Guizhou Mountainous Agricultural Machinery Research Institute Guizhou Mountainous Agricultural Machinery Research Institute
Guizhou, China Guizhou, China
[email protected] [email protected]

Abstract—This paper analyzed in detail the application of farmland based on MATLAB curve fitting tools. Applying
computer vision technology to four aspects of intelligent particle swarm optimization (PSO) algorithm to farmland[1].
agricultural automation production, which saved a lot of Brun-Laguna K deployed with two low profile networks, one
manpower and material resources for the whole process of in a smart farming environment (Peachland) and one in a
agricultural products from growth to harvest to logistics. smart building. Both lists of networks use out-of-the-box
Finally, combined with the realization of automatic SmartMesh IP technology to collect sensor values, as well as
classification of apples, the real-time monitoring and a large volume of network statistics. The performance of these
classification technology of fruit quality is discussed, which has two networks was analyzed and contrasted in depth. Both
great practical significance and economic benefits for improving
nodes exhibited 100% end-to-end reliability with expected
the market competitiveness of Chinese fruit products. The error
rate of machine grading results is 11.8% and below, and the
lifespans ranging from 4 to 8 years. Due to the use of Time
error rate of apple grading outside the grade is 0, which can Slot Channel Hopping (TSCH), the network topology is
basically be applied in real life. Through computer vision consistent and only up to 15 links change in the network per
technology, the automatic detection and grading of fruits are day. It is conclusively argued that TSCH enabled by
realized, thus improving the consistency and accuracy of SmartMesh IP is a totally suitable IoT application solution for
agricultural products quality. This is very important to meet the smart agriculture and smart building applications [2]. Aniley
demand of modern market for high-quality agricultural AA reported on a thin-film dual-probe heat pulse (DPHP)
products. microheater network used for the collection of large-scale data
on soil moisture. These microheater nodes were placed in a
Keywords—intelligent agriculture, automated production, large agricultural area where the entire installation was
computer vision, internet of things observed to a great extent of space and time accuracy. The
FTO and ITO coated cores of the microheaters were built on
I. INTRODUCTION a glass backplane. The sensor nodes are powered by a 5V DC
From traditional agriculture to modern agriculture, people power supply and supported by a central Atmega328
must start from the aspects of soil degradation, low technology microcontroller for bi-directional and database connectivity.
content and resource shortage. Take Jilin Province, China as The network can be extended to monitor other soil parameters
an example. The average climate is low, which is not suitable such as soil nutrients or pH. The maximum temperatures
for planting tropical fruits. Moreover, the land resources are recorded by the FTO and ITO heaters for the sensor nodes
not particularly sufficient, and the planting efficiency of were 71.85°C and 45.45°C correspondingly, while the
traditional fruit trees is low. Whether the above problems can maximum change in temperature (ΔTM) was 39.12°C and
be solved in time will be directly related to the output of 13.19°C [3]. Thermal imaging has shown the possibility to
China's agricultural production. Therefore, in order to meet help intelligent irrigation management in many aspects.
the actual needs of China's agricultural development, it is Roopaei M discussed the key technical and legal issues and
necessary to master relevant technologies to speed up the requirements that support the use of IoT to manage water-
process of agricultural informatization and the construction of related data before discussing potential solutions [4]. Yan Rui
modern agriculture. What modern agriculture needs to think made high-performance agricultural sensors, built special
about is how to use the corresponding technology to maximize agricultural wireless networks, developed high-performance
the output of agricultural products per unit area, and carry out intelligent algorithms and other feasible countermeasures [5].
scientific management and fine planting, so that agricultural By analyzing the research status of global smart agriculture,
resources can be effectively used, thereby reducing the cost of Ren Ni provides data reference and information support for
agriculture to a certain extent and increasing the income of scientific and technological innovation and management
farmers. decision-making in the field of smart agriculture in China [6].

II. RELATED WORKS Real-time detection of fruit in the orchard environment is

one of the key technologies of a number of applications in
In the past few years, wireless sensor networks (WSNs) precise agriculture, ranging from estimation of production to
have received great attention in the field of smart agriculture. automated harvests. Chen W proposed a fast method for citrus
One of the main challenges in deploying sensor nodes in the recognition by modifying the state-of-the-art You Only Look
field of smart agriculture is path loss under dense grass or Once version 4 (YOLOv4) detector. The quantity and size of
under tall trees. This results in a reduction in the quality of the the former frame are selected automatically from these RGB
communication link due to absorption, scattering and images using the Canopy algorithm and the K-Means++
attenuation by foliage or trees through crops. Jawad H M algorithm. Experimental results showed that the improved
formulated two new path loss models for ZigBee WSN in

979-8-3503-1545-5/23/$31.00 ©2023 IEEE

Authorized licensed use limited to: AISSMS's Institute of Info Technology - PUNE. Downloaded on January 02,2025 at 05:26:13 UTC from IEEE Xplore. Restrictions apply.
 YOLOv4 detector performed well in detection of various collection, data processing, data analysis, intelligent
growth steps of citrus in natural environment with an average agricultural expert system, etc. [11].
accuracy enhancement of 3.15% (from 92.89% to 96.04%).
The suggested method is applicable to the rapid detecting of A. Intelligent Agriculture Architecture
citrus kinds and positions in natural environments and can be The development goal of intelligent agriculture is to
implemented for citrus harvesting and yield assessment in real update traditional agriculture to realize intelligent control, fine
orchards [7]. Morshed A believed that the emerging Internet management, and scientific farming, so as to make
of Things (IoT) devices (and their varying sophistication and agricultural products high-yield, efficient and high-quality
types), and the vast amount of data (potentially) generated by [12]. Intelligent agriculture uses a series of information
these devices can make a dramatic effect on people’s lives. It technologies, such as WSN, Internet, automatic control,
is driving the growth of key next-generation services and computer, artificial intelligence and other technologies to
applications that include healthcare, smart grid, finance, transmit, analyze and process agricultural production data, so
disaster managing, aviation, transportation, and water as to achieve the purpose of agricultural production [13].
managing. Deep learning techniques that have been used very Agricultural environmental parameters are collected,
effectively in the past in the areas of computer vision and transmitted, processed, and applied in four major links, which
language modeling are now finding applications in new areas, are processed through computer technology [14]. The
driven by different large data sets. One example is the progress intelligent agriculture architecture is shown in Fig. 1.
in medical diagnosis and prediction to enhance human fitness
through the use of deep learning techniques. However, the
timely and reliable delivery of large data streams (a
requirement for deep learning techniques to achieve high
accuracy) to concentrated locations, such as cloud data center
environments, is seen as a key limit to the expansion of the
application of such techniques [8]. The leaves of plants are
rich in information in identifying plants. In general,
agricultural experts completed the extraction of a leaf's
information. Kheirkhah F M studied a new method for
identifying plant species using GIST texture features. Then,
the main and suitable features were selected by principal
component analysis (PCA) algorithm. The results revealed
that the method outperformed other methods in terms of time Fig. 1. Intelligent Agriculture Architecture
and accuracy. The best results were attained by application of
the PCA algorithm to the GIST feature vector and using the The application layer is the top layer in the intelligent
cosine KNN classifier [9]. These approaches offered some agricultural system, which can meet different user
information for the research, but due to the short time and requirements and meet different application requirements.
small sample size of the related studies, this research did not The main function of the application layer is to display the
be received by public. collected agricultural environmental information in real time,
so as to provide scientific guidance and precise control for
This paper studied the design of an intelligent agricultural agricultural production, and to maximize agricultural
automation system based on computer vision. When exploring production efficiency. Intelligent agriculture is a smart
the matching results of apples under natural lighting agriculture that integrates various high technologies. Its core
conditions and ring light sources, it is found that the 8mm lens technology can be seen in Fig. 2.
of the CCD camera has a better matching effect than the
images collected by the 16mm lens. The error rate of machine
classification results is not higher than 11.8%, which basically
meets the requirements of intelligent sorting. Under normal
circumstances, the fluctuation of air temperature, air humidity,
soil temperature and soil humidity in the warehouse is very
small, the fluctuation of air temperature does not exceed 2℃,
and the fluctuation of air humidity does not exceed 2%. The
design of intelligent agricultural automation systems based on
computer vision has significant practical significance and
economic benefits.
III. PROPOSED METHOD
As far as agricultural production is concerned, due to the ,
rudimentary traditional agricultural production facilities, Fig. 2. Intelligent Agriculture Technology System Model
backward production management and technical methods, the
consumption of agricultural resources is not proportional to 1) Perception and Recognition Technology: Perception
agricultural output, resulting in a lot of waste of manpower and recognition technology is an important technology in
and material resources, making agricultural development intelligent agriculture, and it is an important link to obtain
unable to proceed well [10]. Intelligent agriculture is an agricultural environmental information.
agricultural high-tech involving many cutting-edge 2) Network and Communication Technology: As the
technologies, including artificial intelligence and other high- carrier of information exchange, the communication network
tech. Intelligent agriculture is realized from the aspects of data
is of great significance to the information transmission and

Authorized licensed use limited to: AISSMS's Institute of Info Technology - PUNE. Downloaded on January 02,2025 at 05:26:13 UTC from IEEE Xplore. Restrictions apply.
 ∬
= , ∈
interaction in the agricultural environment. On this basis, the
∬
transmission of information through the sensing nodes in the (2)
, ∈
wireless sensor network also requires more information to be
transmitted and communicated remotely. Therefore, how to
Apple represents the agricultural products that have been
transmit information efficiently, safely and reliably becomes tested, and carries out quality inspection and classification. Dx
the problem that the network and communication technology and dy are tiny variations, which are used to describe tiny
must solve. areas in the integration process. Here, they represent small
3) Computing and Service Technology: Information changes in integrating x and y.
computing: With the rapid development of intelligent
Projected area method: Since the apple image captured by
agriculture, the amount of collected data continues to the camera is a two-dimensional plan, its projected area S can
increase, and how to ensure the efficient processing of approximately reflect the area S of the apple using (3) and (4).

=
massive data is an urgent problem. On the one hand,
technologies such as cloud computing and parallel processing (3)

S=4 =4
must be used to improve the processing of collected data; on
the other hand, it is necessary to extract massive amounts of (4)
data by combining data, extracting useful information from it
and effectively utilizing known information. Service Because the area calculated in the image is in pixels, the
computing: The application of intelligent agriculture is a projected area obtained is also in pixels, and the value is large,
dynamic process. According to the differences in its which is usually reduced by (5).
application scenarios and application fields, it is necessary to
r=
conduct in-depth research on its core technologies to form a
(5)
public service application system.
4) Management and Support Technology: In intelligent
agriculture, due to the increasing demand for intelligent Among them, the projection of the apple is approximated
agriculture, how to achieve high-quality management and as a circle, S is the projected area of the apple, R is the radius
high stability of supporting technologies in intelligent of the apple, and its approximate radius is used instead of the
apple area to determine the size of the apple.Equivalent
agriculture needs to improved from the aspects of improving
diameter method: This method needs to extract the edge
the traditional business models and exploring the network contour of the apple image first, and then calculate the
methods. The working principle of the system is shown in perimeter of its contour. Because the circumference and radius
Fig. 3. of a circle have the following relationship given in (6) and (7):

C = 2πR (6)

R=
#
$
(7)

It can be seen from the formulas that the size of the

circumference of a circle depends on the size of its radius, and
the size of the circumference is proportional to the radius.
From this, it can be inferred that the size of the apple can be
judged by the size of the circumference of the
circle.Perimeters were calculated, as shown in Table I.

TABLE I. PERIMETERS OF DIFFERENT GRADES OF APPLES (UNIT:

PIXELS)
Fig. 3. Schematic Diagram of System Operation
Number Excellent First Level Second Level
B. Application of Computer Vision Technology 1 842.9 755.2 660.2
2 829.4 759.6 675.4
Quality inspection of agricultural products. That is, the 3 833.7 762.2 666.6
inspection and classification of agricultural products. The use 4 840.1 752.4 710.2
of machine vision technology for the quality inspection of 5 829.4 738.5 689.5
agricultural products greatly reduces the labor intensity and
human error, and has the advantages of speed and accuracy. It can be seen from the Table I that the perimeters of
Fruit diameter method: because the curved surface of the fruit different grades of apples are different, and the unit is in
surface is closed, the position coordinates of its centroid can pixels. After converting it to the equivalent diameter, the
be calculated by (1) and (2). equivalent diameter is shown in Table II.

∬
= , ∈
TABLE II. EQUIVALENT DIAMETERS OF DIFFERENT GRADES OF

∬
APPLES (UNIT: PIXELS)
(1)
, ∈ Number 1 2 3 4 5 Average
value
Excellent 268.4 264.1 265.5 267.5 264.1 265.9
First Level 240.5 241.9 242.7 239.6 235.2 240.0

Authorized licensed use limited to: AISSMS's Institute of Info Technology - PUNE. Downloaded on January 02,2025 at 05:26:13 UTC from IEEE Xplore. Restrictions apply.
 point P in the three-dimensional space are (:; , <; , ,1)> and
Second 210.3 215.1 212.3 226.2 219.6 216.7 system, the homogeneous coordinates corresponding to a

(:? , <? , @? , 1)> respectively, and the conversion relationship

Level

Application of automatic harvester, such as fruit automatic is as (12):

:? :;
picking machine, etc. The machine vision part of this

< B C <;
- ?1 = A > D- 1
automatic harvester is generally combined with a CCD camera

@? 0 1 @;
and a laser sensor to complete the comprehensive (12)
1 1
measurement of three-dimensional objects. The
transformation relationship between a point in the image pixel

Among them, A is an orthogonal identity matrix of 3 × 3,

system of coordinates and the physics coordinate system given

a is a three-dimensional translation vector, 0 = (0,0,0)G

as (8) and (9).

u= +u (8) The correlation between the world coordinate system and

v= +v
the physical coordinate system of the image is obtained, as
(9) seen in Fig. 4.
P ( X w , Yw , Z w )
It is converted to homogeneous coordinates and matrix
Zw
form (10): Zc

0 u
.
) /
(* , = - 0 v 1
Xw

.
1
Yw
(10)
0 0 1
It is converted to an inverse relationship as (11): Xc

3 0 −u dx )
( ,=20 3 −v dy8 (* ,
1 1
Yc

0 0 1
(11)
Fig. 4. Pinhole Model of the Camera

Let the rotation matrix be A and the translation vector be Fig. 5 shows the main function implementation process.
a. In the world coordinate system and the camera coordinate

Fig. 5. Process Flow of Visual Task Processing

CCD cameras can identify and interpret objects in the form

of images. The overall composition of the computer vision
system is shown in Fig. 6.

Fig. 6. Components of A Computer Vision System

Authorized licensed use limited to: AISSMS's Institute of Info Technology - PUNE. Downloaded on January 02,2025 at 05:26:13 UTC from IEEE Xplore. Restrictions apply.
 IV. RESULTS AND DISCUSSION suitable for small focal length lenses. It can be seen from Fig.
8 that the error rate of machine grading results is 11.8% and
TABLE III. APPLE GRADING PARAMETERS below, and the error rate of apple grading outside the grade is
0. The error rate is mainly concentrated between the primary
Level Small Fruit Medium-sized Large fruit and secondary results. The detected results can basically
fruits
reflect the actual size and can be applied in actual production.
Excellent ≥60mm ≥65mm ≥80mm
First Level ≥55mm ≥60mm ≥75mm The measurement results are shown in Fig. 9 and Fig. 10.
Second Level ≥50mm ≥55mm ≥70mm The air temperature value from 0:00 to 24:00 of the day is
Other <50mm <55mm <70mm
shown in Fig. 9(a), the air humidity value is shown in Fig.
In this paper, collected the data of apple planting in Jilin 9(b), the soil temperature value is shown in Fig. 10(a), and the
Province, China, and classified the apple size in Jilin Province. soil humidity value is shown in Fig. 10(b).
In the apple grading standard, the grade indicators of large,
medium and small fruits are different, among which the grade
indicators of large, medium and small are shown in Table 3.
V. INTELLIGENT AGRICULTURAL AUTOMATION SYSTEM
In the experiment, under the conditions of ring light source
and natural light, a video was shot through the 8mm lens and
16mm lens of the CCD camera. 280 frames of pictures were
extracted and tested, and the system was tested. Fig. 7 shows
the matching results of apples under natural lighting
conditions in the laboratory and under the ring light source.
Fig. 7(a) shows the matching results of the 8mm lens under
two light sources, and Fig. 7(b) shows the matching results of
Fig. 9. Air Temperature and Humidity at No. 1 Node In the Warehouse
the 16mm lens under the two light sources. In the experiment,
the equivalent diameter method was applied to detect the
maximum diameter of the apple, which was used as the basis
for its size detection. The specific experimental detection
results are shown in Fig. 8. Fig. 8(a) shows the experimental
results of manual and machine grading of apples, and Fig. 8(b)
shows the error probability of machine grading.

Fig. 10. Soil Temperature and Humidity at No. 1 Node in the Warehouse

The performances of mean absolute error and intersection

over union are shown in table IV.

Fig. 7. Statistical Results of Apple Azimuth and Orientation Sorting in Each TABLE IV. Mean Absolute Error And Intersection Over
Experimental Environment Union
Model MEA Iou
This paper model 0.012 0.85
SVM 0.024 0.72
RF 0.034 0.84
DT 0.032 0.83

In this study, the automatic grading of fruits and the quality

inspection of agricultural products were realized by applying
computer vision technology to intelligent agricultural
automation system. The experimental results show that the
system can quickly and accurately detect and grade apples in
Jilin Province in the laboratory environment. The error rate of
Fig. 8. Experimental Test Results machine classification is not higher than 11.8%, and zero error
can be achieved outside the classification. This paper aims to
When the extraction target area is more accurate, the solve the challenges faced by agriculture in Jilin Province,
matching effect is better, and the sorting accuracy is higher. including poor soil, cold climate and fruit quality
The 8mm lens of the CCD camera has a better matching effect management. Computer vision technology has potential
than the 16mm lens, because the experimental size is smaller application in intelligent agriculture, which can significantly
and the width of the field of view is narrower, so it is more improve the efficiency of agricultural product quality

Authorized licensed use limited to: AISSMS's Institute of Info Technology - PUNE. Downloaded on January 02,2025 at 05:26:13 UTC from IEEE Xplore. Restrictions apply.
 inspection and fruit grading. Compared with other similar AUTHOR CONTRIBUTIONS
intelligent agricultural automation systems, the text-based The corresponding author is Xingyuan Qi
system shows a lower error rate in fruit grading. Through
automatic grading and quality inspection, farmers can reduce REFERENCES
production costs, increase income and provide high-quality [1] Jawad H M , Jawad A M , Nordin R. Accurate Empirical Path-Loss
fruit products. Model Based on Particle Swarm Optimization for Wireless Sensor
Networks in Smart Agriculture[J]. IEEE Sensors Journal, 2019,
It can be seen from Fig. 9 and Fig. 10 that under normal 20(1):552-561.
circumstances, the air temperature value, air humidity value, [2] Brun-Laguna K , Laura Diedrichs A , Dujovne D. Using
soil temperature value and soil humidity value in the SmartMesh IP in Smart Agriculture and Smart Building
warehouse fluctuate very little. The fluctuation value of air applications[J]. Computer Communications, 2018, 121(MAY):83-90.
temperature value and air humidity value both should not [3] Kiran, B. D. Parameshachari, and H. T. Panduranga. "Medical image
exceed 2%, the fluctuation of soil temperature value should encryption using SCAN technique and chaotic tent map system." In
not exceed 0.5℃, and the fluctuation of soil moisture value Recent Advances in Artificial Intelligence and Data Engineering:
Select Proceedings of AIDE 2020, pp. 181-193. Springer Singapore,
should not exceed 1%. 2022.
VI. CONCLUSION [4] Roopaei M , Rad P , Choo K . Cloud of Things in Smart
Agriculture: Intelligent Irrigation Monitoring by Thermal Imaging[J].
The automatic sorting system for apples designed in this IEEE Cloud Computing, 2017, 4(1):10-15.
paper not only adapts to the automatic sorting of the [5] Yan Rui, Wang Zhen, Li Yanhao, Li Zhemin, Li Xian. "Application,
orientation of the fruit, but also realizes the automation of Problems and Development of Agricultural Intelligent Sensors in
production and the intelligentization of industrial operations. China." Journal of Agricultural Big Data 3.2 (2021): 3-15.
From the perspective of the trend of China's agricultural [6] Ren Ni, Guo Ting, Sun Yiwei, Dai Hongjun, Zhang Cheng. "Analysis
of the research situation in the field of global smart agriculture."
development, China must transform traditional agriculture Agricultural Books journal of the china society for scientific and
through informatization means to promote its modernization technical information 33.9 (2021): 48-63.
and informatization. By classifying apples into small, medium [7] Chen W , Lu S , Liu B. Detecting Citrus in Orchard Environment
and large fruits and using machine learning algorithm, the by Using Improved YOLOv4[J]. Scientific Programming, 2020,
error rate of machine classification results is not higher than 2020(1):1-13.
11.8%. Compared with the traditional manual classification [8] Morshed A , Jayaraman P P , Sellis T . Deep Osmosis: Holistic
method, it reduces the labor intensity and human error, and Distributed Deep Learning in Osmotic Computing[J]. IEEE Cloud
Computing, 2018, 4(6):22-32.
can realize agricultural automation. The environmental
parameters in the warehouse were monitored in real time, [9] Kheirkhah F M , Asghari H . Plant Leaf Classification Using GIST
Texture Features[J]. IET Computer Vision, 2019, 13(4):369-375.
including air temperature, air humidity, soil temperature and
[10] A Sanz-Cobena, W Ding, C Mller. Mitigating greenhouse gas
soil humidity. This is helpful to accurately manage emissions from croplands and pasturelands — climate-smart
agricultural resources and improve production efficiency. agriculture[J]. Pedosphere, 2021, 31(2):227-230.
Therefore, it is an significant direction for the future [11] Demarchi D , Georgiou J , Grimblatt V . Guest Editorial Circuits
development of China's agriculture to fully understand the and Systems for Smart Agriculture and Healthy Foods[J]. IEEE Journal
theoretical basis and technical principles of international on Emerging and Selected Topics in Circuits and Systems, 2021,
intelligent agriculture development based on the current 11(3):431-434.
situation of China's agriculture, and then combine China's [12] Kiran, P., and B. D. Parameshachari. "Resource optimized selective
image encryption of medical images using multiple chaotic systems."
specific conditions to research and develop an intelligent Microprocessors and Microsystems 91 (2022): 104546.
agricultural technology system suitable for China's national [13] Roy A M , Bose R , Bhaduri J . A fast accurate fine-grain object
conditions. The experiment is carried out under specific detection model based on YOLOv4 deep neural network[J]. Neural
environmental conditions, and different climate and planting Computing and Applications, 2022, 34(5):3895-3921.
conditions may affect the performance of the system. Expand [14] Kasinathan T , Uyyala S R . Machine learning ensemble with image
the research scope in the future, including different types of processing for pest identification and classification in field crops[J].
agricultural products and planting environment. Neural Computing and Applications, 2021, 33(13):7491-7504.

Authorized licensed use limited to: AISSMS's Institute of Info Technology - PUNE. Downloaded on January 02,2025 at 05:26:13 UTC from IEEE Xplore. Restrictions apply.