Computers and Electronics in Agriculture 205 (2023) 107596

Contents lists available at ScienceDirect

Computers and Electronics in Agriculture

Original papers

Prediction of the remaining time of the foraging activity of honey bees using
spatio-temporal correction and periodic model re-fitting
Paweł Majewski a ,∗, Piotr Lampa b , Robert Burduk a , Jacek Reiner b
a
Faculty of Information and Communication Technology, Wrocław University of Science and Technology, Poland
b
Faculty of Mechanical Engineering, Wrocław University of Science and Technology, Poland

ARTICLE INFO ABSTRACT

Keywords: The problem of bee poisoning causes significant losses to the beekeeping sector every year. One cause of bee
Honey bee poisoning is spraying before the end of the foraging activity of bees. Information about the estimated end of
Foraging activity this foraging activity can significantly help a farmer plan his spraying. The aim of our research was to develop
Machine learning
a method based on machine learning models to predict the remaining time of the foraging activity, taking
Computer vision
into account bee activity, weather conditions, and the amount of time to sunset. Data were collected using
Concept drift
IoT
an IoT system from 3 hives in the 2021 and 2022 beekeeping seasons. The proposed method addresses the
challenge of the changing nature of data during the beekeeping season by using periodic model re-fitting with
automatically generated semi-true target values. The veracity of semi-true target values was also improved
by a spatio-temporal correction mechanism based on the position and orientation of the bees, which made it
possible to distinguish foraging from other patterns of bee behavior (dead bees, hive ventilation by bees). The
results of the RMSE prediction error of 23.1 min (season 2021) and 26.5 min (season 2022) prove the high
potential of the proposed method to predict the remaining time of the foraging activity of bees, as well the
lack of need for expert annotation of data during the season. The used approach, based on density occurrence
maps in spatio-temporal correction, can also be used in the future to detect and study bee behavior patterns.

1. Introduction that supports the farmer’s decision on the possibility of spraying; and
(III) unclear legal regulations. A comprehensive advisory system with a
The need to increase food production for an ever-growing human platform for information exchange between the farmer and beekeeper
population means that the intensification of agriculture is unavoidable. could help in terms of making win-win decisions. The operation of the
Plant protection products are undoubtedly necessary for the effective system would be based on the registration of spraying that is planned
control of pests and weeds, and also for the prevention of diseases, but
(carried out) by the farmer, and would take into account the location
their improper use can lead to environmental degradation.
of the farmer’s field, the type of crop, and the type of used pesticide.
In the age of agricultural intensification, bee poisoning is not un-
The system would also have information about: (1) legal regulations;
common. According to a report by the Apiculture Division in Puławy
(Poland), more than 25,000 bee colony poisonings were reported in (2) expert knowledge and good agricultural practices; (3) the location
Poland in 2020 (Semkiw, 2020), the main cause of which was the of apiaries in the region; and (4) the current state of apiaries in terms
spraying of rapeseed (especially spraying against the rapeseed pollen of ongoing bee foraging activity. Taking into account the input data,
beetle, Brassicogethes aeneus at the wrong time, including during the the system would suggest the optimal time for the farmer to carry out
flowering of crops or associated weeds, and also when the bees had not the spraying. An important part of such a system would be a model
finished foraging. Losses caused by bee poisoning are mainly connected for the prediction of the time remaining to the end of bee foraging
with: (1) the lack, or reduction, in the amount of obtained bee prod- activity, which would in turn allow the farmer to schedule spraying
ucts (honey, bee pollen, wax, propolis, royal jelly); (2) the collapse, in advance and maximize the time between spraying and the start of
weakening, or inhibition of the development of bee colonies; and
bee activity for the following day. The prediction model would use
(3) the failure of bees to pollinate crops (Skubida, 2007). The reasons
processed raw data from the IoT system (e.g. images, weather data).
for the occurrence of bee poisoning include: (I) poor communication
With up-to-date information from the apiary, the problem of poisoning
between the beekeeper and the farmer; (II) the lack of a uniform system

∗ Corresponding author.
E-mail address: [email protected] (P. Majewski).

https://doi.org/10.1016/j.compag.2022.107596
Received 14 June 2022; Received in revised form 26 September 2022; Accepted 26 December 2022
Available online 3 January 2023
0168-1699/© 2022 Elsevier B.V. All rights reserved.
P. Majewski et al. Computers and Electronics in Agriculture 205 (2023) 107596

caused by spraying too early, when the bees have not yet finished their hive, or with the daily functioning of the bees. Minimizing the impact
foraging activity, would also be eliminated. of an IoT system on these two aspects should be a key consideration
Researchers have often addressed the use of computer vision and when developing monitoring systems for an apiary.
machine learning methods to monitor bees. The first such studies The aim of our work was to develop an efficient and robust method
concerned the detection of bees at the entrance to the hive. For bee de- for predicting the time remaining to the end of the daily foraging
tection, both classical computer vision methods (Campbell et al., 2008), activity of bees. The method takes into account the varying nature of
and newer models for object detection based on deep convolutional the input data, which is based on data acquired from an IoT system. The
networks (Ryu et al., 2021; Dembski and Szymański, 2020) were used. main achievements of our work are: (1) the development of a model for
In the literature, studies dedicated to issues such as detecting bee pollen predicting the remaining time of the foraging activity of bees, which
loads (Rodriguez et al., 2018; Stojnić et al., 2018), the cell classification takes into account bee activity, weather conditions, and the amount of
of bee frames (Alves et al., 2020), the detection of Varroa destructor time to sunset; (2) the development of a mechanism to maintain the
parasites on bees (Bjerge et al., 2019), the tracking of bees (Bozek et al., quality of the models for long-term observation (the duration of the
2021; Ngo et al., 2019; Bozek et al., 2018), and the re-identification of beekeeping season), using periodic model re-fitting with automatically
bees (Chan. et al., 2022) can also be found. It is also worth noting that generated semi-true target values; (3) the proposed method for spatio-
there are papers on the development of IoT systems for apiaries, which temporal correction taking into account the location and orientation of
enable their monitoring in real-time (Ngo et al., 2021a; Marstaller et al., bees that reduces the error of semi-true target values determination;
2019; Tashakkori et al., 2021). (4) a modular, non-invasive and versatile IoT system enabling real-
A much smaller number of papers concern the analysis of long- time data collection and analysis; and (5) multi-faceted validation and
term bee activity and the prediction of bee behavior. Gomes et al. parameter fine-tuning of the proposed methods based on data from the
(2020) predicted bee foraging activity using recurrent neural networks 2021 and 2022 beekeeping seasons.
based on a time series of activity level, temperature, solar radiation,
2. Materials and methods
and barometric pressure within a time window of a specified length.
The researchers used RFID tagging of bees to record their activity.
This section addresses the following topics successively: (1) the
The activity level was calculated for each hour, and the optimal time
definition of the problem and the scheme of the proposed solution; (2)
window size was 24 h. A significant limitation of the method proposed
the development of the data acquisition station; (3) the characteristics
in this paper is the recording of bee activity through RFID tagging.
of the collected data; (4) methods of detecting bees and determining
This cannot be applied to noninvasive IoT systems, which should be
their orientation; (5) regression models for predicting the remaining
the basis of apiary monitoring. Ngo et al. (2021b) predicted daily
time of the foraging activity of bees; (6) the initial model fitting and
bee losses using temporal convolutional networks (Lea et al., 2016)
periodic model re-fitting strategy; (7) the spatio-temporal correction of
based on bee activity (represented by the number of bees entering and
the registered number of bees; (8) fine-tuning of the parameters for the
leaving the hive), temperature, humidity, and wind and rainfall-related proposed methods; and (9) the types of metrics used in the evaluation
features. Clarke and Robert (2018) modeled the foraging activity of of the proposed methods.
bees (the bee egress rate) based on temperature, solar radiation, atmo-
spheric pressure, humidity, rainfall, wind direction, and speed using 2.1. Definition of the problem
the ordinary-least-squares model. The authors reported that 78% of
the observed variation in bee activity was explained by variations in The considered problem is the prediction of the time remaining to
temperature and solar radiation. Andrijević et al. (2022) modeled the the end of the daily foraging activity of bees, and takes into account
hourly activity of bees entering and exiting the hive using multidomain the following features:
characteristics collected inside and outside the hive. The researchers
used ARIMA (Box et al., 2015), Prophet (Taylor and Letham, 2018), and 1. the time remaining until sunset 𝛥𝑡𝑠𝑢𝑛𝑠𝑒𝑡 (the sunset time is known
LSTM (Hochreiter and Schmidhuber, 1997) models in order to develop for each calendar day),
prediction models. A bee counting sensor array mounted at the entrance 2. the daily bee activity [𝑎1 , 𝑎2 , … , 𝑎𝑛 ] in a time window of length
to the hive was used to record activity. Undoubtedly, the approach 𝑡𝑐ℎ𝑢𝑛𝑘 ,
presented in this paper, with the monitoring of multiple environmental 3. weather characteristics (temperature [𝑇1 , 𝑇2 , … , 𝑇𝑛 ], humidity
factors, can be seen to be reasonable from a research perspective. [𝜙1 , 𝜙2 , … , 𝜙𝑛 ], and barometric pressure [𝑃1 , 𝑃2 , … , 𝑃𝑛 ]) in a time
However, when designing systems to support the beekeeper’s decision, window of length 𝑡𝑐ℎ𝑢𝑛𝑘 .
one must consider the trade-off between the cost and invasiveness of The scheme of the proposed solution is shown in Fig. 1. The next
sensors and the quality of information obtained by the beekeeper. steps of developing the method are described in the following sections.
Although the work described above has demonstrated the possibility
and potential of using computer vision and machine learning to address 2.2. Data acquisition station
issues of long-term bee monitoring and prediction, the researchers
did not explicitly explore the problem of maintaining high model The designed acquisition station (Fig. 2) enabled images from the
performance during the entire beekeeping season. Considering the dy- hive entrance to be captured, and also the weather conditions (tem-
namic nature of bee colony development, as well as changing weather perature, humidity, barometric pressure) at a given sampling period
conditions, it is expected that the character of the input data for the to be recorded. The image acquisition modules were mounted on 3
models will change significantly, with the development of adaptation hives, and the weather sensor was placed near them in a radiation
mechanisms being crucial in the context of the developed solutions. The shield at a height of approximately 1.5 m. Images with a resolution
described phenomenon of changing the distribution of data over time of 1920 × 1080 pixels were collected using Raspberry Pi Cameras V2,
is called concept drift, and methods related to responding to concept which are controlled by SBCs (Single Board Computers) and Raspberry
drift for streaming data are a current research topic, also in the field Pi 4B. The SBCs were connected to a local WiFi network, which was
of insect observations (Rustia et al., 2021; de Souza et al., 2013). In in turn connected to the Internet. For the described study, data were
the literature, studies related to the direct prediction of the end time of collected every 1 min from 3 hive-mounted stations from sunrise to
bee foraging activity, which is important information from the point of sunset with an offset of 1 h. The collected data were uploaded to a cloud
view of a farmer who wants to spray, were not found. It should also be database once a day. The hardware part of the IoT system was non-
noted that a significant limitation of some of the proposed solutions are invasive (both to the bees’ functioning and to the hives’ construction)
IoT systems that significantly interfere with the design of a particular and modular (ease of replicating the station for more hives).

2
P. Majewski et al. Computers and Electronics in Agriculture 205 (2023) 107596

Fig. 1. Scheme for the proposed solution.

Fig. 2. Data acquisition station.

2.3. Data observation was recorded no earlier than 6 h before sunset. An explana-
tion of the pre-processing and the parameters 𝑡𝑐ℎ𝑢𝑛𝑘 , 𝑡𝑏𝑖𝑛 is also provided
The data used in the study were collected in the 2021 and 2022 in Fig. 1.
beekeeping seasons. The end of the foraging activity of the bees was
analyzed from 26 April to 29 June in the 2021 season, and from 12 2.4. Bee detection and orientation determination
April to 5 June in the 2022 season. For each day and for each hive
monitored, an expert determined the end of the bee foraging activity Bee detection was performed using the Mask R-CNN (He et al.,
using a sequence of images collected on that day. Samples collected 2017) model, which was trained on samples of real images. Images
during rainy days were not used for annotation due to the fact that differing in acquisition time (e.g. early morning, evening), bee density,
there is no bee foraging and the farmer is not able to spray. A summary the presence of overexposure, and the bee growth stage were selected
of the end times of the bee foraging activity for the 2021 and 2022 for the training set. Labeling involved manually drawing polygons for
seasons is presented in Fig. 3. subsequent instances using labelme software (Wada, 2018). In total,
The raw data collected from each station was divided into chunks, 143 images containing 1047 labeled bee instances were used for the
which were then used by the proposed machine learning models for training. The bee detection model was validated on a test set that con-
training and prediction. Each chunk was characterized by a specific tained 37 images (211 labeled bee instances). Samples from different
length 𝑡𝑐ℎ𝑢𝑛𝑘 , which determines what historical data should be included days were selected for the training and test sets in order to ensure
in the chunk. To eliminate noise and to reduce dimensionality, instead independence between these sets. ResNet50 (He et al., 2016) was used
of using raw feature values (number of bees, weather indicators), as the backbone for the Mask R-CNN. The obtained binary masks after
averaged values in bins of a specific length 𝑡𝑏𝑖𝑛 were used, where Mask R-CNN inference allowed for the determination of the midpoint
𝑡𝑏𝑖𝑛 < 𝑡𝑐ℎ𝑢𝑛𝑘 . In this study, chunks were used in which the youngest and orientation for the detected bees. The orientation was determined

3
P. Majewski et al. Computers and Electronics in Agriculture 205 (2023) 107596

Fig. 3. Visualization of the end times of the bee foraging activity for the 2021 and 2022 beekeeping seasons.

from the skeleton coordinates obtained after skeletonization (Zhang on the analyzed image sequence. After receiving the time of the end of
and Suen, 1984) of the binary mask using linear regression. The study the bee foraging activity, chunks are sequentially annotated with the
used the Mask R-CNN implementation from the detectron2 (Wu et al., corresponding RBFAT value and added to the training set. The study
2019) library and the skeletonization algorithm implementation from also proposed methods for automatic determination of the end of the
the scikit-image (van der Walt et al., 2014) library. bee foraging activity (type semi_true target values). The most intuitive
method to determine the end of daily bee foraging activity is the time
2.5. Initial fitting and re-fitting of the regression model of observing the last bee located at the entrance of the hive (type
semi_true_raw_last target values).
Regression analysis techniques were used to predict the time re- After each day, the model is re-fitted on the current training set.
maining until the end of the foraging activity of the bees. The op- The study considered the following options for modifying the training
timality of using a specific model depends on the character of the set:
data, and for this reason we checked different regression models, and • fixed - only initial fitting is performed, the training set is not
selected the best one based on the lowest prediction error (RMSE). The modified, no model re-fitting occurs,
following models from the scikit-learn (Pedregosa et al., 2011) Python • landmark - initial fitting and re-fitting of the model is performed,
library were evaluated: GradientBoostingRegressor (GBR), LinearRegres- the training set is continuously increased, no removal of older
sion (LR), HuberRegressor (HR), BayesianRidge (BR), KernelRidge (KR), samples occurs,
MLPRegressor (MLP). The default parameter values for these models
• sliding - initial fitting and re-fitting of the model is performed,
were used. the training set remains similar in size and contains samples from
In addition, the ‘FixedThresholdBaseline’ (FTB) model was defined days within a time window of length 𝑡𝑡𝑟𝑎𝑖𝑛 , removal of samples
as the baseline. It is based on determining such an offset to the time outside the time window occurs.
to sunset that has the lowest prediction error. For the ‘FixedThreshold-
Baseline’ model, features related to bee activity and weather conditions In order to protect the model from semi-true target values that are
are not included. determined with high error, a regularization mechanism characterized
by the parameter 𝜆 is proposed. The regularization involves that the
2.6. Regression model initial fitting and re-fitting final RBFAT value 𝑌𝑛𝑒𝑤 consists of two components: 𝑌𝑠𝑒𝑚𝑖_𝑡𝑟𝑢𝑒 - represent-
ing the determined semi-true RBFAT value, and 𝑌𝑝𝑟𝑒𝑑 - representing the
An important step in the development of regression models is their predicted RBFAT value using the old model. Finally, the final RBFAT
initial fitting and eventual re-fitting. In our study, two types of fitting value is calculated using the formula:
were considered. 𝑌𝑛𝑒𝑤 = 𝜆𝑌𝑠𝑒𝑚𝑖_𝑡𝑟𝑢𝑒 + (1 − 𝜆)𝑌𝑝𝑟𝑒𝑑 (1)
The initial model fitting consisted of fitting the model on an initial
training set that contains samples from days within a time window of For the initial training, only component 𝑌𝑠𝑒𝑚𝑖_𝑡𝑟𝑢𝑒 is used. The 𝜆
length 𝑡𝑡𝑟𝑎𝑖𝑛 . parameter can take values from 0 to 1, and specifies the percentage
Maintaining high accuracy of the model when the nature of the importance of the 𝑌𝑠𝑒𝑚𝑖_𝑡𝑟𝑢𝑒 component in the formation of 𝑌𝑛𝑒𝑤 . By
input data changes requires periodic model re-fitting. The problem using a linear combination of 𝑌𝑠𝑒𝑚𝑖_𝑡𝑟𝑢𝑒 and 𝑌𝑝𝑟𝑒𝑑 in the formula for
under consideration is characterized by the ability to retrieve true (or 𝑌𝑛𝑒𝑤 and a range of ⟨0; 1⟩ for the 𝜆 parameter, 𝑌𝑛𝑒𝑤 is always within
semi-true) target values (represented by the remaining bee foraging the interval ⟨𝑌𝑚𝑖𝑛 , 𝑌𝑚𝑎𝑥 ⟩, where 𝑌𝑚𝑖𝑛 = 𝑚𝑖𝑛{𝑌𝑠𝑒𝑚𝑖_𝑡𝑟𝑢𝑒 , 𝑌𝑝𝑟𝑒𝑑 } and 𝑌𝑚𝑎𝑥 =
activity time (RBFAT) values) with a delay, i.e., at the earliest time after 𝑚𝑎𝑥{𝑌𝑠𝑒𝑚𝑖_𝑡𝑟𝑢𝑒 , 𝑌𝑝𝑟𝑒𝑑 }
the bees have finished their daily foraging activity. Annotation by an
expert (type true target values) for each day and each hive involves the
expert determining the time of the end of bee foraging activity based

4
P. Majewski et al. Computers and Electronics in Agriculture 205 (2023) 107596

Fig. 4. The spatio-temporal correction: (1) subsequent samples taken for the calculation occurrence density map, (2) diagram of the calculation occurrence density map, (3)
diagram of the calculation of the corrected number of bees.

2.7. Spatio-temporal correction correlated bee 𝜃𝑖 is then compared to the orientation of the nearest
bee 𝜃𝑛𝑒𝑎𝑟 in the previous image. If |𝜃𝑖 − 𝜃𝑛𝑒𝑎𝑟 | < 𝑇𝑜𝑟𝑖𝑒𝑛𝑡 , the detected
The presence of bees at the entrance of the hive is not synonymous bee should be considered as dead and is not taken into account when
with foraging activity. It is also possible to observe: (1) dead bees, counting all the bees. Additionally, the nearest bee from the previous
(2) bees ventilating the hive, and (3) bees performing cleaning tasks, image, to be considered as dead, should be at a distance less than
e.g. removing dead bees from the hive. These patterns can be detected 𝑟𝑜𝑟𝑖𝑒𝑛𝑡 . The bee counting with spatio-temporal correction is summarized
by analyzing similarities in the location of bees in successively captured in Fig. 4.
After calculating the corrected number of bees for a given times-
images.
tamp, the occurrence density map 𝐷 is updated. The most relevant
In the context of the conducted research, the described phenomena
(having the highest weight) for the calculation of 𝐷 are the locations of
make it difficult to correctly determine the end of the daily foraging
bees detected in recently captured images. The weights were calculated
activity of bees, because their occurrence can be incorrectly perceived
by taking into account the differences between the recording time of
as ongoing bee foraging. This problem is significant, especially when the current image and the recording time of the previous image 𝑡𝑑𝑖𝑓 𝑓
determining the automatic time of the end of bee foraging activity. The using the following formula:
use of the naive semi-true target values determination strategy of treat-
ing the last detected bee as the end of bee foraging (semi_true_raw_last ) 𝑤𝑒𝑖𝑔ℎ𝑡 = 𝑒−𝑡𝑑𝑖𝑓 𝑓 ∕𝑎 (2)
causes the automatically determined end time of bee foraging to be The weights decrease exponentially as 𝑡𝑑𝑖𝑓 𝑓 increases. A threshold of
later than the true time (when other behavior patterns are present). weights was set at 𝑤𝑡ℎ𝑟𝑒𝑠ℎ in order to eliminate insignificant instances
In order to reduce the error of the automatically determined time of concerning map estimation. After obtaining the raw form of the oc-
the end of bee foraging activity, a spatio-temporal correction algorithm currence density map, a Gaussian blur of the map, characterized by
was proposed. It excludes or reduces the weight of bees, which are the standard deviation for the Gaussian kernel 𝜎𝑙𝑜𝑐 , was applied. The
characterized by a similar location and orientation with respect to bees calculation of the occurrence density map is summarized in Fig. 4.
from previously captured images. In this study, 𝑇𝑜𝑟𝑖𝑒𝑛𝑡 = 10◦ , 𝑟𝑜𝑟𝑖𝑒𝑛𝑡 = 20 pixel, and 𝑤𝑡ℎ𝑟𝑒𝑠ℎ = 0.01 were
For each time step associated with the acquisition of a new im- assumed, and the parameter values 𝑎, 𝜎𝑙𝑜𝑐 , 𝑇𝑙𝑜𝑐 were fine-tuned. The
age after prediction by the Mask R-CNN, a binary mask for each parameter values used for the fine-tuning are listed in Table 1.
detected bee is obtained. For each binary mask, we compute the Two types of semi-true target values, determined after spatio-
midpoint 𝑀𝑖 (𝑥𝑖 , 𝑦𝑖 ) and the orientation 𝜃𝑖 , thus obtaining the set of temporal correction, were defined. The 𝑠𝑒𝑚𝑖_𝑡𝑟𝑢𝑒_𝑆𝑇 𝐶_𝑙𝑎𝑠𝑡 approach is
bee locations [(𝑥1 , 𝑦𝑖 ), (𝑥2 , 𝑦2 ), ...., (𝑥𝑛 , 𝑦𝑛 )] and the set of bee orientations connected with the last bee that is observed after removing the bees
with a high location and orientation similarity (most probably dead
[𝜃1 , 𝜃2 , … , 𝜃𝑛 ], where 𝑛 is the number of detected bees. The location of
bees), and the 𝑠𝑒𝑚𝑖_𝑡𝑟𝑢𝑒_𝑆𝑇 𝐶_𝑛𝑜_𝑐𝑜𝑟𝑟 approach is connected with the
each bee is compared to the actual occurrence density map 𝐷, which
last bee for which 𝐷(𝑥𝑖 , 𝑦𝑖 ) < 𝑇𝑙𝑜𝑐 , allowing bees with a high location
is calculated based on the location of the detected bees in the previous
similarity (e.g., bees that ventilate the hive) to be excluded.
image. If the location of a bee (𝑥𝑖 , 𝑦𝑖 ) indicates a point with a higher
probability of occurrence (𝐷(𝑥𝑖 , 𝑦𝑖 ) > 𝑇𝑙𝑜𝑐 ), its weight is reduced relative 2.8. Fine-tuning of the methods’ parameters
to the bees located in an area with a lower probability of occurrence.
The contribution of a correlated bee to the total number of bees in the A summary of the parameters for the proposed methods is shown
𝑇
image is 𝐷(𝑥𝑙𝑜𝑐,𝑦 ) , while an uncorrelated bee – 1. The orientation of the in Table 1. These parameters were fine-tuned in successive stages. In
𝑖 𝑖

5
P. Majewski et al. Computers and Electronics in Agriculture 205 (2023) 107596

Table 1
Parameters for the proposed method.
Stage Parameter Description Values
I 𝑚𝑜𝑑𝑒𝑙 Machine learning model used for prediction [‘GradientBoostingRegressor’ (GBR), ‘LinearRegression’
(LR), ‘HuberRegressor’ (HR), ‘BayesianRidge’ (BR),
‘KernelRidge’ (BR), ‘MLPRegressor’ (MLP),
‘FixedThresholdBaseline’ (FTB)]
II 𝑡𝑐ℎ𝑢𝑛𝑘 Size of time window used for prediction [30, 60, 90, 150, 240] min
𝑡𝑏𝑖𝑛 Size of bin used for features accumulation [1, 2, 5, 10, 30] min
III 𝑢𝑠𝑒𝑡𝑒𝑚𝑝 Flags indicating the use of temperature, humidity, pressure values [000, 100, 110, 101, 111]
𝑢𝑠𝑒ℎ𝑢𝑚 during inference
𝑢𝑠𝑒𝑝𝑟𝑒𝑠𝑠
IV 𝑡𝑡𝑟𝑎𝑖𝑛 Size of time window used to re-fit model [3, 5, 7, 10, 14] days
𝑤𝑡𝑦𝑝𝑒 Type of window used to re-fit model [‘fixed’, ‘landmark’, ‘sliding’]
V 𝑎 Constant used for calculating weights for samples [5, 10, 30, 60]
𝜎𝑙𝑜𝑐 Standard deviation for Gaussian kernel used in blurring of [50, 100, 400] pixels
occurrence density map
𝑇𝑙𝑜𝑐 Threshold to assess whether bee location similarity is significant [0, 1, 2]
VI 𝑡𝑎𝑟𝑔𝑒𝑡_ Type of target values used to re-fit model [‘true’, ‘semi_true_raw_last, ‘semi_true_STC_last’,
𝑣𝑎𝑙𝑢𝑒𝑠_ ‘semi_true_STC_no_corr’]
𝑡𝑦𝑝𝑒
VII 𝜆 Regularization coefficient that determines contribution of [0, 0.1, . . . , 0.9, 1.0]
determined semi-true RBFAT value to new RBFAT value

Table 2
Settings of subsequent parameter fine-tuning stages for the proposed methods.
Stage 𝑚𝑜𝑑𝑒𝑙 𝑡𝑐ℎ𝑢𝑛𝑘 𝑡𝑏𝑖𝑛 𝑢𝑠𝑒𝑡𝑒𝑚𝑝 𝑢𝑠𝑒ℎ𝑢𝑚 𝑢𝑠𝑒𝑝𝑟𝑒𝑠𝑠 𝑡𝑡𝑟𝑎𝑖𝑛 𝑤𝑡𝑦𝑝𝑒 𝑆𝑇 𝐶 𝑎 𝜎𝑙𝑜𝑐 𝑇𝑙𝑜𝑐 𝑡𝑎𝑟𝑔𝑒𝑡 𝑣𝑎𝑙𝑢𝑒𝑠 𝜆
I var 240 10 1 1 1 7 slid. no – – – true 1
II opt var var 1 1 1 7 slid. no – – – true 1
III opt opt opt var var var 7 slid. no – – – true 1
IV opt opt opt opt opt opt var var no – – – true 1
V opt opt opt opt opt opt opt opt yes var var var semi-true 1
VI opt opt opt opt opt opt opt opt yes opt opt opt var 1
VII opt opt opt opt opt opt opt opt yes opt opt opt opt var

each stage, some of the parameters were fixed (suboptimal), while some 3. Results and discussion
were variable, taking the values given in Table 1. The settings for the
subsequent fine-tuning stages are shown in Table 2. Only data from the This section addresses the following topics successively: (1) bee
2021 season were used for fine-tuning of the parameters. The selected detection and segmentation using Mask R-CNN; (2) parameter fine-
optimal parameters were used for validating the method based on data tuning of the proposed methods; (3) prediction results for consecutive
from the 2022 season. days of the 2021 and 2022 beekeeping seasons (as averaged RMSE val-
ues over hives for the compared approaches); (4) intra-day prediction
2.9. Evaluation results for selected days (as absolute values of predicted time); (5) the
calculation of occurrence density maps and their use for behavioral
The evaluation consisted of comparing the true time remaining until pattern detection; (6) a strategy to combine predictions from different
the end of bee foraging activity 𝛥𝑡𝑡𝑟𝑢𝑒 with the predicted time 𝛥𝑡𝑝𝑟𝑒𝑑 . hives; and (7) the adaptability of the proposed methods for stream
The RMSE metric was used to determine the prediction error and was processing.
calculated according to the formula: The first step in implementing the proposed methods was to train
√
√ 𝑛 the Mask R-CNN instance segmentation model using labeled data from
√∑ (𝛥𝑡𝑖𝑡𝑟𝑢𝑒 − 𝛥𝑡𝑖𝑝𝑟𝑒𝑑 )2
𝑅𝑀𝑆𝐸 = √ (3) the 2021 season. The obtained average precision value 𝐴𝑃50 = 94.5
𝑖=1
𝑛 on an independent test set for bee detection is satisfactory from the
point of view of the addressed problem. The large diversity of the
where 𝛥𝑡𝑖𝑡𝑟𝑢𝑒 , 𝛥𝑡𝑖𝑝𝑟𝑒𝑑 denote the true and predicted time remaining until
samples in the training set enabled the model to be robust to varying
the end of bee foraging activity for the ith chunk, and n is the number
bee appearance and size, dense scenes, and overexposure. Exemplary
of chunks.
results for inference by the Mask R-CNN model are presented in Fig. 5,
The referenced RMSE values for individual days considered data
which also shows the result of determining midpoints and orientations
from 3 three stations, while the RMSE values for the entire 2021 and
for the considered samples.
2022 seasons considered data from all considered days in the particular
After the determination of the considered features (weather indices,
season.
number of bees for the corresponding times), parameter fine-tuning was
The second measure used to evaluate the models was the coefficient
carried out for the proposed method of predicting the time remaining
of determination 𝑅2 , which measures how well the model fits the data.
to the end of the daily bee foraging activity. A summary of the results
We calculate the coefficient of determination 𝑅2 using the formula:
∑𝑛 from selected stages is shown in Fig. 6. The bar heights in the figure
𝑖 𝑖 2
𝑅𝑆𝑆 𝑖=1 (𝛥𝑡𝑡𝑟𝑢𝑒 − 𝛥𝑡𝑝𝑟𝑒𝑑 ) represent the RMSE prediction error averaged over the hives and the
𝑅2 = 1 − = 1 − ∑𝑛 (4) days during the 2021 beekeeping season.
𝑇 𝑆𝑆 (𝛥𝑡𝑖 − 𝛥𝑡 )2
𝑖=1 𝑡𝑟𝑢𝑒 𝑡𝑟𝑢𝑒
In stage I, which is related to selecting the regression model for
where RSS, TSS denote the residual and total sum of squares, and 𝛥𝑡𝑡𝑟𝑢𝑒 prediction, it was noted that the machine learning models generally
- the average of the true time remaining until the end of bee foraging performed better than the FTB reference model. The GBR (Gradi-
activity. entBoostingRegressor) model achieved the best results and was used

6
P. Majewski et al. Computers and Electronics in Agriculture 205 (2023) 107596

Fig. 5. Mask R-CNN inference and determination of midpoints (read points) and bee orientations (angle of inclination of the green sections), e.g. samples captured at: (1) 10 a.m.,
(2) 3 p.m..

Fig. 6. Results of the subsequent parameter fine-tuning stages that are related to the selection of: (1) the regression model, (2) the size of the chunk and bin, (3) weather
characteristics (temperature, humidity, pressure), (4) the settings for model re-fitting (type and length of the time window), (5) the type of target values used for re-fitting (with
information about making a correction), (6) the regularization parameter with regards to the calculation of new target values.

in the subsequent fine-tuning stages. In stage II, the optimal chunk about 6 min (𝑅𝑀𝑆𝐸 = 23.1 min) after applying the regularization
and bin size for pre-processing the time-series data was checked. No mechanism and when fine-tuning the 𝜆 parameter. Finally, the differ-
improvement in inference was observed when increasing the chunk ence between the method based on true target values (𝑅𝑀𝑆𝐸 = 18.5
size. Bee activity collected from the last 60 min was sufficient to obtain min) and the method based on semi-true target values after the fine-
optimal results. The parameter values 𝑡𝑐ℎ𝑢𝑛𝑘 = 60 and 𝑡𝑏𝑖𝑛 = 2 were tuning (𝑅𝑀𝑆𝐸 = 23.1 min) was only about 5 min. This demonstrates
used for the next stages. The analyses carried out in stage III, which are the lack of rationale for expert annotation during the season, and
related to weather conditions, showed that temperature was the only the possibility of relying on automatically generated semi-true target
important characteristic to use in the prediction. In stage IV, different values. A summary of the determined optimal values during fine-tuning
settings were checked for the model’s re-fitting. The best results were can be found in Table 3.
obtained with the ‘‘sliding’’ strategy, together with a training window The obtained optimal parameters of the proposed methods were
size of 𝑡𝑡𝑟𝑎𝑖𝑛 = 10. These results are significantly better than when no used for inference in the 2022 season. A summary of the results for the
re-fitting is applied (‘‘fixed’’ strategy), which proves that re-fitting the 2021 and 2022 seasons, showing the prediction error for consecutive
model during the season is necessary to maintain high model quality. observation days and different approaches, is presented in Fig. 7. The
The results also show that older samples included in the ‘‘landmark’’ results for the most important approaches are also summarized in
strategy can be omitted when re-fitting without reducing model ac- Table 4. The RMSE prediction error values for consecutive days (in
curacy. After choosing the optimal parameters for the spatio-temporal Fig. 7) are the averaged RMSE values taken for each hive, whereas
correction in step V, the prediction error for different approaches for the RMSE prediction error values for the entire beekeeping season (in
determining semi-true target values was compared in step VI. The Table 4) are the averaged RMSE values taken for each hive and day of
use of spatio-temporal correction reduced the error by about 23 min observation.
when compared to the results for the 𝑠𝑒𝑚𝑖_𝑡𝑟𝑢𝑒_𝑆𝑇 𝐶_𝑛𝑜_𝑐𝑜𝑟𝑟 and the In the RMSE charts for the 2021 and 2022 seasons (Fig. 7), it
naive 𝑠𝑒𝑚𝑖_𝑡𝑟𝑢𝑒_𝑟𝑎𝑤_𝑙𝑎𝑠𝑡 approaches. This error was further reduced by can be seen that the semi_true_STC_no_corr approach (associated with

7
P. Majewski et al. Computers and Electronics in Agriculture 205 (2023) 107596

Table 3
Optimal method parameters obtained after parameter fine-tuning.
𝑚𝑜𝑑𝑒𝑙 𝑡𝑐ℎ𝑢𝑛𝑘 𝑡𝑏𝑖𝑛 𝑢𝑠𝑒𝑡𝑒𝑚𝑝 𝑢𝑠𝑒ℎ𝑢𝑚 𝑢𝑠𝑒𝑝𝑟𝑒𝑠𝑠 𝑡𝑡𝑟𝑎𝑖𝑛 𝑤𝑡𝑦𝑝𝑒 𝑆𝑇 𝐶 𝑎 𝜎𝑙𝑜𝑐 𝑇𝑙𝑜𝑐 𝑡𝑎𝑟𝑔𝑒𝑡 𝑣𝑎𝑙𝑢𝑒𝑠 𝜆
GBR 60 2 1 0 0 10 slid. yes 60 100 0 semi_true (STC_no_corr) 0.5

Fig. 7. Summary results for the 2021 and 2022 beekeeping seasons, showing the prediction error RMSE (averaged over hives) for consecutive days of observation and selected
approaches.

Fig. 8. Comparison of the absolute prediction values of the time to the end of bee foraging activity with the true values (ground truth) for the intra-day prediction on selected
days.

Table 4 In charts 1–3 in Fig. 8, a significant increase in the RMSE error (rep-
Comparison of results for selected approaches after parameter fine-tuning for the 2021
resented as the distance to the ground truth line) cannot be seen with an
and 2022 beekeeping seasons.
increasing true time remaining until the end of bee foraging activity.
Season 𝑡𝑎𝑟𝑔𝑒𝑡_𝑣𝑎𝑙𝑢𝑒𝑠 RMSE [min] 𝑅2
The reason for keeping the RMSE constant for large times is due to
true 18.5 0.958
the use of the time-to-sunset feature 𝛥𝑡𝑠𝑢𝑛𝑠𝑒𝑡 , which has regularization
2021 semi_true_raw_last 52.5 0.800
semi_true_STC_no_corr (our) 23.1 0.930
properties for predictions. Based on data from previous days, the model
is able to initially estimate the end of bee foraging activity, which
true 27.0 0.899
2022 semi_true_raw_last 71.2 0.660 manifests itself by keeping the RMSE for large times approximately
semi_true_STC_no_corr (our) 26.5 0.906 constant. The observation of reduced bee foraging activity at the end of
the day results in a reduction of the RMSE error, which can be observed
in the 0–60 min range for charts 1 and 2, and in the 0–120 range for
chart 3.
automatically generated semi-true target values) is able to maintain a The occurrence density maps used in this study not only made it
similar RMSE error level as the true approach (associated with expert- possible to increase the veracity of the semi-true target values, but
determined target values throughout the beekeeping season under con- also allowed different patterns of bee behavior to be observed at the
sideration). In order to better understand the prediction of the model, entrance to the hive. Selected bee behavior patterns are shown in Fig. 9.
it is also useful to analyze the change in prediction as a function Dead bees were observed on the occurrence density map as areas of
of the true time remaining until the end of bee foraging activity, a small area, and had a high probability of occurrence (2b in Fig. 9).
as shown in Fig. 8. The figure shows the absolute values of the pre- Ventilation of the hive by bees could be observed as areas of a larger
dicted time in comparison to the real time (ground truth). area, and had an increased probability close to the hive entrance (1b

8
P. Majewski et al. Computers and Electronics in Agriculture 205 (2023) 107596

Fig. 9. Bee behavior patterns (1a, 2a) and corresponding occurrence density maps (1b, 2b): (1) hive ventilation by bees, (2) a dead bee.

in Fig. 9). Density maps provide much more information than the raw automatically generated semi-true target values, and also meant that it
number of bees found at the hive entrance. They allow spatio-temporal was reasonable to replace the true target values with semi-true target
information to be compressed and stored as a single 2D matrix, which values. The evaluation results (RMSE = 23.1 min for 2021 and RMSE
enables it to be used successfully in stream data processing. = 26.5 min for 2022) show that the proposed method of predicting the
In our study, prediction was performed for each hive separately. remaining time of honey bee foraging activity has great potential for
In general, apiaries may consist of many hives, and it is necessary to application in a real-world scenario. The study also proves that it is
consider in the final system how to combine predictions from many possible to maintain high model quality throughout the season without
hives. The most intuitive strategy is the least favorable case, which is the need for additional time-consuming annotation by an expert.
the hive for which the predicted remaining time is the longest. The proposed method to predict the remaining time of bee foraging
The methods proposed in this work provide opportunities for their can be a valuable component of a comprehensive advisory system for
easy adaptation to stream processing. The most computationally ex- the planning of spraying and for exchanging information between farm-
pensive step in the presented approach is bee segmentation using ers and beekeepers. The developed solution can help in the planning of
Mask R-CNN. In order to increase the frequency of prediction, YOLO advance spraying and transparently assess the end of bee foraging on
(Redmon et al., 2016; Jocher et al., 2020) models that are adapted a given day.
for real-time prediction can be considered in the future for object Future work should include: (1) analysis of in-field bee flight activ-
detection. The ‘sliding’ approach will allow the accumulation of only ity and the checking of the relationship of this activity to hive entry
the most recent data, and the occurrence density maps will compress activity; (2) analysis of bee behavior patterns at the hive entrance
the information about changes in the position of the bees in subsequent using occurrence density maps or new feature representations; (3)
images. the expanding of the dataset with more samples, especially data for
different bee species and synthetic images; and (4) the development of
4. Conclusions a multifaceted system to prevent bee poisoning that is integrated with
the methods proposed in this article.
In our study, a method to predict the remaining time of bee foraging
activity, taking into account bee activity, weather conditions, and time CRediT authorship contribution statement
to sunset, was proposed.
Multistage parameter tuning of the proposed method enabled the Paweł Majewski: Conceptualization, Methodology, Software, Vali-
optimal settings for minimizing prediction errors to be selected. GBR dation, Formal analysis, Investigation, Data curation, Writing – original
(Gradient Boosting Regressor) turned out to be the best regression draft. Piotr Lampa: Conceptualization, Software, Data curation, Visu-
model. Taking into account changes in temperature resulted in a de- alization, Writing – review & editing. Robert Burduk: Supervision,
crease in the prediction error, with no effect on prediction errors being Writing – review & editing. Jacek Reiner: Conceptualization, Super-
observed when humidity and pressure values were considered. The vision, Writing – review & editing, Project administration, Funding
‘sliding’ strategy was found to be the most appropriate when updating acquisition.
the training set.
The observation of significant changes in the nature of the data Declaration of competing interest
during the beekeeping season necessitated the proposal of mechanisms
to maintain the quality of the model. The proposed mechanism of The authors declare that they have no known competing financial
spatio-temporal correction, periodic model re-fitting, and regulariza- interests or personal relationships that could have appeared to
tion enabled significant error reduction for the methods based on influence the work reported in this paper.

9
P. Majewski et al. Computers and Electronics in Agriculture 205 (2023) 107596

Data availability Hochreiter, S., Schmidhuber, J., 1997. Long short-term memory. Neural Comput. 9 (8),
1735–1780.
Jocher, G., Nishimura, K., Mineeva, T., Vilariño, R., 2020. YOLOv5. https://github.
Data will be made available on request.
com/ultralytics/yolov5.
Lea, C., Vidal, R., Reiter, A., Hager, G.D., 2016. Temporal convolutional networks: A
Acknowledgments unified approach to action segmentation. In: European Conference on Computer
Vision. Springer, pp. 47–54.
We would like to thank Henryk Majewski from H.T. Majewscy Marstaller, J., Tausch, F., Stock, S., 2019. Deepbees-building and scaling convolutional
neuronal nets for fast and large-scale visual monitoring of bee hives. In: Proceedings
apiary (Łomnica-Folwark, Poland) for providing a data source and of the IEEE/CVF International Conference on Computer Vision Workshops.
expert comments on the developed system. The research was supported Ngo, T.N., Rustia, D.J.A., Yang, E.-C., Lin, T.-T., 2021a. Automated monitoring and
by the Department of Laser Technology, Automation and Production analyses of honey bee pollen foraging behavior using a deep learning-based imaging
Organization at the Faculty of Mechanical Engineering of Wroclaw Uni- system. Comput. Electron. Agric. 187, 106239.
Ngo, T.-N., Rustia, D.J.A., Yang, E.-C., Lin, T.-T., 2021b. Honey bee colony popu-
versity of Science and Technology, Poland under the research subsidy
lation daily loss rate forecasting and an early warning method using temporal
for K62W10D07. convolutional networks. Sensors 21 (11), 3900.
Ngo, T.N., Wu, K.-C., Yang, E.-C., Lin, T.-T., 2019. A real-time imaging system for
References multiple honey bee tracking and activity monitoring. Comput. Electron. Agric. 163,
104841.
Alves, T.S., Pinto, M.A., Ventura, P., Neves, C.J., Biron, D.G., Junior, A.C., Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blon-
de Paula Filho, P.L., Rodrigues, P.J., 2020. Automatic detection and classification of del, M., Prettenhofer, P., Weiss, R., Dubourg, V., et al., 2011. Scikit-learn: Machine
honey bee comb cells using deep learning. Comput. Electron. Agric. 170, 105244. learning in Python. J. Mach. Learn. Res. 12, 2825–2830.
Andrijević, N., Urošević, V., Arsić, B., Herceg, D., Savić, B., 2022. IoT monitoring and Redmon, J., Divvala, S., Girshick, R., Farhadi, A., 2016. You only look once: Unified,
prediction modeling of honeybee activity with alarm. Electronics 11 (5), 783. real-time object detection. In: Proceedings of the IEEE Conference on Computer
Bjerge, K., Frigaard, C.E., Mikkelsen, P.H.g., Nielsen, T.H., Misbih, M., Kryger, P., 2019. Vision and Pattern Recognition. pp. 779–788.
A computer vision system to monitor the infestation level of Varroa destructor in Rodriguez, I.F., Megret, R., Acuna, E., Agosto-Rivera, J.L., Giray, T., 2018. Recognition
a honeybee colony. Comput. Electron. Agric. 164, 104898. of pollen-bearing bees from video using convolutional neural network. In: 2018
Box, G.E., Jenkins, G.M., Reinsel, G.C., Ljung, G.M., 2015. Time Series Analysis: IEEE Winter Conference on Applications of Computer Vision (WACV). IEEE, pp.
Forecasting and Control. John Wiley & Sons. 314–322.
Bozek, K., Hebert, L., Mikheyev, A.S., Stephens, G.J., 2018. Towards dense object Rustia, D.J.A., Lu, C.-Y., Chao, J.-J., Wu, Y.-F., Chung, J.-Y., Hsu, J.-C., Lin, T.-T., 2021.
tracking in a 2D honeybee hive. In: Proceedings of the IEEE Conference on Online semi-supervised learning applied to an automated insect pest monitoring
Computer Vision and Pattern Recognition. pp. 4185–4193. system. Biosyst. Eng. 208, 28–44.
Bozek, K., Hebert, L., Portugal, Y., Mikheyev, A.S., Stephens, G.J., 2021. Markerless Ryu, J.-S., Jung, J.-W., Jeong, C.-H., Choi, B.-J., Lee, M.-l., Kwon, H.W., 2021.
tracking of an entire honey bee colony. Nature Commun. 12 (1), 1–13. Honeybee in-out monitoring system by object recognition and tracking from
Campbell, J., Mummert, L., Sukthankar, R., 2008. Video monitoring of honey bee real-time webcams. J. Apic. 36 (4), 273–280.
colonies at the hive entrance. In: Visual Observation & Analysis of Animal & Insect Semkiw, P., 2020. Sektor pszczelarski w Polsce w 2020 roku [Beekeeping sector in
Behavior, ICPR, Vol. 8. pp. 1–4. Poland in 2020]. Instytut Ogrodnictwa, PułAwy.
Chan., J., Carrión., H., Mégret., R., Rivera., J., Giray., T., 2022. Honeybee Re- Skubida, P., 2007. Zatrucia pszczol, jako czynnik powodujacy istotne straty w
identification in Video: New datasets and impact of self-supervision. In: Proceedings pszczelarstwie [bee poisoning as a factor causing significant losses in beekeeping].
of the 17th International Joint Conference on Computer Vision, Imaging and Pszczelarz Polski (05), 10–12.
Computer Graphics Theory and Applications - Volume 5: VISAPP,. SciTePress, de Souza, V.M., Silva, D.F., Batista, G.E., 2013. Classification of data streams applied
INSTICC, pp. 517–525. http://dx.doi.org/10.5220/0010843100003124. to insect recognition: Initial results. In: 2013 Brazilian Conference on Intelligent
Clarke, D., Robert, D., 2018. Predictive modelling of honey bee foraging activity using Systems. IEEE, pp. 76–81.
local weather conditions. Apidologie 49 (3), 386–396. Stojnić, V., Risojević, V., Pilipović, R., 2018. Detection of pollen bearing honey bees in
Dembski, J., Szymański, J., 2020. Weighted clustering for bees detection on video hive entrance images. In: 2018 17th International Symposium INFOTEH-JAHORINA
images. In: International Conference on Computational Science. Springer, pp. (INFOTEH). IEEE, pp. 1–4.
453–466. Tashakkori, R., Hamza, A.S., Crawford, M.B., 2021. Beemon: An IoT-based beehive
Gomes, P.A., Suhara, Y., Nunes-Silva, P., Costa, L., Arruda, H., Venturieri, G., monitoring system. Comput. Electron. Agric. 190, 106427.
Imperatriz-Fonseca, V.L., Pentland, A., Souza, P.d., Pessin, G., 2020. An amazon Taylor, S.J., Letham, B., 2018. Forecasting at scale. Amer. Statist. 72 (1), 37–45.
stingless bee foraging activity predicted using recurrent artificial neural networks Wada, K., 2018. Labelme: Image polygonal annotation with python. https://github.
and attribute selection. Sci. Rep. 10 (1), 1–12. com/wkentaro/labelme.
He, K., Gkioxari, G., Dollár, P., Girshick, R., 2017. Mask r-cnn. In: Proceedings of the van der Walt, S., Schönberger, J.L., Nunez-Iglesias, J., Boulogne, F., Warner, J.D.,
IEEE International Conference on Computer Vision. pp. 2961–2969. Yager, N., Gouillart, E., Yu, T., the scikit-image contributors, 2014. Scikit-image:
He, K., Zhang, X., Ren, S., Sun, J., 2016. Deep residual learning for image recog- image processing in Python. PeerJ 2, e453. http://dx.doi.org/10.7717/peerj.453,
nition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern URL https://doi.org/10.7717/peerj.453.
Recognition. pp. 770–778. Wu, Y., Kirillov, A., Massa, F., Lo, W.-Y., Girshick, R., 2019. Detectron2. https://github.
com/facebookresearch/detectron2.
Zhang, T.Y., Suen, C.Y., 1984. A fast parallel algorithm for thinning digital patterns.
Commun. ACM 27 (3), 236–239.

10