See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/314398756

CANdroid: Freeing ISOBUS Data and Enabling Machine Data Analytics

Conference Paper · July 2016

DOI: 10.13031/aim.20162459827

CITATIONS READS
3 1,510

7 authors, including:

Yang Wang Andrew Balmos

Purdue University Purdue University
10 PUBLICATIONS 21 CITATIONS 19 PUBLICATIONS 61 CITATIONS

SEE PROFILE SEE PROFILE

Alex Layton Samuel Noel

Purdue University Purdue University
16 PUBLICATIONS 52 CITATIONS 7 PUBLICATIONS 21 CITATIONS

SEE PROFILE SEE PROFILE

All content following this page was uploaded by Yang Wang on 21 October 2019.

The user has requested enhancement of the downloaded file.

 An ASABE Meeting Presentation
DOI: 10.13031/aim.20162459827
Paper Number: 162459827
1

CANdroid: Freeing ISOBUS Data and Enabling Machine

Data Analytics

Yang Wang1, Andrew D. Balmos1, Alexander W. Layton1,

Samuel Noel2, James V. Krogmeier1, Dennis R. Buckmaster2, Aaron C. Ault1
1
Electrical and Computer Engineering, Purdue University, 465 Northwestern Ave., West Lafayette, IN
2
Agricultural and Biological Engineering, Purdue University, 225 S. University St., West Lafayette, IN
Written for presentation at the
2016 ASABE Annual International Meeting
Sponsored by ASABE
Orlando, Florida
July 17-20, 2016

ABSTRACT. The past decade has seen a tremendous increase in the number of agricultural machines whose internal data
communications follow the ISO 11783 (ISOBUS) standard. In many cases, the ISOBUS data never leaves the vehicle because
the exporting process is commonly a painstakingly manual task with little reward. However, advancements in digital
agriculture have escalated the value of data and created a demand for an easy and efficient way of acquiring ISOBUS
messages. Recent experiments to collect ISOBUS messages for observing and producing data analytics were made (Marx,
et al., 2015; Pitla, Luck, Werner, Lin, & Shearer, 2016). With the emerging demand for high quality farming data, a
systematic way of collecting, processing and distributing ISOBUS data is needed.

With this goal in mind, this paper presents a reference design using the Kappa Architecture (Kreps, 2014) that is scalable
and highly distributed. The design starts with CANdroid, a recent redevelopment of ISOBlue (Layton, et al., 2014) on a
commercially-available Android platform, which collects the ISOBUS data and uploads them to the Cloud. Within the Cloud,
data is firstly organized into a stream and further processed in the Apache Kafka (Apache Kafka, 2016) database. The Kafka
database then feeds the processed data to a server implemented using Open Ag Data Alliance (OADA) API. Finally, a high-
level web application, TrialsTracker, pulls the data from the OADA API server and visualizes the data with a map which
completes the reference design.

The first part of the paper gives an overview of ISOBUS and states the objectives. The second and third part of the paper
present the making of CANdroid as well as how the Apache Kafka database and the OADA API server works. An example
usage of TrialsTracker is also explained in the third part of the paper.

Keywords. Cloud, cloud computing, data, data collection, digital agriculture, ISOBUS, machinery, mobile applications,
network, precision agriculture, technology.

The authors are solely responsible for the content of this meeting presentation. The presentation does not necessarily reflect the official position of the
American Society of Agricultural and Biological Engineers (ASABE), and its printing and distribution does not constitute an endorsement of views which
may be expressed. Meeting presentations are not subject to the formal peer review process by ASABE editorial committees; therefore, they are not to be
presented as refereed publications. Citation of this work should state that it is from an ASABE meeting paper. EXAMPLE: Author’s Last Name, Initials.
2016. Title of presentation. ASABE Paper No. ---. St. Joseph, MI.: ASABE. For information about securing permission to reprint or reproduce a meeting
presentation, please contact ASABE at http://www.asabe.org/copyright (2950 Niles Road, St. Joseph, MI 49085-9659 USA).1
ASABE Annual International Meeting 2
 Introduction
Modern agricultural machinery generates a significant amount of data using a standardized communication protocol
called ISOBUS (ISO, 2007). The data are usually captured and stored within the internal storage of monitors. These monitors
then process the collected data and visualize them to assist farmers in various aspects of planting and harvesting.
Furthermore, by exporting the collected data, predictive analytics can be used to help farmers make smarter off-field
decisions. Yet the exporting process is not straightforward and complicated. As a consequence, a large set of data still resides
within the machinery without being fully utilized. The goal of this paper is to explore a design that offers a method for the
farmers, developers and researchers to collect, process and distribute the ISOBUS data more efficiently.
The following sections briefly discusses what ISOBUS is and the objectives of this paper.

ISOBUS (ISO 11783)

ISOBUS is a communication protocol for most modern agricultural machinery based on the SAE J1939 standard (ISO,
2007). An ISOBUS network consists of two separate Controller Area Network (CAN) busses. One is known as the tractor
bus and the other is the implement bus. The bitrate on each bus is 250 kbps. During normal operation of agricultural
machinery, ISOBUS messages flow on the ISOBUS network. Figure 1 shows the format of a standard ISOBUS message.
The data of one message is defined by a Parameter Group Number (PGN). The interpretation of ISOBUS data depends on
the accessibility of the specification for the particular Parameter Group. There are a certain number of PGNs and their data
specifications that are published; however, there are also a large set of proprietary PGNs which are not publicly known. With
the accessibility for published known PGNs and the corresponding specifications, one could interpret the data, perform data
analysis and use analytics to apply context to the in-field operations. These data often include vehicle speed, geographical
locations, fuel rate, etc.

Figure 1. Format of a ISOBUS Message Sent Over ISOBUS Network

Objectives
The first objective is to create a hardware platform for data collection and distribution to the Cloud using open-source
software. The second objective is to propose a reference design that is efficient, scalable and capable of handling the data
streams in the Cloud. Finally, the last objective is to create a web application that visualizes the collected data.

CANdroid
CANdroid involves both hardware and software components. The main idea of CANdroid is to transform a commercially-
available Android tablet into an ISOBUS message collection platform that incorporates Cloud connectivity using an open-
source app along with custom-developed software libraries. In the following sections, both hardware and software aspects
of CANdroid are discussed.

CANdroid Device
The CANdroid device is the hardware component of the system. It is based on the Nexus 9 tablet manufactured by HTC.
The tablet runs Android, an open-source Linux distribution targeting mobile devices, as its operating system. Additionally,
the CANdroid device uses two USB2CAN adapters which provide two CAN interfaces for communicating with both tractor
and implement bus on the ISOBUS data network. The USB2CAN adapters are compatible with SocketCAN, the de facto
standard CAN driver for Linux. The Nexus 9 is connected to a USB On-The-Go (OTG) Y-cable as a USB host. The
connection is extended by a USB hub, to which the two USB2CAN adapters are connected as USB slaves. In this way, the
Nexus 9 is able to supply power to both USB2CAN adapters and receive ISOBUS data from the adapters. Meanwhile, the
Nexus 9 can be charged by an external power source via the power input of the USB OTG Y-cable.
All the hardware parts are placed into a rigid enclosure. In a real-world setting, if the hardware parts are exposed in the
cabins of tractors or combines, they are more likely to get damaged. Hence, using a rigid enclosure gives the CANdroid
device shown in Figure 3, which offers more durability. Figure 4 shows that the Nexus 9 is secured onto the top panel of the
enclosure such that it can endure the vibrations during normal operations of tractors or combines. In Figure 5, different
hardware parts within the enclosure are annotated. Moreover, in Figure 6, a diagram that shows how all the hardware parts
are connected is shown. Finally, the list of parts that are needed to build a CANdroid device is given in Table 1 along with
their vendor and price information.

ASABE Annual International Meeting Page 3

 Figure 3. CANdroid device enclosure top view.

Figure 4. CANdroid device enclosure’s top panel with the Nexus 9 fastened onto the panel

ASABE Annual International Meeting Page 4

 ISOBUS Diagnostic Port Adapter USB2CAN Adapters

USB Hub

USB On-The-Go Y-Cable OTG Cable to Nexus 9 Charging Port for External Power
Figure 5. Inside of CANdroid device enclosure. Different components are annotated on the figure.

Figure 6. Hardware connection diagram for CANdroid device.

Item Name Vendor Price (as of June 2016)

Nexus 9 Tablet HTC $599.00
USB2CAN Adapter 8devices $65.00
USB OTG Y-cable Amazon $4.59
4-Port USB Hub Amazon $6.26
Enclosure Polycase $40.31

Table 1. Necessary parts for building a CANdroid device with their vendor and price listed.

Software
Android Kernel Modification
The preinstalled Linux kernel on the Nexus 9 offers neither support for the SAE J1939 protocol nor the driver for the
USB2CAN adapters. As a result, modifications to the preinstalled kernel are required. For enabling support for the SAE
J1939 protocol, a J1939 kernel module (Dijck, 2016) was created and integrated into SocketCAN. The J1939 kernel module
and the USB2CAN adapter driver were added to the kernel to build a customized Android kernel image that was flashed
onto the Nexus 9.
CANdroid-lib Java Native Interface (JNI) Library
We developed an open-source software library to expose the low-level J1939 kernel module to a high level Android app.
Conventionally, Android app development libraries does not offer support to handle the communications between an
Android app and low-level Linux kernel modules. Therefore, we implemented the CANdroid-lib, a Java Native Interface
library (Wang, Balmos, & Layton, 2016) to bridge between the CANdroid Logger app and the J1939 kernel module. It
allows the app to directly access and manipulate the CAN interfaces using the SAE J1939 protocol to receive ISOBUS
messages. Furthermore, this enables developers and researchers to utilize the CAN features on Android platforms more
efficiently and intuitively. Figure 7 shows that CANdroid-lib interfaces bidirectionally with low-level J1939 kernel module
and the CANdroid Logger app to provide data communications.

ASABE Annual International Meeting Page 5

 Figure 7. CANdroid-lib, as a bridge, handles communications between low-level J1939 kernel module and CANdroid Logger App. With
CANdroid-lib, a high-level Android app can communicate with low-level kernel modules without kernel programming.

CANdroid Logger Android App

We developed the CANdroid Logger app to initiate and manage ISOBUS message collection (Wang, Balmos, & Layton,
2016). The app receives ISOBUS messages using the CANdroid-lib, decodes the ISOBUS message and displays the useful
ISOBUS data in the app. Meanwhile, the app launches two background services. One service stores the decoded ISOBUS
data on the tablet’s internal storage and the other service uploads the data to the Cloud. Moreover, the app allows the user
to filter the ISOBUS messages based on Parameter Group Numbers (PGNs), source addresses and destination addresses.
Figure 6 shows a screenshot of CANdroid Logger App collecting ISOBUS messages.

Figure 7: Screenshot of CANdroid Logger App data collection. Users can see the data stream as they collect ISOBUS message in the field.

Building a CANdroid
The process of building CANdroid is covered in this documentation (Wang, 2015). It is a quick tutorial to guide
developers and researchers through the necessary steps to transform a regular Nexus 9 into a CANdroid with the modified
kernel image and the CANdroid Logger app.
ASABE Annual International Meeting Page 6
 Kappa Architecture
The past decade’s technological innovations in agricultural machinery have accelerated the rate at which machine sensor
data is generated. This rapidly expanding data set is pushing the traditional data processing infrastructures to their limit
(Stonebraker, Cetintemel, & Zdonik, 2005). Therefore, in order to address the challenge of processing high-volume machine
data, discovering an efficient and scalable real-time processing scheme that is suitable for agricultural big data becomes
critical. The Kappa Architecture is one of the stream processing architectures that is designed to handle, process and
distribute large data sets in real-time with little overhead.
Figure 8 is a high-level view of the Kappa Architecture for CANdroid’s use case. Specifically, CANdroid establishes the
connection to the Cloud by uploading the collected ISOBUS messages to the Kafka Database. The ISOBUS messages are
processed and then forwarded to an OADA API server for storage. Finally, the high-level applications would retrieve the
necessary data for either post-processing or data visualization.

Figure 8. CANdroid’s integration within the Kappa architecture which initiates the continuous ISOBUS data flow. The data are processed and
stored within the Cloud and then visualized using high-level applications.

The corresponding components within the architecture, particularly, Apache Kafka and an Open Ag Data Alliance
(OADA) API server, are discussed in detail in the subsequent sections.

Apache Kafka
Motivation
There are a couple of requirements for an efficient real-time processing scheme. First of all, a real-time processing scheme
should allow a continuous data flow. Secondly, the live services within the scheme should response instantaneously to
facilitate real-time processing of the data. Further, real-time data are not simply used for analysis and reporting; they are
also a dependency within the real-time data pipeline (Goodhope, et al., 2012). Most importantly, the scheme should be
scalable and capable of handling large data sets. To illustrate the last point, some simple calculations on the data set of
interest are appropriate. Equation 1 calculates the theoretical message rate per bus over the ISOBUS network:
b
r= (1)
m
where,
 messages 
r = message rate  ,
 second 
 bits 
b = ISOBUS bitrate  ,
 second 
 bits 
m = message size  .
 message 
Using Equation (1), assuming a constant message size of 150 bits and a bitrate of 250 kbps, the total message rate
generated by both tractor and implement bus is approximately 3334 messages per second. This message rate increases
ASABE Annual International Meeting Page 7
quickly as the number of tractors or combines increases and eventually becomes a concern for real-time data processing in
terms of its data size.
Kafka is the one of schemes that satisfies the aforementioned requirements. It originated from LinkedIn and it is still used
within LinkedIn as part of the data infrastructure for managing real-time data streams (Auradkar, et al., 2012). The next two
subsections focus on how Kafka works and its specific use case in CANdroid.
Kafka Database
In Kafka terminology, a stream of messages is defined by a topic. A producer publishes messages to a topic to which
consumers can subscribe and consume the published message. Brokers store the published messages and manage delivering
data to the consumers upon request. In addition, brokers are horizontally scalable. This means that if a producer sends out
an enormous amount of data to cause overloads on the any of the brokers, one could simply add another broker to the
database to reduce the workload. Figure 9 illustrates the Kafka workflow from a high level.

Figure 9. An illustration of a basic Kafka database. Producers publish data streams to brokers. Brokers manages the data streams and deliver
them upon consumers’ requests.

Topics use partitions which stores messages. The partitions sequence the messages by the order in which they are
received. The database architect can add more partitions to one topic to reduce the workload. Every time a producer sends
out a new message, the broker appends the message to the last segment of the partition. A serial ordering of messages within
one partition conforms to the idea of a data stream which allows efficient and parallelized stream processing (Auradkar, et
al., 2012). Moreover, another benefit that Kafka brings is that the messages contained in each broker get automatically
deleted after a configurable period of time. Intuitively, in CANdroid’s use case, it makes sense since the needed data have
already been consumed in real-time. Therefore, the rest of the data could still be retained within the broker temporarily for
a period but not permanently. Figure 10 presents how the data stream is structured within a Kafka broker.

Figure 10. Details within a Kafka broker. Each data stream has a topic and partitions in which data are appended to.

Within this paper’s scope, a microservice is described as a live service that act as both a producer and a consumer. As a
consumer, it extracts valuable information from the Kafka data stream; meanwhile, as a producer, it sends the extracted data
back onto the data stream on a different topic. In Figure 11, Microservice A could be stacked and distributed by different
third-parties who are interested in extracting information out of Topic A. The same principle applies to Microservice B. This
vertically stackable working scheme of microservices enables parallel stream processing which offers more efficiencies and
less overhead for accessing the data streams.

ASABE Annual International Meeting Page 8

 Figure 11. Vertically stackable microservices parse out data from different topics. The microservices can be distributed in different Kafka
database. As long as they are pointing to the right topic, they will receive the requested message correctly.

Kafka in the Kappa Architecture

We used a Kafka database for this paper in order to help us to manage and process the ISOBUS message streams more
efficiently. In the particular example shown in Figure 12, CANdroid, as an external producer in Kafka, collects the data and
uploads them onto a Kafka topic named Raw ISOBUS Message. This data stream contains all unprocessed ISOBUS
messages coming out of the farming machinery.

Figure 12. Communications between CANdroid and Kafka Database in the Kappa Architecture.

There are three microservices connected to the Kafka database. They work as consumers and producers simultaneously.
The data the microservices are interested in are GPS coordinates and fuel rate. Hence, the GPS parser microservice listens
exclusively for the corresponding PGN that contains GPS coordinates data on the Raw ISOBUS Message topic. After the
microservice observes the known PGN, it utilizes the corresponding ISOBUS data to parse out GPS coordinates and stream
them back onto the GPS Message topic. The same mechanism applies for the Fuel Rate Parser microservice which creates
the Fuel Rate Message topic and sends the parsed fuel rate data onto this topic. Meanwhile, as soon as data starts flowing
on both GPS Message topic and Fuel Rate Message topic, the Fuel Rate Map Creator microservice takes these two streams,
builds data objects that correlate GPS coordinates with fuel rate data using timestamps, and puts them onto a new topic
called Fuel Rate Map Message.

Open Ag Data Alliance (OADA)

Motivation
In the modern agricultural community, farmers are using hardware systems to generate data that would support them for
in better decision-making. Software solutions are used to interpret the data and turn the data into formats like images, charts,
and reports. However, the current solutions lack portability of data. As a result, farmers fall into the dilemma of having
operational data on hand but having difficulties in sharing data across platforms provided by different manufacturers.
Agricultural operational data should flow freely without the intervention of farmers or manufacturers. This would enable
the farmers to focus more on the decision-making instead of worrying about the handling and distribution of data. Hence,
ASABE Annual International Meeting Page 9
Open Ag Data Alliance (OADA) and its specifications were created with mindset of catalyzing the development of open
source data sharing APIs and software library to improve the transferability of farmers’ data among solution providers
(OADA, 2014).
OADA Application Program Interface (API)
The OADA API is a REpresentative State Transfer (REST) API with its specification emphasizing the agricultural data
transferability between different platforms. In the OADA API’s specification, every data endpoint is a resource
(/resources/<resourceId>). The resource could be in any format although OADA has a preference of using JSON-
based formats. A JSON resource can use its identifier (_id) property to link to another resource. Every time a resource
changes, its revision (_rev) property’s value increments. A bookmark is a type of resource that merely links to other
resources of interest. Users and developers could create their own bookmarks to describe their data. Moreover, OADA adopts
the OAuth 2.0 standard (Hardt, 2012) as the authorization mechanism for data exchange to ensure the security and privacy
of farmer’s data (OADA, 2014).
OADA in the Kappa Architecture
We have developed a server that implements the OADA API. It has two main use cases described in the OADA
documentation (OADA, 2014). One use case of the OADA API is enabling the synchronization of data to a OADA-
compliant cloud. The Kappa Architecture enables this use case by using Kafka database to send the processed ISOBUS data
to the OADA API server. Another use case is that the server could share the stored data with higher level applications. An
example of a high level application pulling data from the OADA API server is given in the next section.

Application Using the Kappa Architecture

Figure 13. The complete Kappa architecture which integrates TrialsTracker for data visualization.

TrialsTracker
TrialsTracker is a web application that demonstrates the usefulness of the processed data and to complete the Kappa
architecture (Noel, 2016). Figure 13 illustrates an example usage of the application. TrialsTracker retrieves the processed
data from the OADA API server and it uses these data to render a fuel rate map. The goal of TrialsTracker is to create a real-
time interactive agricultural data processing platform that visualizes the aggregate data. At the time this paper was written,
the application was still under development.

Conclusion
With a plethora amount of data being generated by modern agricultural machinery, the digital agriculture community
needs a systematic method of performing quality analytics on this big data. In this work, we proposed a reference design
that offers an efficient solution to both free the ISOBUS data and gives real-time stream processing capability for a large set
of data in the Cloud.
We showed that CANdroid, which runs open-source software on a commercially available hardware platform, offers an
Android app that enables users to collect ISOBUS data easily. In addition, we also presented the Kappa architecture that
integrates CANdroid to establish the connection to the Cloud to upload the ISOBUS data stream. Furthermore, we illustrated
how the Kafka database and an OADA API server work in processing and storing the data stream. Finally, an example web
application, TrialsTracker, which continues the data flow and visualizes the processed data, is also demonstrated. With the
usage of this architecture gives the manufacturers, developers and researchers a valuable method for accessing the data in
real time. Useful agricultural insights could be further discovered.
ASABE Annual International Meeting Page 10
 For future development, a series of CANdroids will be made and deployed in tractor and combines during the harvesting
and planting seasons to collect more ISOBUS data. This would help us to test and verify the effectiveness of the Kappa
architecture for agricultural data acquisition and processing.

Acknowledgements
This project has been supported by Spensa Technologies, Inc., Enterprise Group Ltd., and Oxbow Agriculture LLC.

References
Apache Kafka. (2016). Apache Kafka, a high-throughput distributed message system. Retrieved from Apache Kafka:
http://kafka.apache.org/
Auradkar, A., Botev, C., Das, S., De Maagd, D., Feinberg, A., Ganti, P., . . . Zhang, J. (2012). Data Infrastructure at LinkedIn.
2012 IEEE 28th International Conference on Data Engineering, (pp. 1370-1381).
Dijck, K. V. (2016). Linux Kernel Source Tree. Retrieved from Github: https://github.com/kurt-vd/linux
Goodhope, K., Koshy, J., Kreps, J., Narkhede, N., Park, R., Rao, J., & Ye, V. (2012). Building LinkedIn's Real-time Activity
Data Pipeline. IEEE Data Eng. Bull., 35(2).
Hardt, D. (2012, October). The OAuth 2.0 Authorization Framework. Retrieved from https://tools.ietf.org/html/rfc6749
ISO. (2007). Tractors and machinery for agriculture and forestry Serial control and communications data network - Part 1:
General standard for mobile data communication. 11783-1.
Kreps, J. (2014, July 2). Questioning the Lambda Architecture. Retrieved from O'Reilly:
https://www.oreilly.com/ideas/questioning-the-lambda-architecture
Layton, A., Balmos, A., Sabpisal, S., Ault, A., Krogmeier, J., & Buckmaster, D. (2014). ISOBlue: An Open Source Project
to Bring Agricultural Machinery Data into the Cloud. 1.
Marx, S., Luck, J., Hoy, R., Pitla, S., Blankenship, E., & Darr, M. (2015). Validation of machine {CAN} bus {J1939} fuel
rate accuracy using Nebraska Tractor Test Laboratory fuel rate data . Computers and Electronics in Agriculture ,
118, 179 - 185.
Noel, S. (2016). TrialsTracker-Cerebral. Retrieved from https://github.com/OpenATK/TrialsTracker-Cerebral
OADA. (2014). OADA Use Cases. Retrieved from Github: https://github.com/OADA/oada-docs/blob/master/functional-
specs/Use-Cases.md
OADA. (2014). Principals & Use Cases. Retrieved from Open Ag Data Alliance: openag.io/about-us/principals-use-cases/
OADA. (2014). Technical Summary & Road Map. Retrieved from Open Ag Data Alliance: openag.io/about-us/technical-
summary-road-map/
Pitla, S., Luck, J., Werner, J., Lin, N., & Shearer, S. (2016). In-field fuel use and load states of agricultural field machinery
. Computers and Electronics in Agriculture , 121, 290 - 300.
Stonebraker, M., Cetintemel, U., & Zdonik, S. (2005, dec). The 8 Requirements of Real-time Stream Processing. SIGMOD
Rec., 34(4), 42--47.
Wang, Y. (2015). Turn a Nexus 9 into CANdroid. Retrieved from Github: https://github.com/OATS-Group/CANdroid
Wang, Y., Balmos, A., & Layton, A. (2016, June 3). CANdroid Logger App. West Lafayette, IN, United States: Zenodo.
doi:10.5281/zenodo.54760
Wang, Y., Balmos, A., & Layton, A. (2016, June 2). CANdroid-lib. West Lafayette, IN, United States: Zenodo.
doi:10.5281/zenodo.54627

ASABE Annual International Meeting Page 11

View publication stats