Scribd
Upload
9 views

Activity. Streaming Sensor Data

The document describes an activity to explore streaming sensor data from a weather station. It involves viewing semi-structured data streaming in real-time, downloading resources, viewing sample weather data and its format, plotting temperature from sample data, viewing live streaming data, and creating a live updating plot of streaming data.

Uploaded by

Santiago Cardenas Jimenez
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
9 views

Activity. Streaming Sensor Data

The document describes an activity to explore streaming sensor data from a weather station. It involves viewing semi-structured data streaming in real-time, downloading resources, viewing sample weather data and its format, plotting temperature from sample data, viewing live streaming data, and creating a live updating plot of streaming data.

Uploaded by

Santiago Cardenas Jimenez
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 6

Activity: Exploring streaming sensor data

Learning Goals:

By the end of this activity, you will be able to:

1. View semi-structured data streaming in real-time from a weather station


2. Create plots of streaming weather station data

Instructions:
Step 1. Open a terminal shell. Open a terminal shell as shown below:

Step 2. Download the resources. Download the resources to any of folders, e.g. the Downloads
folder. Ensure the Python scripts are executable files and have the right privileges so that they can
be used.

Run ls to see the sensor data files and scripts:


Step 3. Rerun a few of the steps from previous lecture.

Step 3a. View weather station data. Twenty-four hours of weather station data is in the file wx-
data.txt . View this data by running more wx-data.txt.

Each line in the file contains a timestamp in the first column and a set of measurements in the
second column. These columns are separated by a tab. The measurements are also a set of columns
separated by commas.

The timestamp is the number of seconds since January 1, 1970. By looking at the successive
timestamps in the file, we can see that the measurements arrive about every second.

The measurements are prefixed with R0, R1, R2 , etc. Each R-value holds a different set of
measurements. For example, R1 has Dn, Dm, Dx , etc. Additionally, the different types are measured
at different frequencies. R1 is measured every second, but R0 and R2 are much less frequent.

Step 3b. Look at key for measurements. We can learn what the measurement fields mean by
looking at the measurements key file. You can do this in a second terminal shell if you want to keep
both at the same time.

Run more wxt-520-format.txt:


This file provides a list of what each field means. For example, Ta is the Air temperature, and Dn is
the Wind direction minimum.

Step 3c. Plot data. Plot the air temperature by running the command:
./plot-data.py wx-data.txt Ta

Notice that this time your python script, instead of saving the plot as a file is opening a plot. That is
thanks to the libraries we installed at the beginning of this activity:

Step 4. View streaming weather station data. Run stream-data.py to see streaming data from
the weather station:
./stream-data.py
Note: Here, we are connecting to Weather Sensors - HPwren - University of California San Diego.
https://hpwren.ucsd.edu/Sensors/. I encourage you to look at the python script to discover a bit
more about how to do this.

The measurements are appearing as they are produced by the weather station. By looking at the
timestamp, we can see that they arrive about every second. Additionally, different measurement
types are produced at different frequencies. For example, R1 is measured every second, but R2 is
less frequent.

Step 5. Create plot of streaming data. We can plot the streaming data by running stream-plot-
data.py:

./stream-plot-data.py Sm

… and continues.

This is the plot generated:


This plot shows the average wind speed (Sm), and it updated every time a new measurement is
produced. Notice that the plot is updated every second. We can create plots for other
measurements by specifying the measurement in the last argument to stream-plot-data.py. For
example, we can plot air temperature (Ta) by running:
./stream-plot-data.py Ta

This plot is updated less frequently than the first plot since air temperature measurements are
produced less frequently.

You might also like