7/25/24, 4:16 PM ex13-Using SQLite as a Time Series Database.

ipynb - Colab

keyboard_arrow_down ex13-Using SQLite as a Time Series Database

SQLite supports five date and time functions as follows:

1. date(timestring, modifier, modifier, ...)

2. time(timestring, modifier, modifier, ...)
3. datetime(timestring, modifier, modifier, ...)
4. julianday(timestring, modifier, modifier, ...)
5. strftime(format, timestring, modifier, modifier, ...)

All five date and time functions take a time string as an argument. The time string is followed by zero or more modifiers. The strftime() function
also takes a format string as its first argument. With these date and time functions, we can save time series data into a SQLite database so as
to use SQLite as a time series database.

In this notebook, we will create a table to save man-made temperature data at different dates and times. You can presume that they are
collected from some kind of temperature sensors.

This notebook will present:

Practice date and time functions

Create a temperature table and index
Insert data into the temperature table
Query

%load_ext sql

from random import uniform, choice

import time
from datetime import datetime, timedelta
import pandas as pd
%matplotlib inline

keyboard_arrow_down 1. Connect to database

It was mentioned before the demo.db3 is extracted from a hydrological modelling. As a result, the data in each table is tidy and complete
without NULL values. However, we can create a table with NULL values for demo.

%sql sqlite:///data/demo.db3

u'Connected: @data/demo.db3'

keyboard_arrow_down 2. Play with date and time functions

You can find more example from lang_datefunc or sqlite_date_time

Compute the current date.

%sql SELECT date('now');

* sqlite:///data/demo.db3
Done.
date('now')
2018 10 17

Compute the last day of the current month.

%sql SELECT date('now','start of month','+1 month','-1 day');

* sqlite:///data/demo.db3
Done.
date('now','start of month','+1 month','-1 day')
2018 10 31

https://colab.research.google.com/drive/1pdjrZp1qleIX5M2RghFJavNxa-Xoasmx#printMode=true 1/6
7/25/24, 4:16 PM ex13-Using SQLite as a Time Series Database.ipynb - Colab

Compute the date of the first Tuesday in October for the current year.

%sql SELECT date('now','start of year','+9 months','weekday 2');

* sqlite:///data/demo.db3
Done.
date('now','start of year','+9 months','weekday 2')
2018 10 02

keyboard_arrow_down 3. Create a table of time series temperature

3.1 Create table
only with two columns of timestamp and Temperature

%%sql sqlite://
DROP TABLE IF EXISTS Temperature;
CREATE TABLE Temperature (Timestamp DATETIME NOT NULL, Temperature NUMERIC NOT NULL)

Done.
Done.
[]

keyboard_arrow_down 3.2 Create an index

For efficient querying, we'll want to index the timestamp column.

%%sql sqlite://
CREATE UNIQUE INDEX idx_timestamp ON Temperature (Timestamp);

Done.
[]

keyboard_arrow_down 3.3 Insert data

Make 500 rows of data and temperature ranges between 18 to 26.

def dt(days):
return timedelta(days=days)

N_rows = 500
now = datetime.now()

for i in range(N_rows):
timestamp = now - dt(days=(N_rows - i))
temperature = uniform(18, 26)
%sql INSERT INTO Temperature VALUES (:timestamp, :temperature);

* sqlite:///data/demo.db3
1 rows affected.
* sqlite:///data/demo.db3
1 rows affected.
* sqlite:///data/demo.db3
1 rows affected.
* sqlite:///data/demo.db3
1 rows affected.
* sqlite:///data/demo.db3
1 rows affected.
* sqlite:///data/demo.db3
1 rows affected.
* sqlite:///data/demo.db3
1 rows affected.
* sqlite:///data/demo.db3
1 rows affected.
* sqlite:///data/demo.db3
1 rows affected.
* sqlite:///data/demo.db3
1 rows affected.
* sqlite:///data/demo.db3
1 rows affected.
* sqlite:///data/demo.db3

https://colab.research.google.com/drive/1pdjrZp1qleIX5M2RghFJavNxa-Xoasmx#printMode=true 2/6
7/25/24, 4:16 PM ex13-Using SQLite as a Time Series Database.ipynb - Colab
1 rows affected.
* sqlite:///data/demo.db3
1 rows affected.
* sqlite:///data/demo.db3
1 rows affected.
* sqlite:///data/demo.db3
1 rows affected.
* sqlite:///data/demo.db3
1 rows affected.
* sqlite:///data/demo.db3
1 rows affected.
* sqlite:///data/demo.db3
1 rows affected.
* sqlite:///data/demo.db3
1 rows affected.
* sqlite:///data/demo.db3
1 rows affected.
* sqlite:///data/demo.db3
1 rows affected.
* sqlite:///data/demo.db3
1 rows affected.
* sqlite:///data/demo.db3
1 rows affected.
* sqlite:///data/demo.db3
1 rows affected.
* sqlite:///data/demo.db3
1 rows affected.
* sqlite:///data/demo.db3
1 rows affected.
* sqlite:///data/demo.db3
1 rows affected.
* sqlite:///data/demo.db3
1 rows affected.
* sqlite:///data/demo.db3
1 ff t d

keyboard_arrow_down 4. Play with time series table

keyboard_arrow_down 4.1 Simple Query

%sql SELECT datetime(Timestamp) as Date, Temperature FROM Temperature LIMIT 5

* sqlite:///data/demo.db3
Done.
Date Temperature
2017-06-04 14:57:28 19.1891167256
2017-06-05 14:57:28 25.5105711309
2017-06-06 14:57:28 24.4625398612
2017-06-07 14:57:28 20.8742018644
2017 06 08 14:57:28 23 9660706665

keyboard_arrow_down 4.2 Filter with a datetime

%%sql sqlite://
SELECT Timestamp as Date, Temperature
FROM Temperature
WHERE Date <= '2017-12-31 14:21:45'

https://colab.research.google.com/drive/1pdjrZp1qleIX5M2RghFJavNxa-Xoasmx#printMode=true 3/6
7/25/24, 4:16 PM ex13-Using SQLite as a Time Series Database.ipynb - Colab

Done.
Date Temperature
2017-06-04 14:57:28.059000 19.1891167256
2017-06-05 14:57:28.059000 25.5105711309
2017-06-06 14:57:28.059000 24.4625398612
2017-06-07 14:57:28.059000 20.8742018644
2017-06-08 14:57:28.059000 23.9660706665
2017-06-09 14:57:28.059000 19.5525914543
2017-06-10 14:57:28.059000 21.909136455
2017-06-11 14:57:28.059000 25.5311195651
2017-06-12 14:57:28.059000 21.0259723504
2017-06-13 14:57:28.059000 21.0310871716
2017-06-14 14:57:28.059000 20.0734349482
2017-06-15 14:57:28.059000 22.5695958825
2017-06-16 14:57:28.059000 23.1701603287
2017-06-17 14:57:28.059000 21.4837702278
2017-06-18 14:57:28.059000 22.0611682235
2017-06-19 14:57:28.059000 18.7065624316
2017-06-20 14:57:28.059000 23.290111464
2017-06-21 14:57:28.059000 18.0716912304
2017-06-22 14:57:28.059000 20.4378580348
2017-06-23 14:57:28.059000 24.9952883007
2017-06-24 14:57:28.059000 21.5374302583
2017-06-25 14:57:28.059000 21.0335597012
2017-06-26 14:57:28.059000 25.4478648134
2017-06-27 14:57:28.059000 21.532404188
2017-06-28 14:57:28.059000 25.9751954682
2017-06-29 14:57:28.059000 18.4765827648
2017-06-30 14:57:28.059000 23.2185925605
2017-07-01 14:57:28.059000 23.2625825083
2017-07-02 14:57:28.059000 21.0665494967
2017-07-03 14:57:28.059000 25.1787469085
2017-07-04 14:57:28.059000 25.3199114911
2017-07-05 14:57:28.059000 21.0752862379
2017-07-06 14:57:28.059000 24.3126804784
2017-07-07 14:57:28.059000 25.4159813891
2017-07-08 14:57:28.059000 24.4921743528
2017-07-09 14:57:28.059000 24.1700562885
2017-07-10 14:57:28.059000 25.7825009814
2017-07-11 14:57:28.059000 21.4144957206
2017-07-12 14:57:28.059000 19.1243812264
2017-07-13 14:57:28.059000 23.9167777677
2017-07-14 14:57:28.059000 20.2607835061
2017-07-15 14:57:28.059000 22.535627006
2017-07-16 14:57:28.059000 18.7696498564
2017-07-17 14:57:28.059000 19.6575435997
2017-07-18 14:57:28.059000 19.2728202698
2017-07-19 14:57:28.059000 25.8709032979
2017-07-20 14:57:28.059000 22.0936191208
2017-07-21 14:57:28.059000 20.6130090965
2017-07-22 14:57:28.059000 22.6370451102
2017-07-23 14:57:28.059000 24.8307240557
2017-07-24 14:57:28.059000 19.8728991347
2017-07-25 14:57:28.059000 19.8825284148
2017-07-26 14:57:28.059000 21.2829937779
2017-07-27 14:57:28.059000 20.4233614967
2017-07-28 14:57:28.059000 25.4607138435
2017-07-29 14:57:28.059000 18.1611994106
2017-07-30 14:57:28.059000 22.5078523766
2017-07-31 14:57:28.059000 20.4616804919
2017-08-01 14:57:28.059000 18.7342706644
2017-08-02 14:57:28.059000 18.6565428062
2017-08-03 14:57:28.059000 22.0955717864
2017-08-04 14:57:28.059000 19.275097344
2017-08-05 14:57:28.059000 24.5248062785
2017-08-06 14:57:28.059000 18.3334314964
2017-08-07 14:57:28.059000 19.3999459951
2017-08-08 14:57:28.059000 22.7849960404
2017-08-09 14:57:28.059000 25.8032766629
https://colab.research.google.com/drive/1pdjrZp1qleIX5M2RghFJavNxa-Xoasmx#printMode=true 4/6
7/25/24, 4:16 PM ex13-Using SQLite as a Time Series Database.ipynb - Colab
2017 08 09 14:57:28.059000 25.8032766629
2017-08-10 14:57:28.059000 24.5434584639
2017-08-11 14:57:28.059000 19.0678702211
2017-08-12 14:57:28.059000 18.8053176171
2017-08-13 14:57:28.059000 22.4138688197
2017-08-14 14:57:28.059000 18.362485749
2017-08-15 14:57:28.059000 19.929203462
2017-08-16 14:57:28.059000 24.1160665514
2017-08-17 14:57:28.059000 25.6875651592
2017-08-18 14:57:28.059000 24.9316823463
2017-08-19 14:57:28.059000 19.0290547931
2017-08-20 14:57:28.059000 21.222419192
2017-08-21 14:57:28.059000 24.7206527209
2017-08-22 14:57:28.059000 19.1204225001
2017-08-23 14:57:28.059000 20.5971865274
2017-08-24 14:57:28.059000 24.4396307407
2017-08-25 14:57:28.059000 19.7076305182
2017-08-26 14:57:28.059000 19.2780871155
2017-08-27 14:57:28.059000 23.5486706154
2017-08-28 14:57:28.059000 25.1555876245
2017-08-29 14:57:28.059000 22.6844688563
2017-08-30 14:57:28.059000 23.5723750082
2017-08-31 14:57:28.059000 23.6509745867
2017-09-01 14:57:28.059000 22.7079065256
2017-09-02 14:57:28.059000 23.40618275
2017-09-03 14:57:28.059000 19.1441669041
2017-09-04 14:57:28.059000 25.7840721207
2017-09-05 14:57:28.059000 25.7160356837
2017-09-06 14:57:28.059000 18.5869010986
2017-09-07 14:57:28.059000 24.2057918072
2017-09-08 14:57:28.059000 23.1757194933
2017-09-09 14:57:28.059000 23.6473188712
2017-09-10 14:57:28.059000 22.1054936045
2017-09-11 14:57:28.059000 23.1988361634
2017-09-12 14:57:28.059000 24.4425636851
2017-09-13 14:57:28.059000 21.8916490769
2017-09-14 14:57:28.059000 21.7506861499
2017-09-15 14:57:28.059000 19.3485348947
2017-09-16 14:57:28.059000 21.2745895496
2017-09-17 14:57:28.059000 25.729723131
2017-09-18 14:57:28.059000 22.5500182379
2017-09-19 14:57:28.059000 20.0285678799
2017-09-20 14:57:28.059000 23.2303339995
2017-09-21 14:57:28.059000 20.6036054117
2017-09-22 14:57:28.059000 19.8063854831
2017-09-23 14:57:28.059000 18.0207211838
2017-09-24 14:57:28.059000 24.6406205386
2017-09-25 14:57:28.059000 24.1227972298
2017-09-26 14:57:28.059000 23.235534493
2017-09-27 14:57:28.059000 24.3523956322
2017-09-28 14:57:28.059000 20.3271574804
2017-09-29 14:57:28.059000 18.8878371057
2017-09-30 14:57:28.059000 22.3018848157
2017-10-01 14:57:28.059000 19.2316601922
2017-10-02 14:57:28.059000 25.2239550421
2017-10-03 14:57:28.059000 22.83957333
2017-10-04 14:57:28.059000 18.1896630711
2017-10-05 14:57:28.059000 22.6740289211
2017-10-06 14:57:28.059000 21.0219600173
2017-10-07 14:57:28.059000 18.0502991719
2017-10-08 14:57:28.059000 20.0530207009
2017-10-09 14:57:28.059000 24.9297395365
2017-10-10 14:57:28.059000 23.0467613044
2017-10-11 14:57:28.059000 25.797040768
2017-10-12 14:57:28.059000 22.0719339958
2017-10-13 14:57:28.059000 23.8479346264
2017-10-14 14:57:28.059000 24.705667306
2017-10-15 14:57:28.059000 18.9247036468
2017-10-16 14:57:28.059000 24.3797510557
2017-10-17 14:57:28 059000 19 6469260216
https://colab.research.google.com/drive/1pdjrZp1qleIX5M2RghFJavNxa-Xoasmx#printMode=true 5/6
7/25/24, 4:16 PM ex13-Using SQLite as a Time Series Database.ipynb - Colab
2017 10 17 14:57:28.059000 19.6469260216
2017-10-18 14:57:28.059000 23.3529747643
2017-10-19 14:57:28.059000 22.3179591842
2017-10-20 14:57:28.059000 23.2721882317
2017-10-21 14:57:28.059000 23.6366902524
2017-10-22 14:57:28.059000 24.8680759855
2017-10-23 14:57:28.059000 18.5931959314
2017-10-24 14:57:28.059000 25.2317183806
2017-10-25 14:57:28.059000 25.5396208592
2017-10-26 14:57:28.059000 18.3904546322
2017-10-27 14:57:28.059000 22.9469888277
2017-10-28 14:57:28.059000 24.0932260057
2017-10-29 14:57:28.059000 22.8969895882
2017-10-30 14:57:28.059000 23.4297454526
2017-10-31 14:57:28.059000 18.5515978394
2017-11-01 14:57:28.059000 18.8085419607
2017-11-02 14:57:28.059000 20.7322003743
2017-11-03 14:57:28.059000 18.9359162381
2017-11-04 14:57:28.059000 23.7882876334
2017-11-05 14:57:28.059000 24.4663362677
2017-11-06 14:57:28.059000 21.9076789936
2017-11-07 14:57:28.059000 22.2908480291
2017-11-08 14:57:28.059000 22.3925054081
2017-11-09 14:57:28.059000 23.6957571049
2017-11-10 14:57:28.059000 20.5878794002
2017-11-11 14:57:28.059000 19.4218570986
2017-11-12 14:57:28.059000 22.2167752562
2017-11-13 14:57:28.059000 24.289132669
2017-11-14 14:57:28.059000 19.4812750982
2017-11-15 14:57:28.059000 22.8324951652
2017-11-16 14:57:28.059000 21.6552058362
2017-11-17 14:57:28.059000 20.8928992366
2017-11-18 14:57:28.059000 25.3698706687
2017-11-19 14:57:28.059000 23.6515437379
2017-11-20 14:57:28.059000 19.3316610161
2017-11-21 14:57:28.059000 19.7546385923
2017-11-22 14:57:28.059000 23.6782027975
2017-11-23 14:57:28.059000 25.2359002827
2017-11-24 14:57:28.059000 18.1146948305
2017-11-25 14:57:28.059000 25.9160822869
2017-11-26 14:57:28.059000 18.8136996314
2017-11-27 14:57:28.059000 20.1738862281
2017-11-28 14:57:28.059000 25.4861241591
2017-11-29 14:57:28.059000 21.1561404046
2017-11-30 14:57:28.059000 21.5878915513
2017-12-01 14:57:28.059000 25.2886133172
2017-12-02 14:57:28.059000 25.473154527
2017-12-03 14:57:28.059000 18.8324862157
2017-12-04 14:57:28.059000 22.4004864757
2017-12-05 14:57:28.059000 23.6705665673
2017-12-06 14:57:28.059000 18.9401742964
2017-12-07 14:57:28.059000 23.6959231626
2017-12-08 14:57:28.059000 25.3582087381
2017-12-09 14:57:28.059000 22.1228851296
2017-12-10 14:57:28.059000 20.0959712148
2017-12-11 14:57:28.059000 18.9074994664
2017-12-12 14:57:28.059000 23.5755356783
2017-12-13 14:57:28.059000 23.5191920638
2017-12-14 14:57:28.059000 24.8392962851
2017-12-15 14:57:28.059000 22.4796243738
2017-12-16 14:57:28.059000 19.6169403909
2017-12-17 14:57:28.059000 21.7104340797
2017-12-18 14:57:28.059000 21.7457176506
2017-12-19 14:57:28.059000 23.9967928662
2017-12-20 14:57:28.059000 21.9619090139
2017-12-21 14:57:28.059000 22.7953986498
2017-12-22 14:57:28.059000 23.6674168496
2017-12-23 14:57:28.059000 19.2265012544
2017-12-24 14:57:28.059000 18.1134346984
2017 12 25 14:57:28 059000 19 6699975705
https://colab.research.google.com/drive/1pdjrZp1qleIX5M2RghFJavNxa-Xoasmx#printMode=true 6/6