10/11/22

Mixing Relational with NoSQL

Heresy or Harmony?
Presented on 6th October 2022
at

RoOUG
ConTech 2022
Bucharest, Romania
by
Niall Mc Phillips - Long Acre sàrl
[email protected]
@Niall_McP

1
 10/11/22

2
 10/11/22

About me: Niall Mc Phillips

Owner - Long Acre sàrl (founded 2015)

Co-founder and Director - Stephenson and Associates (founded 1995)
Irish 🇮🇪 / 🇨🇭 Swiss Living in Geneva, Switzerland.

• Oracle ACE
• Using Oracle database as a Developer and DBA for >30 years
• Developing web applications with Oracle DB since 1995
• Developing with APEX since 2005
• Organizer of the Swiss APEX Meetup group

@NiallMcP
[email protected]

500+ technical experts

helping peers globally

The Oracle ACE Program recognizes and

rewards community members for their
technical and community contributions to the
Oracle community
3 membership tiers

Nominate
yourself or someone you know:
ace.oracle.com/nominate
For more details on Oracle ACE Program:
ace.oracle.com

Connect: [email protected] Facebook.com/OracleACEs @oracleace

3
 10/11/22

Relational – very-condensed history

• 1970 - First defined by E.F.Codd of IBM and was published in
the IBM Systems Journal

Relational – very-condensed history

• 1970 - First defined by E.F.Codd of IBM and was published in
the IBM Systems Journal

• 1979 - a start-up company called “Relational Software Inc.”

(RSI) released a product that they named “Oracle”
Interesting factoid, the first Oracle release was “version 2” – because no
one would want to buy version 1

4
 10/11/22

Relational - Normalisation
Let's start with a list of data representing short-term apartment rentals

Apartments
Address Description Landlord Landlord Landlord Currency Price / Amenities
phone e-mail week

21 Rue du Saut blah, blah D. Jepp 022 678 d.jepp@ap CHF 980 Wifi
4322 t.ch Kitchen
Balcony
62 Rue du Pirate blah, blah D. Jepp 022 678 d.jepp@ap CHF 1480 Wifi
4322 t.ch Kitchen
Garden
42 Rue des blah, blah M. Curphy 01 78 43 m.curphy CHF 520 Wifi
Caraïbes 22 56 @xyz.ch Kitchenette

Relational – 1st Normal Form

Multiple values not allowed in columns

Apartments
Address Description Landlord Landlord Landlord Currency Price / Amenities
phone e-mail week

21 Rue du Saut blah, blah D. Jepp 022 678 d.jepp@ap CHF 980 Wifi,
4322 t.ch Kitchen,
Balcony
62 Rue du Pirate blah, blah D. Jepp 022 678 d.jepp@ap CHF 1480 Wifi,
4322 t.ch Kitchen,
Garden
42 Rue des blah, blah M. Curphy 01 78 43 m.curphy CHF 520 Wifi
Caraïbes 22 56 @xyz.ch

10

5
 10/11/22

Relational – 1st Normal Form

Multiple values not allowed in columns

Apartments
Address Descriptio Landlord Landlord Landlord Currency Price /
n phone e-mail week

Address Amenity
21 Rue du Saut blah, blah D. Jepp 022 678 d.jepp@a CHF 980
4322 pt.ch 21 Rue du Saut Wifi
62 Rue du Pirate blah, blah D. Jepp 022 678 d.jepp@a 21 Rue du1480
CHF Saut Kitchen
4322 pt.ch
21 Rue du Saut Balcony
42 Rue des blah, blah M. Curphy 01 78 43 m.curphy CHF 520
62 Rue du Pirate Wifi
Caraïbes 22 56 @xyz.ch
62 Rue du Pirate Kitchen
62 Rue du Pirate Garden
42 Rue des Caraïbes Wifi

11

Relational – 2nd Normal Form

2nd Normal Form can be achieved by adding a single-value primary key

Apartments
ID Address Descript Landlor Landlor Landlor Currenc Price /
ion d d phone d e-mail y week

ID Address Amenit
1 21 Rue du blah, D. Jepp 022 678 d.jepp@ CHF 980 y
Saut blah 4322 apt.ch
1 21 Rue du Wifi
2 62 Rue du blah, D. Jepp 022 678 d.jepp@ CHF 1480Saut
Pirate blah 4322 apt.ch
2 21 Rue du Kitche
3 42 Rue des blah, M. 01 78 m.curph CHF 520 Saut n
Caraïbes blah Curphy 43 22 y@xyz.
56 ch 3 21 Rue du Balcon
Saut y
4 62 Rue du Wifi
Pirate
5 62 Rue du Kitche
Pirate n
6 62 Rue du Garde
Pirate n
12 7 42 Rue des Wifi
Caraïbes

6
 10/11/22

Relational – 3rd Normal Form

Landlords
Remove redundancies Name Phone e-mail
ID

Apartments 1 D. Jepp 022 678 4322 d.jepp@ap

ID Address Description Landlord ID Curr Price / t.ch
ency week M. 01 78 43 22 56 m.curphy
2
Curphy @xyz.ch
1 21 Rue du Saut blah, blah 1 CHF 980
Apartment
2 62 Rue du Pirate blah, blah 1 CHF 1480
Amenities
3 42 Rue des Caraïbes blah, blah 2 CHFApartment
520 Amenity Amenities
ID ID
ID Amenity
1 1
1 Wifi
1 2
2 Kitchen
1 3
3 Balcony
2 1
4 Garden
2 2
2 4
3 1
13

Relational – Normalisation
We could now construct SQL statements joining tables to
answer questions such as :
• Which apartments have kitchens and how much are they?
• Which apartments are operated by D. Jepp and what are
their amenities?
• etc.

14

7
 10/11/22

Relational – Major Benefits

• Data Integrity is ensured
• Structure is explicitly defined outside of the data
• Reliability, tried and tested approaches
• Easily-defined transactions

15

Relational – Some Drawbacks

• Lack of flexibility
• Complexity

16

8
 10/11/22

JSON
• Dates from the early-2000's by Douglas Crockford

17

JSON
• Dates from the early-2000's by Douglas Crockford
• First standardized in 2013 (ECMA-404)

18

9
 10/11/22

JSON
• Dates from the early-2000's by Douglas Crockford
• First standardized in 2013 (ECMA-404)
• 2017 – ISO/IEC standard (ISO/IEC 21778:2017)

19

JSON
• Dates from the early-2000's by Douglas Crockford
• First standardized in 2013 (ECMA-404)
• 2017 – ISO/IEC standard (ISO/IEC 21778:2017)
• Independent of underlying technologies

20

10
 10/11/22

JSON
• Dates from the early-2000's by Douglas Crockford
• First standardized in 2013 (ECMA-404)
• 2017 – ISO/IEC standard (ISO/IEC 21778:2017)
• Independent of underlying technologies
• Wide adoption in the development community

21

JSON

Let's take another look at our short-stay

apartment list

22

11
 10/11/22

A JSON object for one apartment

{
"id":1,
"address":"21 Rue du Saut",
"description":"blah, blah",
"weeklyPrice":"980",
"currency":"CHF",
"landlord":{”name":”D.Jepp",
"phone":"022 678 4322",
"email":"[email protected]"},
"amenities":["Wifi",
"Kitchen",
"Balcony"]
}

23

JSON

«Great - But…»
let's look at this in a different way

24

12
 10/11/22

A JSON object for one landlord

{
"id":"1",
"name":"D.Jepp",
"phone":"022 678 4322", {"id":”2",

"email": "[email protected]", "address":"62 Rue du Pirate",

"apartments":[ "description":"blah, blah",

{"id":"1", "weeklyPrice":"1480",

"address":"21 Rue du Saut", "currency":"CHF",

"description":"blah, blah", "amenities":["Wifi",

"Kitchen",
"weeklyPrice":"980",
"Garden"]
"currency":"CHF",
}
"amenities":["Wifi",
"Kitchen", ]

"Balcony"] }

},

25

Adding reviews
Let's add reviews from people that have stayed in the
apartments
• Reviewer ID
• Reviewer Name
• Stars Given
• Review Text

26

13
 10/11/22

Adding reviews - Relational

Data Model changes – add at least 2 tables
• a table of Reviewers with ID and Name
• a table of Reviews
Reviews
APT ID Reviewer ID Stars Review Text
1 1 5 Great apartment! Reviewers
1 2 4 Nice apartment, but ID Name
1 James Plunkett
2 James Connolly

27

Adding reviews - JSON

• Add an array of reviews
{
"id":1,
"address":"21 Rue du Saut",
…
,
"reviews":[{"reviewerId":1,
"name":"James Plunkett",
"stars":5,
"text":"Great apartment!”},
{"reviewerId":2,
"name","James Connolly",
"stars":4,
"text":"Nice Apartment, but"}
]
}
28

14
 10/11/22

Adding even more reviews

What if we have 1,000's of reviews?

• Relational – the Reviews table just gets more entries

• JSON – we get a gigantic array and the JSON object

becomes huge.

29

Adding even more reviews

What if we have 1,000's of reviews?

30

15
 10/11/22

Adding even more reviews

What if we have 1,000's of reviews?
Another approach from the JSON viewpoint would be to
predict application usage patterns.

31

Adding even more reviews

What if we have 1,000's of reviews?
Another approach from the JSON viewpoint would be to
predict application usage patterns.
For example:
• Users usually view the apartment listing along with the
description and the most recent reviews.

32

16
 10/11/22

Adding even more reviews

What if we have 1,000's of reviews?
Another approach from the JSON viewpoint would be to
predict application usage patterns.
For example:
• Users usually view the apartment listing along with the
description and the most recent reviews.
• So, why not just keep the 5 most recent reviews in the
JSON object?

33

Once upon a time

Many, many years ago when I was learning about data
modelling, one thing was repeatedly hammered into my
brain
“Never consider the application

when modelling the data

- let the data speak for itself”

34

17
 10/11/22

More recently
“Let's write the applications,
we'll structure the data as we evolve”

35

Culture Clash?
“Never consider the application
when modelling the data
- let the data speak for itself”
vs.
“Let's write the application,
we'll structure the data as we evolve”

36

18
 10/11/22

Culture Clash?

Maybe there's another path...

37

What if we could do both?

• The benefits of the structure of a relational database for
the core model

PLUS

• The flexibility and ease of deployment of JSON

38

19
 10/11/22

JSON in the Oracle Database

With 21c we can now store JSON as a native datatype
JSON Binary Type
New datatype in SQL, PL/SQL
Natively supported in all drivers
OCI, JDBC, node.js, python (.NET in progress)
Based on OSON - Optimized Binary representation
Self-contained format
Fast field lookups
Piecewise Updates possible
Performance Benefits
Scans up to 5x faster than textual JSON
Updates up to 10x faster than textual JSON

39

Defining a JSON column

create table apartments

(apt_id number,
address varchar2(255),
recent_reviews JSON);

40

20
 10/11/22

Inserting JSON
insert into apartments values
(4, '78 Rue de l''Avenir',
'{"reviews":
[{"name": "James Plunkett",
"stars": 5,
"text": "Great apartment!" },
{"name": "James Connolly",
"stars": 4,
"text": "Nice Apartment, but..."}
]
}');

41

Inserting JSON using JSON_OBJECT

insert into apartments (apt_id, address, recent_reviews)
select 7, '6 Rue des Autres',

json_object('reviews'
value json_arrayagg(
json_object(key 'name' is d.name,
key 'stars' is d.stars,
key 'text' is d.text)
returning clob)
returning clob)
from (select 'Ard Rí' as name, 5 as stars, 'Enjoyable stay' as text from dual
union
select 'Ard Banríon', 5, 'Good enough for me' from dual) d

42

21
 10/11/22

Demo time!

43

Querying JSON –
Dot notation
select a.apt_id,
a.address,
a.recent_reviews.reviews[0].name,
a.recent_reviews.reviews[0].name.string(),
a.recent_reviews.reviews[1].name,
a.recent_reviews.reviews[1].name.string()
from apartments a;

44

22
 10/11/22

Querying JSON –
All array elements as a JSON array
select a.apt_id,
a.address,
a.recent_reviews.reviews[*].name as reviewers
from apartments a;

45

Querying JSON –
Using JSON_TABLE to return multiple rows
select a.apt_id, a.address,
j.name as reviewer,
j.stars, j.text as review
FROM apartments a,
json_table(a.recent_reviews, '$.reviews[*]'
columns (name varchar2(30) path '$.name',
stars number path '$.stars',
text varchar2(50) path '$.text')) j;

46

23
 10/11/22

Querying JSON –
Creating a view on the JSON values
create or replace view vw_apartment_reviews as
select a.apt_id, a.address,
j.name as reviewer,
j.stars, j.text as review
FROM apartments a,
json_table(a.recent_reviews, '$.reviews[*]'
columns (name varchar2(30) path '$.name',
stars number path '$.stars',
text varchar2(50) path '$.text')) j;

47

Updating JSON – JSON_TRANSFORM

changing a value
update apartments a
set a.recent_reviews
= json_transform
(a.recent_reviews,
set '$.latestStay' = to_char(sysdate,'YYYY-MM-DD'))
where a.apt_id = 7;

48

24
 10/11/22

Updating JSON – JSON_TRANSFORM

removing an array element
update apartments a
set a.recent_reviews
= json_transform
(a.recent_reviews,
remove '$.reviews[*]?(@.name=="Ard Banríon")')
where a.apt_id = 7;

49

Updating JSON – JSON_TRANSFORM

removing an array element
update apartments a
set a.recent_reviews
= json_transform
(a.recent_reviews,
append '$.reviews'
= json_object(key 'name' is 'ConTech',
key 'stars' is 5,
key 'text' is 'Approved by RoOUG'))
where a.apt_id = 7;

50

25
 10/11/22

Updating JSON – JSON_TRANSFORM

appending an array element
update apartments a
set a.recent_reviews
= json_transform
(a.recent_reviews,
remove '$.reviews[*]?(@.name=="Ard Banríon")')
where a.apt_id = 7;

51

Indexing JSON

• Function indexes for simple cases

• Multivalue indexes for array elements
• Search Index for other searches

See Search indexes for JSON – Roger Ford, Oracle - 23rd Nov 2021

52

26
 10/11/22

Indexing JSON - Simple cases

• Function indexes for simple cases

create index ind_apartments$1

on apartments
(recent_reviews.latestStay.string());

then
select * from apartments a
where a.recent_reviews.latestStay.string() = '2022-09-30';

53

Indexing JSON - Multivalue indexes

For array elements:
create multivalue index ind_apartments$2
on apartments a
(a.recent_reviews.reviews[*].name.string());

then...
select a.* from apartments a
where json_exists(a.recent_reviews,
'$.reviews?(@.name == "James Plunkett")');

54

27
 10/11/22

Indexing JSON – Search Indexes

For textual searches (similar to Oracle Text):
create search index ind_apartments$3
on apartments (recent_reviews) for json;

then...
select a.* from apartments a where
json_textcontains(a.recent_reviews,
'$.reviews.name’,
'james');

55

JSON in the DB Use Cases – some examples

• Equipment certification – the certificates should reflect only
the certificate information issued at the date of issue despite
any changes to the data structure since certification.

• Auditing – allows data changes to be tracked over an

evolving data model

• Fast-moving, “temporary” data – i.e. this month's “special

pick”

56

28
 10/11/22

Upcoming Oracle 23c JSON features

JSON Schema
• various options to validate documents against a JSON
Schema definition. The JSON Schema can be defined on a
table column – almost as a check constraint. It can be
used in a query to only select values that satisfy the
schema and using package dbms_json_schema validation
reports can be retrieved on specific values against a JSON
Schema.

57

Upcoming Oracle 23c JSON features

JSON Schema

58

29
 10/11/22

JSON in Oracle – Multiple avenues

• Oracle SODA – accepts JSON from multiple environments
• Java, Node.js, REST, C, Python, PL/SQL

• Oracle's new MongoDB Drivers and Tools

• REST and ORDS

• SQL and PL/SQL

59

PL/SQL - APEX_JSON
There is a DB package called APEX_JSON that can be used
for parsing and generating JSON – available since APEX 5.0

apex_json.open_object();
apex_json.write('latestStay','2022-10-06’);
apex_json.openArray('reviews’);
apex_json.open_object();
apex_json.write('name','RoOUG Reviewer');
apex_json.write('stars','5');
apex_json.write(‘text’,’Lovely place’);
apex_json.close_object();
apex_json.close_array();
apex_json.close_object();

60

30
 10/11/22

PL/SQL – JSON Object Types

JSON_OBJECT_T
JSON_ARRAY_T

Each object type has multiple methods are available for

parsing, building, modifying and inspecting JSON
structures

61

PL/SQL – JSON Object Types vs APEX_JSON

A recent article by Jon Dixon would indicate that the PL/SQL
JSON Object types are many times faster than APEX_JSON for
parsing JSON.

For Speeds Sake, Stop Using APEX_JSON (Jon Dixon, 5th June 2022)

62

31
 10/11/22

Advantages of Relational / JSON Hybrid models

• Less tables, more flexibility
• Very infrequently used attributes don’t need to be
modelled as stringently
• More care needs to ensure integrity

63

Challenges of Hybrid models

• With less tables, more flexibility - more attention
needs to be paid to data integrity

• Finding the right balance between relational and

JSON for your application and environment

64

32
 10/11/22

NoSQL / JSON in the Database

• JSON in the Database is here to stay

• Let’s embrace it and add it to our toolkit

65

Any Questions ??

66

33