ZetCode Apache Derby Tutorial
ZetCode Apache Derby Tutorial
This is Apache Derby tutorial. This tutorial will cover the Derby database engine, the SQL
understood by the Derby and programming Derby with the JDBC API.
Derby
Derby is a database engine written in Java language. It has a relatively small footprint of 23MB. It works in two modes. In embedded or client-server mode.
Derby has roots in 1996 as JBMS. Later it was renamed to Cloudscape. In 1999 the company
developing Cloudscape was bought by Informix, which was later acquired by IBM. In 2004
IBM contributed the code to the Apache Software Foundation. The Derby project was born.
Sun joined the project and included the Derby in Java 6. The project is rebranded as Java DB
in JDK.
Introduction to Derby
In this chapter, we will cover the basic concepts and definitions of Derby. It is necessary to
know these basics because Derby is quite involved. It is not as simple as SQLite.
Derby is a relational database management system written in Java. It implements an SQL-92
core subset, as well as some SQL-99 features. It uses IBM DB2 SQL syntax. Derby has a
small footprint around 2MB. It has transaction support. The database format used by Derby is
portable and platform independent.
Deployment options
Derby can operate in two modes. Embedded or client-server. In the embedded mode the
Derby engine runs within the Java Virtual Machine of the application. The application is
accessing the database directly and exclusively. No other application can access the database
simultaneously. There is no need to set up a Derby server before connecting to the database.
The Derby engine is started when we load the embedded driver. In the client-server mode
Derby provides multi-user connectivity across a network. The Derby engine runs in the JVM
of the server. Other applications can connect to the Database from different JVMs. We must
start a database server before connecting to the database. This is similar to other RDBMS like
Oracle or MySQL.
The Derby system is specified with the derby.system.home property. If the directory
specified in the property does not exist, Derby creates it automatically. If we do not specify
the property explicitly, the current directory is used. The current directory is the value of the
JVM user.dir property. It is recommended to always specify the derby.system.home
explicitly.
-Dderby.system.home=/home/janbodnar/programming/derby/dbs
An example of how we set the property. It can be set on the command line, in a dialog box in
case we use an IDE or in the derby.properties file.
The first JDBC call shuts down the testdb database. The second call ends the whole Derby
system. Note, that na SQLExpection is raised when a system shuts down. This exception
should be programmatically handled. We should also close all existing resources before
shutting down the database or the system.
In the client-server mode, the Derby server is started with the startNetworkServer script and
ended with the stopNetworkServer script. They are located in the bin subdirectory of the
installation directory.
locale libraries. They are used to provide translated messages. For example, the
derbyLocale_cs.jar provides Czech messages.
The above is a syntax for a Derby database connection URL. The default subprotocol is
directory: and it is often omitted. We can work with databases only in memory when we
specify the memory: subprotocol. The databaseName is the name of the database that we
want to create and/or connect to.
We can use several attributes in the connection URL. We can use attributes to create a
database, to connect to a secured database with a user name and a password. Further we use
the connection attributes to shut down a database or a Derby system, to encrypt data or to
restore a database from a backup.
jdbc:derby:testdb
jdbc:derby://localhost:1527/testdb
jdbc:derby:testdb;create=true
jdbc:derby:testdb;shutdown=true
jdbc:derby:memory:testdb
We have another three connection strings. The first connection string creates the testdb
database. The second one shuts down the testdb database. The third one connects to a testdb
created in memory.
SQL files
In the following two SQL files, cars.sql and authors_books.sql, we create three tables which
we will use throughout this tutorial.
$ cat cars.sql
SET SCHEMA USER12;
CREATE TABLE CARS(ID INT PRIMARY KEY, NAME VARCHAR(30), PRICE INT);
INSERT INTO CARS VALUES(1, 'Audi', 52642);
INSERT INTO CARS VALUES(2, 'Mercedes', 57127);
INSERT INTO CARS VALUES(3, 'Skoda', 9000);
INSERT INTO CARS VALUES(4, 'Volvo', 29000);
INSERT INTO CARS VALUES(5, 'Bentley', 350000);
INSERT INTO CARS VALUES(6, 'Citroen', 21000);
INSERT INTO CARS VALUES(7, 'Hummer', 41400);
INSERT INTO CARS VALUES(8, 'Volkswagen', 21600);
INTO
INTO
INTO
INTO
INTO
INSERT INTO
INSERT INTO
INSERT INTO
INSERT INTO
INSERT INTO
INSERT INTO
INSERT INTO
INSERT INTO
INSERT INTO
Tiffany');
AUTHORS(ID,
AUTHORS(ID,
AUTHORS(ID,
AUTHORS(ID,
AUTHORS(ID,
BOOKS(ID,
BOOKS(ID,
BOOKS(ID,
BOOKS(ID,
BOOKS(ID,
BOOKS(ID,
BOOKS(ID,
BOOKS(ID,
BOOKS(ID,
NAME)
NAME)
NAME)
NAME)
NAME)
VALUES(1,
VALUES(2,
VALUES(3,
VALUES(4,
VALUES(5,
AUTHOR_ID,
AUTHOR_ID,
AUTHOR_ID,
AUTHOR_ID,
AUTHOR_ID,
AUTHOR_ID,
AUTHOR_ID,
AUTHOR_ID,
AUTHOR_ID,
TITLE)
TITLE)
TITLE)
TITLE)
TITLE)
TITLE)
TITLE)
TITLE)
TITLE)
'Jack London');
'Honore de Balzac');
'Lion Feuchtwanger');
'Emile Zola');
'Truman Capote');
VALUES(1,
VALUES(2,
VALUES(3,
VALUES(4,
VALUES(5,
VALUES(6,
VALUES(7,
VALUES(8,
VALUES(9,
1,
1,
2,
2,
3,
4,
4,
5,
5,
Sources
The following materials were used to create this tutorial. Derby Developer's Guide, Derby
Server and Administration Guide, Getting Started with Derby, Derby Tools and Utilities
Guide and Derby Reference Manual.
In the chapter, we have introduced the basic concepts of the Derby database.
docs
index.html
javadoc
KEYS
lib
LICENSE
NOTICE
RELEASE-NOTES.html
test
We list the contents of the installation directory. In the bin subdirectory, we have several
Derby tools. The docs directory provides documentation for Derby. In HTML and PDF
format. The documentation is very good and we highly recommend to have a look at these
files. In the lib subdirectory there are various JAR files needed for working with Derby.
The Java DB
Derby is included in the JDK since Java 6. Java DB is a supported version of Apache Derby
and contains the same binaries as Apache Derby.
$ ls jdk1.6.0_30/db
3RDPARTY demo index.html
bin
docs javadoc
lib
LICENSE
NOTICE
register.html
RELEASE-NOTES.html
In the JDK installation directory, we have a subdirectory called db, where we find the Java
DB files.
In the Overview section of the release notes we read the following:
Derby is a pure Java relational database engine using standard SQL and JDBC as its APIs.
Derby functionality includes:
Environment variables
Setting environment variables is optional. Using environment variables will make the life
easier for us.
$ export DERBY_HOME=~/bin/derby
$ export PATH=$PATH:~/bin/derby/bin
$ export DERBY_HOME=Dderby.system.home=/home/janbodnar/programming/derby/dbs
We set a DERBY_HOME variable to the Derby installation directory. Some tools may use
this variable. Then we add a bin subdirectory to the PATH variable. This way we do not have
to fully specify the path to the Derby tools. In the DERBY_OPTS environment variable we
can set various Derby or Java related options. For example, we set the derby.system.home
property.
$ java -jar $DERBY_HOME/lib/derbyrun.jar sysinfo
------------------ Java Information -----------------Java Version:
1.6.0_30
Java Vendor:
Sun Microsystems Inc.
Java home:
/home/janbodnar/bin/jdk1.6.0_30/jre
Java classpath: /home/janbodnar/bin/derby/lib/derbyrun.jar
OS name:
Linux
OS architecture: i386
...
We use the derbyrun.jar file to execute the sysinfo tool to get some info about the Java and
Derby.
Creating a database
Derby does not have a CREATE DATABASE statement like MySQL or Oracle. We have to
create a database by creating a connection and setting a connection property create=true.
$ java -Dderby.system.home=/home/janbodnar/programming/derby/dbs \
> -jar $DERBY_HOME/lib/derbyrun.jar ij
ij version 10.8
ij>
We start the ij tool with the derbyrun.jar file. We set the derby system directory. It is the
directory, where we create databases and where log file is created.
ij> connect 'jdbc:derby:tmpdb;user=tmpuser;create=true';
A database tmpdb is created inside the Derby system directory and we also connect to it. The
connection is created in the embedded mode.
ij> CREATE TABLE FRIENDS(ID INT PRIMARY KEY, NAME VARCHAR(25));
0 rows inserted/updated/deleted
ij> INSERT INTO FRIENDS(ID, NAME) VALUES(1, 'Jane');
1 row inserted/updated/deleted
ij> INSERT INTO FRIENDS(ID, NAME) VALUES(2, 'Thomas');
1 row inserted/updated/deleted
When we created a connection, we have specified a user name. The user name is the database
schema, in which the FRIENDS table is created.
ij> DISCONNECT;
ij> SHOW CONNECTIONS;
No connections available.
ij> EXIT;
$
We disconnect from the tmpdb database. The SHOW CONNECTIONS statement informs that
there are no open connections. We exit the ij tool with the EXIT command.
$ rm -rf dbs/tmpdb
We delete the database from the Derby system. Derby has no DROP DATABASE statement.
This is the end of the Derby tutorial chapter, in which we have installed and configured
Derby.
Derby tools
In this chapter, we will mention Derby tools. The Derby tools and utilities are a set of scripts
supplied with Derby. They are typically used to create, inspect, and update a Derby database.
In this page, we will mention the sysinfo, the dblook, the ij, the startNetworkServer and the
stopNetworkServer tools. Note that these tools have .bat extension on Windows.
Assuming that the Derby bin directory is added to the PATH environment variable, we can
launch the ij tool by specifying the name of the script in the terminal. The second line runs
the ij using the derbyrun.jar file.
sysinfo
The sysinfo tool provides information about the Operating system, Java and Derby. It will
print among others Java version, Java home directory, OS version, Java runtime version,
Derby version, current and supported locales. The tool can be useful to track down some
installation or configuration issues with Derby.
$ sysinfo
------------------ Java Information -----------------Java Version:
1.6.0_30
Java Vendor:
Sun Microsystems Inc.
Java home:
/home/janbodnar/bin/jdk1.6.0_30/jre
Java classpath:
/home/janbodnar/bin/derby/lib/derby.jar:/home/janbodnar/bin/derby/lib/
derbynet.jar:/home/janbodnar/bin/derby/lib/derbytools.jar:/home/janbodnar/b
in/
derby/lib/derbyclient.jar
OS name:
Linux
OS architecture: i386
OS version:
3.0.0-15-generic
Java user name: janbodnar
Java user home: /home/janbodnar
...
ij
The ij is an interactive scripting tool. It is used for running scripts or interactive queries
against a Derby database.
$ cat cars.sql
CREATE SCHEMA USER12;
CREATE TABLE CARS(ID INT PRIMARY KEY, NAME VARCHAR(30), PRICE INT);
INSERT INTO CARS VALUES(1, 'Audi', 52642);
INSERT INTO CARS VALUES(2, 'Mercedes', 57127);
INSERT INTO CARS VALUES(3, 'Skoda', 9000);
INSERT INTO CARS VALUES(4, 'Volvo', 29000);
INSERT INTO CARS VALUES(5, 'Bentley', 350000);
INSERT INTO CARS VALUES(6, 'Citroen', 21000);
INSERT INTO CARS VALUES(7, 'Hummer', 41400);
INSERT INTO CARS VALUES(8, 'Volkswagen', 21600);
We have a cars.sql file which creates a database schema and a CARS table.
$ ij
ij version 10.8
ij> connect 'jdbc:derby:dbs/testdb;user=user12;create=true';
We start the ij tool. If we had not added the Derby's bin directory to the PATH variable, we
would have to specify the whole path to the ij tool. We create a testdb database in the dbs
directory and make a connection to it. The dbs directory is a subdirectory of our current
directory, where the ij was launched.
ij> SHOW CONNECTIONS;
CONNECTION0* - jdbc:derby:dbs/testdb
* = current connection
We load and execute the cars.sql site. We are informed about the ongoing operations.
ij> SELECT * FROM CARS;
ID
|NAME
|PRICE
-----------------------------------------------------1
|Audi
|52642
2
|Mercedes
|57127
3
|Skoda
|9000
4
|Volvo
|29000
5
|Bentley
|350000
6
|Citroen
|21000
7
|Hummer
|41400
8
|Volkswagen
|21600
8 rows selected
Shutting down a database in Derby results in an exception. The ERROR 08006 is expected.
ij> SHOW CONNECTIONS;
No current connection
ij> EXIT;
We quit the ij tool with the EXIT command. Note that each command is followed by
semicolon.
dblook
The dblook tool is used to save the data definition language of database objects including
tables, views, indexes or triggers.
$ dblook -d jdbc:derby:dbs/testdb
-- Timestamp: 2012-02-12 13:33:53.677
-- Source database is: dbs/testdb
-- Connection URL is: jdbc:derby:dbs/testdb
-- appendLogs: false
-- ----------------------------------------------- DDL Statements for schemas
-- ---------------------------------------------CREATE SCHEMA "USER12";
-- ----------------------------------------------- DDL Statements for tables
-- ---------------------------------------------CREATE TABLE "USER12"."CARS" ("ID" INTEGER NOT NULL,
"NAME" VARCHAR(30), "PRICE" INTEGER);
-- ----------------------------------------------- DDL Statements for keys
-- ----------------------------------------------- primary/unique
ALTER TABLE "USER12"."CARS" ADD CONSTRAINT "SQL120212131535700" PRIMARY KEY
("ID");
In the above example, we have dumped the objects from the testdb database. With the -d
option we have provided the connection URL to the database. In our case the dblook tool
saved a database schema and one table. With the -o option the output can be redirected to a
file.
Here we start the Derby Network Server with the startNetworkServer script.
10
Here we connect to the testdb database via the Derby Network Server. The connection URL
is different for networked connections.
ij> SELECT * FROM USER12.CARS;
ID
|NAME
|PRICE
-----------------------------------------------------1
|Audi
|52642
2
|Mercedes
|57127
3
|Skoda
|9000
4
|Volvo
|29000
5
|Bentley
|350000
6
|Citroen
|21000
7
|Hummer
|41400
8
|Volkswagen
|21600
8 rows selected
We select all cars from the CARS table. Since we have not provided the database schema in
the connection URL, we must specify it now. The database schema is the user name; in our
case USER12.
$ stopNetworkServer
Sun Feb 12 14:48:51 CET 2012 : Apache Derby Network Server
- 10.8.2.2 - (1181258) shutdown
$ Sun Feb 12 14:48:51 CET 2012 : Apache Derby Network Server
- 10.8.2.2 - (1181258) shutdown
The ij tool
In the fourth chapter, we are going to look at Derby's ij tool in a greater detail.
The ij is an interactive scripting tool supplied with Derby. It is a command line client for the
Derby database system. It can be used in two ways. Either run SQL files or interactively
execute SQL statements. The ij is located in the bin directory of the Derby installation
directory.
$ ls $DERBY_HOME/bin | grep ij
ij
ij.bat
There are two scripts. The one with the .bat extension is for Windows.
Starting ij
The ij can be started in three basic ways.
$ ij
11
ij version 10.8
ij>
Derby ij can be started with the ij script. The above example assumes that we have the
Derby's bin directory in our PATH variable. Otherwise we must use the full path to the ij
tool.
$ java -cp $DERBY_HOME/lib/derbytools.jar org.apache.derby.tools.ij
ij version 10.8
ij>
Another way is to execute the compiled Java program. We must have the derbytools.jar in
our classpath. The first way does essentially the same within the script file. It also works with
environment variables.
$ java -jar $DERBY_HOME/lib/derbyrun.jar ij
ij version 10.8
ij>
In the third way, we use the derbyrun.jar file to start the ij tool.
A CARS table is created in schema USER12 and five rows are inserted.
ij> RUN 'cars.sql';
ij> SET SCHEMA USER12;
0 rows inserted/updated/deleted
ij> CREATE TABLE CARS(ID INT PRIMARY KEY, NAME VARCHAR(30), PRICE INT);
0 rows inserted/updated/deleted
ij> INSERT INTO CARS VALUES(1, 'Audi', 52642);
1 row inserted/updated/deleted
...
We use the RUN command to execute the cars.sql file. The file is located in the same
directory, where ij was launched.
ij> SELECT * FROM CARS;
ID
|NAME
|PRICE
12
-----------------------------------------------------1
|Audi
|52642
2
|Mercedes
|57127
3
|Skoda
|9000
4
|Volvo
|29000
5
|Bentley
|350000
6
|Citroen
|21000
7
|Hummer
|41400
8
|Volkswagen
|21600
8 rows selected
We add a CONNECT statement to the cars2.sql file. When we launch the ij tool, we are not
yet connected to the database.
ij> DROP TABLE CARS;
0 rows inserted/updated/deleted
ij> EXIT;
$
We set the Derby system directory and launch the ij tool with the cars2.sql as a parameter.
The CARS table is created again.
Basic commands
We can issue two kinds of commands. Commands specific to the ij tool and SQL statements.
Each command in ij is terminated with a semicolon. All ij commands, identifiers, and
keywords are case-insensitive.
ij> HELP;
13
The CONNECT command connects to a database. In our case the database name is testdb.
This example assumes that we have set the Derby system directory. (More about it in the next
section.) Derby by default does not require a user name and a password. We can configure
Derby to require it.
ij> SHOW CONNECTIONS;
CONNECTION0* - jdbc:derby:testdb
* = current connection
The SHOW CONNECTIONS statement lists all opened connections. In our case we can see an
opened connection to the testdb database.
ij> SHOW TABLES;
TABLE_SCHEM
|TABLE_NAME
|REMARKS
-----------------------------------------------------------------------SYS
|SYSALIASES
|
SYS
|SYSCHECKS
|
SYS
|SYSCOLPERMS
|
...
The SHOW TABLES command shows all tables in a database. There are a few SYS tables.
ij> SHOW TABLES IN USER12;
TABLE_SCHEM
|TABLE_NAME
|REMARKS
-----------------------------------------------------------------------USER12
|CARS
|
1 row selected
We can list tables in a specific schema. The SHOW TABLES IN USER12 shows tables in the
USER12 schema.
ij> DESCRIBE USER12.CARS;
COLUMN_NAME
|TYPE_NAME|DEC&|NUM&|COLUM&|COLUMN_DEF|CHAR_OCTE&|
IS_NULL&
----------------------------------------------------------------------------ID
|INTEGER |0
|10 |10
|NULL
|NULL
|NO
NAME
|VARCHAR |NULL|NULL|30
|NULL
|60
|YES
PRICE
|INTEGER |0
|10 |10
|NULL
|NULL
|YES
14
3 rows selected
The DESCRIBE command provides a decription of the specified table or view. If the
USER12 schema is not the current schema of the connection, we have to specify it before the
table name. The current schema is specified in the connection string as the user name.
ij> DISCONNECT;
ij> SHOW CONNECTIONS;
No connections available.
The DISCONNECT command disconnects from the database. The subsequent SHOW
CONNECTIONS command shows no available connections.
ij> EXIT;
$
Finally, we exit the ij tool. In case of an embedded environment, it also shuts down the
database. Which is equivalent to the CONNECT 'jdbc:derby:testdb;shutdown=true';
command.
The main purpose of the ij tool is to issue SQL commands. We reconnect to the testdb
database.
$ ij
ij version 10.8
ij> CONNECT 'jdbc:derby:testdb;user=USER12';
Now the current schema is the USER12 schema. When issuing SQL statements for tables
located in the USER12 schema, we can omit the schema name.
ij> SELECT * FROM CARS WHERE ID IN (1, 3, 5);
ID
|NAME
|PRICE
-----------------------------------------------------1
|Audi
|52642
3
|Skoda
|9000
5
|Bentley
|350000
3 rows selected
In the above SQL statement, we select all three columns for rows with IDs 1, 3 and 5.
ij> SELECT * FROM APP.FRIENDS;
ID
|NAME
------------------------------------1
|Jane
2
|Thomas
3
|Beka
3 rows selected
Three rows from the FRIENDS table were selected. Since the table is not located in the
current schema, we must fully qualify the table name.
15
We insert a new row into the FRIENDS table and later select it.
dbs
doc
doc~
dump
ij.properties
lib
links
The current working directory has a subdirectory called dbs. We want this directory to be our
Derby system directory. The current working directory is the place from where we launch the
ij tool.
$ java -Dderby.system.home=/home/janbodnar/programming/derby/dbs \
> -jar $DERBY_HOME/lib/derbyrun.jar ij
ij version 10.8
Here we specify the Derby system directory with the -D JVM option. The derby.log file is
created in the system directory. In our case the log file should appear in the dbs directory.
The derby.log file is recreated each time we connect to a Derby database. We can look at the
timestamp. If the derby.log file appears ouside the intended directory, we have not set the
Derby system directory correctly.
ij> CONNECT 'jdbc:derby:testdb';
ij> SHOW CONNECTIONS;
CONNECTION0* - jdbc:derby:testdb
* = current connection
Since we have correctly set the Derby system directory, we only specify the database name in
the connection URL and omit the full path. The SHOW CONNECTIONS shows the opened
connection.
We might not want to specify the Derby system directory each time. We could utilize the
DERBY_OPTS environment variable.
$ echo $DERBY_OPTS
-Dderby.system.home=/home/janbodnar/programming/derby/dbs
16
$ ij
ij version 10.8
ij> CONNECT 'jdbc:derby:testdb';
ij> SHOW CONNECTIONS;
CONNECTION0* - jdbc:derby:testdb
* = current connection
Creating a connection to a Derby database is now a bit easier. We save a few key strokes.
The ij properties
When starting up the ij tool, we can specify properties on the command line or in a properties
file. The properties are various parameters taken by the ij tool. They can save us some
repetitive work.
$ java -Dij.user=USER12 -Dij.database=testdb -Dij.protocol=jdbc:derby: \
> -Dderby.system.home=/home/janbodnar/programming/derby/dbs \
> -jar $DERBY_HOME/lib/derbyrun.jar ij
ij version 10.8
CONNECTION0* - jdbc:derby:testdb
* = current connection
ij> SELECT * FROM CARS WHERE ID = 1;
ID
|NAME
|PRICE
-----------------------------------------------------1
|Audi
|52642
1 row selected
ij>
We provide three ij properties on the command line with the -D option. The ij.user specifies
the user name to establish a connection. The supplied user name becomes the current schema.
The ij.database has the database name to which we connect. The ij.protocol property specifies
the default protocol and subprotocol portions of the database connection URL. We are ready
to launch SQL statements.
In the next example, we create an ij.properties file, where we set three ij properties.
$ cat ij.properties
ij.user=USER12
ij.database=testdb
ij.protocol=jdbc:derby:
With the cat command, we show the contents of the ij.properties file. We set the same
properties as in the first example.
$ java -Dderby.system.home=/home/janbodnar/programming/derby/dbs \
> -jar $DERBY_HOME/lib/derbyrun.jar ij -p ij.properties
ij version 10.8
CONNECTION0* - jdbc:derby:testdb
* = current connection
ij> SELECT * FROM CARS WHERE ID=2;
ID
|NAME
|PRICE
-----------------------------------------------------2
|Mercedes
|57127
17
1 row selected
SQL
In the following pages, we will work with the SQL understood by the Derby database engine.
We will not go into much detail about the SQL language. This chapter is a quick recap of the
most important SQL statements present in the Derby database.
SQL (Structured Query Language) is a database computer language designed for
managing data in relational database management systems. Derby supports only a limited set
of SQL statements. Some important statements known from other database systems are
missing. Derby implements an SQL-92 core subset, as well as some SQL-99 features.
ij> DROP TABLE AUTHORS;
0 rows inserted/updated/deleted
ij> DROP TABLE BOOKS;
0 rows inserted/updated/deleted
We have previously created the AUTHORS and BOOKS tables. We are going to recreate
them. The DROP TABLE SQL statement drops the table from the database. Note that the
DROP TABLE IF EXISTS statement does not exist in Derby.
ij> CREATE TABLE AUTHORS(ID INT PRIMARY KEY, NAME VARCHAR(25));
0 rows inserted/updated/deleted
The CREATE TABLE statement creates a new table. It has two columns, ID and NAME. In
the ID column we will place integers, in the NAME column strings with up to 25 characters.
A PRIMARY KEY uniquely identifies each record in the table. Each author is a unique
personality. Even if there are authors with the same name, each of them is in a separate row
in the AUTHORS table. Only one column in a table can have this constraint.
ij> CREATE TABLE BOOKS(ID INT PRIMARY KEY, AUTHOR_ID INT,
> TITLE VARCHAR(100), FOREIGN KEY(AUTHOR_ID) REFERENCES AUTHORS(ID));
0 rows inserted/updated/deleted
We create a BOOKS table, which has three columns. The FOREIGN KEY specifies that the
values in the AUTHOR_ID column must match the values in the ID column of the
AUTHORS table. Foreign keys provide a way to enforce the referential integrity of a
database. Each book was written by one or more authors. So in the BOOKS table for the
AUTHOR_ID column we can have only values that are present in the AUTHORS table.
ij>
ij>
ij>
ij>
ij>
INSERT
INSERT
INSERT
INSERT
INSERT
INTO
INTO
INTO
INTO
INTO
AUTHORS(ID,
AUTHORS(ID,
AUTHORS(ID,
AUTHORS(ID,
AUTHORS(ID,
NAME)
NAME)
NAME)
NAME)
NAME)
VALUES(1,
VALUES(2,
VALUES(3,
VALUES(4,
VALUES(5,
'Jack London');
'Honore de Balzac');
'Lion Feuchtwanger');
'Emile Zola');
'Truman Capote');
We add five rows to the AUTHORS table using the INSERT INTO SQL statement.
18
ij> INSERT
Wild');
ij> INSERT
ij> INSERT
ij> INSERT
ij> INSERT
ij> INSERT
ij> INSERT
Paris');
ij> INSERT
ij> INSERT
Tiffany');
BOOKS(ID,
BOOKS(ID,
BOOKS(ID,
BOOKS(ID,
BOOKS(ID,
BOOKS(ID,
AUTHOR_ID,
AUTHOR_ID,
AUTHOR_ID,
AUTHOR_ID,
AUTHOR_ID,
AUTHOR_ID,
TITLE)
TITLE)
TITLE)
TITLE)
TITLE)
TITLE)
VALUES(2,
VALUES(3,
VALUES(4,
VALUES(5,
VALUES(6,
VALUES(7,
1,
2,
2,
3,
4,
4,
'Martin Eden');
'Old Goriot');
'Cousin Bette');
'Jew Suess');
'Nana');
'The Belly of
The above SQL query joins the two tables. It assigns each book title to an author.
Autoincrement
Derby allows to create an autoincrement column. The value of an autoincrement column
increments automatically with every INSERT.
CREATE TABLE FRIENDS(ID INT PRIMARY KEY
GENERATED ALWAYS AS IDENTITY
(START WITH 1, INCREMENT BY 1),
NAME VARCHAR(20));
INSERT INTO FRIENDS(NAME) VALUES('Jane');
INSERT INTO FRIENDS(NAME) VALUES('Thomas');
INSERT INTO FRIENDS(NAME) VALUES('Beka');
In the above SQL code, the ID column is an autoincrement column. If we omit the ID in the
SQL INSERT statements the Derby automatically generates a unique number. It starts with
number 1 and is incremented by one with each subsequent INSERT.
Queries
Queries are used to look up data from the database tables. The SELECT statement is the main
statement to perform queries.
19
In the first example, we have fetched only the first 4 rows from the BOOKS table.
ij> SELECT * FROM BOOKS OFFSET 4 ROWS;
ID
|AUTHOR_ID |TITLE
----------------------------------------------5
|3
|Jew Suess
6
|4
|Nana
7
|4
|The Belly of Paris
8
|5
|In Cold blood
9
|5
|Breakfast at Tiffany
With the OFFSET cluase, we skip the first four rows and display the rest.
ij> SELECT * FROM BOOKS OFFSET 4 ROWS FETCH NEXT 3 ROWS ONLY;
ID
|AUTHOR_ID |TITLE
----------------------------------------------------------------5
|3
|Jew Suess
6
|4
|Nana
7
|4
|The Belly of Paris
3 rows selected
We can select a portion of the rows using the combination of OFFSET and FETCH clauses.
Derby functions
Derby supports a few useful functions. These built-in functions are expressions in which an
SQL keyword or special operator executes some operation.
ij> SELECT COUNT(ID) FROM AUTHORS;
1
----------5
The COUNT() is an aggregate function that counts the number of rows accessed in an
expression. There are 5 authors in the AUTHORS table.
ij> SELECT MIN(PRICE) AS "PRICE", MAX(PRICE) AS "MAX",
> AVG(PRICE) AS "AVG", SUM(PRICE) AS "SUM" FROM CARS;
PRICE
|MAX
|AVG
|SUM
-----------------------------------------------
20
9000
|350000
|72721
|581769
1 row selected
In the above query we use other four functions. MAX(), MIN(), AVG() and SUM(). The AS
clause gives a label for a column.
ij> VALUES CURRENT_DATE;
1
---------2012-02-15
ij> VALUES CURRENT SCHEMA;
1
-------------------------USER12
VALUES CURRENT_DATE returns the current date and VALUES CURRENT SCHEMA
returns the current schema of the connection.
With the WHERE clause we only select the cars which have a price higher than 40000.
ij> SELECT NAME FROM CARS WHERE NAME LIKE '%en';
NAME
-----------------------------Citroen
Volkswagen
2 rows selected
With a LIKE clause we select specific car names that fit the search pattern. In our case it is
cars that end in 'en' characters.
ij> SELECT * FROM CARS WHERE ID IN (2, 5, 7);
ID
|NAME
|PRICE
-----------------------------------------------------2
|Mercedes
|57127
5
|Bentley
|350000
7
|Hummer
|41400
3 rows selected
21
The IN clause can be used to select rows from a specific range of values. The above SQL
statement returns rows that have IDs equal to 2, 5 and 7.
ij> SELECT * FROM CARS WHERE PRICE BETWEEN 20000 AND 50000;
ID
|NAME
|PRICE
-----------------------------------------------------4
|Volvo
|29000
6
|Citroen
|21000
7
|Hummer
|41400
8
|Volkswagen
|21600
4 rows selected
We select cars that cost in the range 20000 and 50000. For this we use the BETWEEN AND
keywords following the WHERE clause.
Ordering data
Ordering data can be done with the ORDER BY clause.
ij> SELECT * FROM CARS ORDER BY PRICE;
ID
|NAME
|PRICE
-----------------------------------------------------3
|Skoda
|9000
6
|Citroen
|21000
8
|Volkswagen
|21600
4
|Volvo
|29000
7
|Hummer
|41400
1
|Audi
|52642
2
|Mercedes
|57127
5
|Bentley
|350000
8 rows selected
We order the cars by price. The default order type is ascending order.
ij> SELECT * FROM CARS ORDER BY PRICE DESC;
ID
|NAME
|PRICE
-----------------------------------------------------5
|Bentley
|350000
2
|Mercedes
|57127
1
|Audi
|52642
7
|Hummer
|41400
4
|Volvo
|29000
8
|Volkswagen
|21600
6
|Citroen
|21000
3
|Skoda
|9000
22
The UPDATE statement is used to modify data in a database table. The PRICE of a Mercedes
car is set to 58000.
ij> SELECT * FROM CARS WHERE ID=2;
ID
|NAME
|PRICE
-----------------------------------------------------2
|Mercedes
|58000
1 row selected
We insert all the rows from the CARS table into the CARS2 table, thus copying all data.
ij> SELECT * FROM CARS2;
ID
|NAME
|PRICE
-----------------------------------------------------1
|Audi
|52642
2
|Mercedes
|58000
3
|Skoda
|9000
4
|Volvo
|29000
5
|Bentley
|350000
6
|Citroen
|21000
7
|Hummer
|41400
8
|Volkswagen
|21600
8 rows selected
We check the CARS2 table and see that all data was copying OK.
ij> DELETE FROM CARS2 WHERE ID=8;
1 row inserted/updated/deleted
The DELETE FROM statement without the WHERE clause deletes all rows in the table.
ij> DROP TABLE CARS2;
0 rows inserted/updated/deleted
The DROP TABLE statement deletes the table completly from the database.
RENAME statements
The RENAME statement belongs to the data definition language of the SQL.
23
JDBC
JDBC is an API for the Java programming language that defines how a client may access a
database. It provides methods for querying and updating data in a database. JDBC is oriented
towards relational databases. From a technical point of view, the API is as a set of classes in
the java.sql package. To use JDBC with a particular database, we need a JDBC driver for that
database.
java.sql.Connection;
java.sql.DriverManager;
java.sql.SQLException;
java.sql.Statement;
java.util.logging.Level;
java.util.logging.Logger;
24
57127)");
350000)");
21000)");
41400)");
21600)");
con = DriverManager.getConnection(url);
st = con.createStatement();
st.executeUpdate("CREATE TABLE CARS(ID INT PRIMARY KEY,"
+ "NAME VARCHAR(30), PRICE INT)");
st.executeUpdate("INSERT INTO CARS VALUES(1, 'Audi', 52642)");
st.executeUpdate("INSERT INTO CARS VALUES(2, 'Mercedes',
st.executeUpdate("INSERT INTO CARS VALUES(3, 'Skoda', 9000)");
st.executeUpdate("INSERT INTO CARS VALUES(4, 'Volvo', 29000)");
st.executeUpdate("INSERT INTO CARS VALUES(5, 'Bentley',
st.executeUpdate("INSERT INTO CARS VALUES(6, 'Citroen',
st.executeUpdate("INSERT INTO CARS VALUES(7, 'Hummer',
st.executeUpdate("INSERT INTO CARS VALUES(8, 'Volkswagen',
DriverManager.getConnection("jdbc:derby:;shutdown=true");
}
}
The example connects to the Derby in embedded mode. Creates a CARS table and adds 8
rows into it. Finally, it shuts down Derby.
String url = "jdbc:derby:testdb;user=USER12";
25
This is the URL to connect to the testdb database. In the embedded mode and USER12
schema.
System.setProperty("derby.system.home",
"/home/janbodnar/programming/derby/dbs");
A connection to the Derby database is created. When the connection is created, the Derby
database is booted.
st.executeUpdate("CREATE TABLE CARS(ID INT PRIMARY KEY,"
+ "NAME VARCHAR(30), PRICE INT)");
st.executeUpdate("INSERT INTO CARS VALUES(1, 'Audi', 52642)");
...
We execute the SQL statements which create the database and fill it with some data. For
INSERT, UPDATE and DELETE statements and DDL statements like CREATE TABLE we
use the executeUpdate() method.
DriverManager.getConnection("jdbc:derby:;shutdown=true");
We catch the SQLException. We use the Logger class to log the error message.
if (((ex.getErrorCode() == 50000)
&& ("XJ015".equals(ex.getSQLState())))) {
lgr.log(Level.INFO, "Derby shut down normally", ex);
}
When the Derby engine is shut down, an SQLException is thrown. We catch this exception
and log an information message.
} finally {
try {
if (st != null) {
st.close();
}
if (con != null) {
con.close();
}
26
$ javac zetcode/CreateCars.java
$ java -cp .:lib/derby.jar zetcode.CreateCars
Feb 17, 2012 11:34:02 PM zetcode.CreateCars main
INFO: Derby shut down normally
java.sql.SQLException: Derby system shutdown.
...
We compile and run the example. The shut down of Derby will end in an SQLException.
This is a feature of the Derby database.
Retrieving data
Next we will show, how to retrieve data from a database table. We get all data from the
CARS table.
package zetcode;
import java.sql.*;
import java.util.logging.Level;
import java.util.logging.Logger;
public class SelectAllCars {
public static void main(String[] args) {
Connection con = null;
Statement st = null;
ResultSet rs = null;
String url = "jdbc:derby:testdb";
try {
System.setProperty("derby.system.home",
"/home/janbodnar/programming/derby/dbs");
con = DriverManager.getConnection(url);
st = con.createStatement();
rs = st.executeQuery("SELECT * FROM USER12.CARS");
while (rs.next()) {
System.out.print(rs.getInt(1));
System.out.print(" ");
System.out.print(rs.getString(2));
System.out.print(" ");
System.out.println(rs.getString(3));
}
DriverManager.getConnection("jdbc:derby:;shutdown=true");
} catch (SQLException ex) {
Logger lgr = Logger.getLogger(SelectAllCars.class.getName());
if (((ex.getErrorCode() == 50000)
&& ("XJ015".equals(ex.getSQLState())))) {
lgr.log(Level.INFO, "Derby shut down normally", ex);
27
} else {
}
} finally {
try {
if (rs != null) {
rs.close();
}
if (st != null) {
st.close();
}
if (con != null) {
con.close();
}
} catch (SQLException ex) {
Logger lgr =
Logger.getLogger(SelectAllCars.class.getName());
lgr.log(Level.WARNING, ex.getMessage(), ex);
}
}
}
}
We get all the cars from the CARS table and print them to the console.
st = con.createStatement();
rs = st.executeQuery("SELECT * FROM USER12.CARS");
We execute a query that selects all columns from the CARS table. We use the executeQuery()
method. The method executes the given SQL statement, which returns a single ResultSet
object. The ResultSet is the data table returned by the SQL query. Also note that since we
have not specified the user name in the URL, we have to explicitly mention the schema name
in the SQL statement.
while (rs.next()) {
System.out.print(rs.getInt(1));
System.out.print(" ");
System.out.print(rs.getString(2));
System.out.print(" ");
System.out.println(rs.getString(3));
}
The next() method advances the cursor to the next record of the result set. It returns false,
when there are no more rows in the result set. The getInt() and getString() methods retrieve
the value of the designated column in the current row of this ResultSet object; an int/String in
the Java programming language.
$
$
1
2
3
4
javac zetcode/SelectAllCars.java
java -cp .:lib/derby.jar zetcode.SelectAllCars
Audi 52642
Mercedes 57127
Skoda 9000
Volvo 29000
28
5 Bentley 350000
6 Citroen 21000
7 Hummer 41400
8 Volkswagen 21600
Feb 18, 2012 12:33:31 AM zetcode.SelectAllCars main
INFO: Derby shut down normally
java.sql.SQLException: Derby system shutdown.
...
We compile and run the example. We have a list of all cars from the CARS table of the testdb
database.
Properties
It is a common practice to put the configuration data outside the program in a separate file.
This way the programmers are more flexible. We can change the user, a password or a
connection url without needing to recompile the program. It is especially useful in a dynamic
environment, where is a need for a lot of testing, debugging, securing data etc.
In Java, the Properties is a class used often for this. The class is used for easy reading and
saving of key/value properties.
db.url=jdbc:derby:testdb;user=USER12
db.user=USER12
db.passwd=34klq*
db.syshome=/home/janbodnar/programming/derby/dbs
We have a database.properties file, in which we have four key/value pairs. These are
dynamically loaded during the execution of the program.
package zetcode;
import
import
import
import
import
import
import
java.io.FileInputStream;
java.io.FileNotFoundException;
java.io.IOException;
java.sql.*;
java.util.Properties;
java.util.logging.Level;
java.util.logging.Logger;
29
30
} finally {
try {
if (rs != null) {
rs.close();
}
if (pst != null) {
pst.close();
}
if (con != null) {
con.close();
}
} catch (SQLException ex) {
Logger lgr =
Logger.getLogger(PropertiesExample.class.getName());
lgr.log(Level.WARNING, ex.getMessage(), ex);
}
}
}
}
We connect to the testdb and select all authors from the AUTHORS table. The configuration
data for the example is read from the database.properties file.
Properties props = new Properties();
FileInputStream in = null;
try {
in = new FileInputStream("database.properties");
props.load(in);
The Properties class is created. The data is loaded from the file called database.properties,
where we have our configuration data.
String url = props.getProperty("db.url");
String user = props.getProperty("db.user");
String passwd = props.getProperty("db.passwd");
Prepared statements
Now we will concern ourselves with prepared statements. When we write prepared
statements, we use placeholders instead of directly writing the values into the statements.
Prepared statements increase security and performance.
In Java a PreparedStatement is an object which represents a precompiled SQL statement.
package zetcode;
31
import java.sql.*;
import java.util.logging.Level;
import java.util.logging.Logger;
public class Prepared {
public static void main(String[] args) {
Connection con = null;
PreparedStatement pst = null;
ResultSet rs = null;
String url = "jdbc:derby:testdb;user=USER12";
int price = 58000;
int id = 2;
try {
System.setProperty("derby.system.home",
"/home/janbodnar/programming/derby/dbs");
con = DriverManager.getConnection(url);
= ?");
32
These are the values that are going to be set to the prepared statement. These values could
come from a user and everything coming from users should be considered potentionally
dangerous.
pst = con.prepareStatement("UPDATE CARS SET PRICE = ? WHERE ID = ?");
The prepared statement is executed. We use the executeUpdate() method of the statement
object when we don't expect any data to be returned. This is when we create databases or
execute INSERT, UPDATE, DELETE statements.
$ javac zetcode/Prepared.java
$ java -cp .:lib/derby.jar zetcode.Prepared
Feb 18, 2012 11:08:47 AM zetcode.Prepared main
SEVERE: Database 'testdb' shutdown.
java.sql.SQLNonTransientConnectionException: Database 'testdb' shutdown.
...
ij> SELECT * FROM CARS WHERE ID=2;
ID
|NAME
|PRICE
-----------------------------------------------------2
|Mercedes
|58000
1 row selected
We compile the example. Run it. And check the outcome with the ij tool.
33
Column headers
Next we will show, how to print column headers with the data from the database table. We
refer to column names as MetaData. MetaData is data about the core data in the database.
package zetcode;
import
import
import
import
java.sql.*;
java.util.Formatter;
java.util.logging.Level;
java.util.logging.Logger;
34
} finally {
try {
if (rs != null) {
rs.close();
}
if (pst != null) {
pst.close();
}
if (con != null) {
con.close();
}
} catch (SQLException ex) {
Logger lgr =
Logger.getLogger(ColumnHeaders.class.getName());
lgr.log(Level.WARNING, ex.getMessage(), ex);
}
}
}
}
In this program, we select authors from the AUTHORS table and their books from the
BOOKS table. We print the names of the columns returned in the result set. We format the
output. The SQL file to create the tables is located in the first chapter of this tutorial.
String query = "SELECT NAME, TITLE From AUTHORS, "
+ "Books WHERE AUTHORS.ID=BOOKS.AUTHOR_ID";
This is the SQL statement which joins authors with their books.
ResultSetMetaData meta = rs.getMetaData();
To get the column names we need to get the ResultSetMetaData. It is an object that can be
used to get information about the types and properties of the columns in a ResultSet object.
String colname1 = meta.getColumnName(1);
String colname2 = meta.getColumnName(2);
We print the column names to the console. The Formatter object formats the data.
while (rs.next()) {
Formatter fmt2 = new Formatter();
fmt2.format("%-21s", rs.getString(1));
35
System.out.print(fmt2);
System.out.println(rs.getString(2));
}
We print the data to the console. We again use the Formatter object to format the data. The
first column is 21 characters wide and is aligned to the left.
$ javac zetcode/ColumnHeaders.java
$ java -cp .:lib/derby.jar zetcode.ColumnHeaders
NAME
TITLE
Jack London
Call of the Wild
Jack London
Martin Eden
Honore de Balzac
Old Goriot
Honore de Balzac
Cousin Bette
Lion Feuchtwanger
Jew Suess
Emile Zola
Nana
Emile Zola
The Belly of Paris
Truman Capote
In Cold blood
Truman Capote
Breakfast at Tiffany
Feb 18, 2012 12:15:21 PM zetcode.ColumnHeaders main
INFO: Derby shut down normally
java.sql.SQLException: Derby system shutdown.
...
Writing images
Some people prefer to put their images into the database, some prefer to keep them on the file
system for their applications. Technical difficulties arise when we work with millions of
images. Images are binary data. Derby has a special data type to store binary data called
BLOB (Binary Large Object).
We create a new table called IMAGES for this and the following example.
ij> CREATE TABLE IMAGES(ID INT PRIMARY KEY, DATA BLOB);
0 rows inserted/updated/deleted
The DATA column has the BLOB type. There we will insert the encoded binary data.
package zetcode;
import
import
import
import
import
import
java.io.File;
java.io.FileInputStream;
java.io.FileNotFoundException;
java.sql.*;
java.util.logging.Level;
java.util.logging.Logger;
36
}
}
37
In this example, we read a jpg image from the current working directory and insert in into the
IMAGES table.
File img = new File("woman.jpg");
fin = new FileInputStream(img);
We create a File object for the image file. To read bytes from this file, we create a
FileInputStream object.
pst = con.prepareStatement("INSERT INTO IMAGES(ID, DATA) VALUES(1, ?)");
The binary stream is set to the prepared statement. The parameters of the setBinaryStream()
method are the parameter index to bind, the input stream and the number of bytes in the
stream.
pst.executeUpdate();
Reading images
In the previous example, we have inserted an image into the database table. Now we are
going to read the image back from the table.
package zetcode;
import
import
import
import
import
java.io.FileOutputStream;
java.io.IOException;
java.sql.*;
java.util.logging.Level;
java.util.logging.Logger;
38
con = DriverManager.getConnection(url);
String query = "SELECT DATA FROM IMAGES WHERE ID = 1";
pst = con.prepareStatement(query);
ResultSet result = pst.executeQuery();
result.next();
fos = new FileOutputStream("woman2.jpg");
Blob blob = result.getBlob("DATA");
int len = (int) blob.length();
byte[] buf = blob.getBytes(1, len);
fos.write(buf, 0, len);
DriverManager.getConnection("jdbc:derby:;shutdown=true");
} catch (IOException ex) {
Logger lgr = Logger.getLogger(ReadImage.class.getName());
lgr.log(Level.SEVERE, ex.getMessage(), ex);
} catch (SQLException ex) {
Logger lgr = Logger.getLogger(ReadImage.class.getName());
if (((ex.getErrorCode() == 50000)
&& ("XJ015".equals(ex.getSQLState())))) {
lgr.log(Level.INFO, "Derby shut down normally", ex);
} else {
lgr.log(Level.SEVERE, ex.getMessage(), ex);
}
} finally {
try {
if (rs != null) {
rs.close();
}
if (pst != null) {
pst.close();
}
if (con != null) {
con.close();
}
}
}
39
The FileOutputStream object is created to write to a file. It is meant for writing streams of
raw bytes such as image data.
Blob blob = result.getBlob("DATA");
We get the image data from the DATA column by calling the getBlob() method.
int len = (int) blob.length();
We find out the length of the blob data. In other words, we get the number of bytes.
byte[] buf = blob.getBytes(1, len);
The getBytes() method retrieves all bytes of the BLOB object, as an array of bytes.
fos.write(buf, 0, len);
The bytes are written to the output stream. The image is created on the filesystem.
Transaction support
A transaction is an atomic unit of database operations against the data in one or more
databases. The effects of all the SQL statements in a transaction can be either all committed
to the database or all rolled back.
When a connection is created, it is in autocommit mode. This means that each individual
SQL statement is treated as a transaction and is automatically committed right after it is
executed. This is true for all JDBC drivers, including the Derby's one. To start a new
transaction, we turn the autocommit off.
In direct SQL, a transaction is started with BEGIN TRANSACTION statement and ended
with END TRANSACTION/COMMIT statement. In Derby these statements are BEGIN and
COMMIT. However, when working with drivers these statements are omitted. They are
handled by the driver. Exact details are specific to the driver. For example psycopg2 Python
driver starts a transaction after the first SQL statement. If we want the autocommit mode, we
must be set the autocommit property to True. In constrast, JDBC driver is by default in the
autocommit mode. And to start a new transaction, the autocommit must be turned off.
package zetcode;
import
import
import
import
import
import
java.sql.Connection;
java.sql.DriverManager;
java.sql.SQLException;
java.sql.Statement;
java.util.logging.Level;
java.util.logging.Logger;
40
41
}
if (con != null) {
con.close();
}
}
}
In this program, we want to change the name of the author in the first row of the AUTHORS
table. We must also change the books associated with this author. If we change the author and
do not change the author's books, the data is corrupted.
con.setAutoCommit(false);
To work with transactions, we must set the autocommit to false. By default, a database
connection is in autocommit mode. In this mode each statement is committed to the database,
as soon as it is executed. A statement cannot be undone. When the autocommit is turned off,
we commit the changes by calling the commit() or roll it back by calling the rollback()
method.
st.executeUpdate("UPDATE BOOKS SET TITL = 'Anna Karenina' "
+ "WHERE Id = 2");
The third SQL statement has an error. There is no TITL column in the BOOKS table.
con.commit();
try {
con.rollback();
} catch (SQLException ex1) {
lgr.log(Level.WARNING, ex1.getMessage(), ex1);
}
In case of an exception other than the Derby system shutdown, the transaction is rolled back.
No changes are committed to the database.
$ javac zetcode/Transaction.java
$ java -cp .:lib/derby.jar zetcode.Transaction
Feb 18, 2012 3:02:05 PM zetcode.Transaction main
SEVERE: 'TITL' is not a column in table or VTI 'USER12.BOOKS'.
java.sql.SQLSyntaxErrorException: 'TITL' is not a column in table or VTI
'USER12.BOOKS'.
...
ij> SELECT NAME, TITLE FROM AUTHORS, BOOKS WHERE
42
The execution fails with the "'TITL' is not a column in table" message. An exception was
thrown. The transaction was rolled back and no changes took place.
java.sql.Connection;
java.sql.DriverManager;
java.sql.SQLException;
java.sql.Statement;
java.util.logging.Level;
java.util.logging.Logger;
43
} finally {
try {
if (st != null) {
st.close();
}
if (con != null) {
con.close();
}
} catch (SQLException ex) {
Logger lgr =
Logger.getLogger(NonTransaction.class.getName());
lgr.log(Level.WARNING, ex.getMessage(), ex);
}
}
}
}
We have the same example. This time, without the transaction support.
$ javac zetcode/NonTransaction.java
$ java -cp .:lib/derby.jar zetcode.NonTransaction
Feb 18, 2012 3:13:45 PM zetcode.NonTransaction main
SEVERE: 'TITL' is not a column in table or VTI 'USER12.BOOKS'.
java.sql.SQLSyntaxErrorException: 'TITL' is not a column in table or VTI
'USER12.BOOKS'.
...
ij> SELECT NAME, TITLE FROM AUTHORS, BOOKS WHERE
> AUTHORS.ID = BOOKS.AUTHOR_ID;
NAME
|TITLE
---------------------------------------------------------------Leo Tolstoy
|War and Peace
Leo Tolstoy
|Martin Eden
Honore de Balzac
|Old Goriot
Honore de Balzac
|Cousin Bette
Lion Feuchtwanger
|Jew Suess
Emile Zola
|Nana
Emile Zola
|The Belly of Paris
Truman Capote
|In Cold blood
Truman Capote
|Breakfast at Tiffany
9 rows selected
44
An exception is thrown again. Leo Tolstoy did not write Martin Eden. The data is corrupted.
Batch updates
When we need to update data with multiple statements, we can use batch updates. Batch
updates are available for INSERT, UPDATE, DELETE statements as well as for CREATE
TABLE and DROP TABLE statements.
package zetcode;
import java.sql.*;
import java.util.logging.Level;
import java.util.logging.Logger;
public class BatchUpdates {
public static void main(String[] args) {
Connection con = null;
Statement st = null;
ResultSet rs = null;
String url = "jdbc:derby:testdb;user=USER12";
try {
System.setProperty("derby.system.home",
"/home/janbodnar/programming/derby/dbs");
con = DriverManager.getConnection(url);
con.setAutoCommit(false);
st = con.createStatement();
st.addBatch("DELETE
st.addBatch("INSERT
st.addBatch("INSERT
st.addBatch("INSERT
st.addBatch("INSERT
st.addBatch("INSERT
st.addBatch("INSERT
st.addBatch("INSERT
st.addBatch("INSERT
st.addBatch("INSERT
FROM
INTO
INTO
INTO
INTO
INTO
INTO
INTO
INTO
INTO
CARS");
CARS VALUES(1,
CARS VALUES(2,
CARS VALUES(3,
CARS VALUES(4,
CARS VALUES(5,
CARS VALUES(6,
CARS VALUES(7,
CARS VALUES(8,
CARS VALUES(9,
'Audi', 52642)");
'Mercedes', 57127)");
'Skoda', 9000)");
'Volvo', 29000)");
'Bentley', 350000)");
'Citroen', 21000)");
'Hummer', 41400)");
'Volkswagen', 21600)");
'Jaguar', 95000)");
45
This is an example program for a batch update. We delete all rows from the CARS table and
insert nine rows into it.
con.setAutoCommit(false);
FROM
INTO
INTO
INTO
CARS");
CARS VALUES(1, 'Audi', 52642)");
CARS VALUES(2, 'Mercedes', 57127)");
CARS VALUES(3, 'Skoda', 9000)");
After adding all commands, we call the executeBatch() to perform a batch update. The
method returns an array of committed changes.
46
con.commit();
Security
In the next chapter, we will mention security options with Derby.
There are two basic security concepts that we will briefly mention in this chapter. The user
authentication and user authorization. User authentication is verifying user credentials
before giving access to the Derby system. User authorization is a means of giving
permissions to read and/or write to a Derby database.
Furthermore, Derby allows to encrypt database files stored on the disk. Derby network traffic
may be encrypted with SSL/TLS cryptographic protocols.
47
Database encryption
Derby provides a way for us to encrypt the data on disk. The user who boots the database
must provide a boot password. A database can be encrypted at the moment of its creation. It
is also possible to encrypt an existing non-encrypted database. When we encrypt a database
we must also specify a boot password, which is an alpha-numeric string used to generate the
encryption key.
ij> CONNECT 'jdbc:derby:testdb;create=true;dataEncryption=true;
bootPassword=3344kkllqq**';
We can encrypt the database, when we create it. We set the dataEncryption property to true
and provide a boot password. Now every time the database is booted, we must provide the
boot password.
ij> CONNECT 'jdbc:derby:testdb';
ERROR XJ040: Failed to start database 'testdb' with class loader
sun.misc.Launcher$AppClassLoader@360be0, see the next exception for
details.
ERROR XBM06: Startup failed. An encrypted database cannot be accessed
without
the correct boot password.
In the embedded mode, when we connect to the database, we also boot it. The Derby shows
the above error message, when we try to connect to an ecrypted database without the boot
password.
ij> CONNECT 'jdbc:derby:testdb;bootPassword=3344kkllqq**';
ij> SHOW CONNECTIONS;
CONNECTION0* - jdbc:derby:testdb
* = current connection
With the correct boot password, we have successfully connected to the testdb database.
Authentication
Authentication is restricting access to the proper users. Authentication is turned off by default
in Derby.
Derby has three ways to provide authentication.
The official Derby documentation warns that the Derby's built-in authentication mechanism
is suitable only for development and testing purposes. It is strongly recommended that
production systems rely on LDAP or a user-defined class for authentication.
Embedded
Authentication can be set at two levels. At a system level or at a database level.
48
The above two statements enable user authentication for the currently connected database at a
database level. We have created a user with a password and have enabled the
derby.connection.requireAuthentication property.
ij> CONNECT 'jdbc:derby:testdb';
ERROR 08004: Connection authentication failure occurred. Reason: Invalid
authentication..
ij> CONNECT 'jdbc:derby:testdb;user=user12;password=34klq*';
ij> SHOW CONNECTIONS;
CONNECTION0* - jdbc:derby:testdb
* = current connection
After enabling user authentication, we must provide user credentials, when we want to
connect to the testdb database.
Client/Server
In the next examples, we will work with the Derby in the Client/Server mode. We have an
encrypted testdb database.
$ startNetworkServer &
When we connect to the testdb database for the first time, we must provide the boot
password. It is because previously we have encrypted the testdb database.
ij> CONNECT 'jdbc:derby://localhost:1527/dbs/testdb';
ij> SHOW CONNECTIONS;
CONNECTION0* - jdbc:derby://localhost:1527/dbs/testdb
* = current connection
We do not need to boot the database in the Client/Server mode once it is already started.
Unlike in the embedded mode, where each time we connect to the database, we also boot it.
In the next step, we are going to enable the user authentication in the Client/Server mode. For
this, we need to edit the derby.properties file.
$ stopNetworkServer
49
First, we stop the Derby server if it is running. Note that after user authentication was enabled
we need to provide user credentials to stop the server. The stopNetworkServer script takes
-user and -password options.
$ cat dbs/derby.properties
derby.connection.requireAuthentication=true
derby.user.user12=34klq*
derby.authentication.provider=BUILTIN
In the Derby system directory, we modify the derby.properties file. If the file is not present,
we create it. In the property file we enable the authentication and create a user with a
password. We also set the authentication provider to the Derby BUILTIN.
$ startNetworkServer &
We try to connect to the testdb database. Since the Derby server was restarted, we provide the
boot password. However, we see an error message. This is because we have enabled user
authentication. We must also provide user credentials.
ij> CONNECT 'jdbc:derby:testdb;user=user12;password=34klq*;
bootPassword=3344kkllqq**';
With this connection string, we have successfully connected to the testdb database.
User authorization
User authorization in Derby enables to grant and revoke permissions to access a system,
database, object or SQL action. We can set the user authorization properties in Derby as
system-level properties or database-level properties.
Derby has several properties, that affect the user authorization. The
derby.database.defaultConnectionMode property controls the default access mode. If the
property is not set, the property defaults to fullAccess, which is read-write access. The other
two options are noAccess and readOnlyAccess. With the derby.database.fullAccessUsers and
derby.database.readOnlyAccessUsers we control which users can have read-write and which
read-only access to a database. The derby.database.sqlAuthorization property enables SQL
standard authorization. When the derby.database.sqlAuthorization property is set to true,
object owners can use the GRANT and REVOKE SQL statements to set the user permissions
for specific database objects or for specific SQL actions.
50
The privileges that we can grant or revoke are: DELETE, EXECUTE, INSERT, SELECT,
REFERENCES, TRIGGER and UPDATE.
The access mode specified for the derby.database.defaultConnectionMode property overrides
the permissions that are granted by the owner of a database object.
$ cat dbs/derby.properties
derby.connection.requireAuthentication=true
derby.user.user12=34klq*
derby.user.user13=33kl33
derby.user.user14=14kl14
derby.user.user15=35rr++
derby.authentication.provider=BUILTIN
derby.database.defaultConnectionMode=readOnlyAccess
derby.database.fullAccessUsers=user12
We modify the derby.properties file. We add three users. One user, user12 has full access to
the database. The other three have the default, read-only access.
export DERBY_OPTS=-Dderby.system.home=/home/janbodnar/programming/derby/dbs
Note that for the network server to know where the system directory with the derby.property
is, we have set the DERBY_OPTS variable to contain the derby system directory.
$ stopNetworkServer
$ startNetworkServer &
$ java -Dderby.system.home=/home/janbodnar/programming/derby/dbs \
-Dij.protocol=jdbc:derby: -jar $DERBY_HOME/lib/derbyrun.jar ij
We connect to the testdb database with the user13 user. Since we are connecting to the
database for the first time we also boot it. So we need the boot password, because the
database was previously encrypted.
ij> SELECT * FROM USER12.CARS;
ID
|NAME
|PRICE
-----------------------------------------------------1
|Audi
|52642
2
|Mercedes
|57127
3
|Skoda
|9000
4
|Volvo
|29000
5
|Bentley
|350000
6
|Citroen
|21000
7
|Hummer
|41400
8
|Volkswagen
|21600
8 rows selected
The user13 has permissions to see the data from the CARS table located in the USER12
schema.
51
However, trying to modify data in the CARS table leads to an error. Permission is not granted
to perform changes.
ij> DISCONNECT;
ij> CONNECT 'jdbc:derby://localhost/testdb;user=user12;
password=34klq*';
We close the connection and connect as user12. This user was given full access in the
properties file. Even if user12 is the owner of the database and owner of the CARS table, he
cannot modify the table unless given full access with the Derby properties.
ij> INSERT INTO CARS VALUES(9, 'Toyota', 27000);
1 row inserted/updated/deleted
ij> SELECT * FROM CARS WHERE ID = 9;
ID
|NAME
|PRICE
-----------------------------------------------------9
|Toyota
|27000
1 row selected
SQL authorization
The owner of the database or an object like table can further restrict permissions to work with
database objects. We can use GRANT and REVOKE statements to give or withdraw
permissions. The owner of the database, table is the current user, that has created the
database, table. Note that the derby.database.defaultConnectionMode overrides the
permissions given by the GRANT statement. So if a user has readOnlyAccess given by the
default connection mode it cannot modify database objects even if he was given permission
by the GRANT statement.
When the derby.database.sqlAuthorization property is set to true, object owners can use the
GRANT and REVOKE SQL statements to set the user permissions for specific database
objects or for specific SQL actions. Note that setting system-wide property in the
derby.properties file is effective only for new databases. For existing databases, we can only
set database-wide derby.database.sqlAuthorization property. After we set the
derby.database.sqlAuthorization property to true, we cannot set the property back to false.
ij> CALL
SYSCS_UTIL.SYSCS_SET_DATABASE_PROPERTY('derby.database.sqlAuthorization',
'true');
The derby.database.sqlAuthorization property has been set to true. The property is static. We
must reboot the testdb database to make the property work.
ij> CONNECT 'jdbc:derby://localhost/testdb;shutdown=true;
user=user12;password=34klq*';
52
The user12 as the owner of the table has full privileges. The above commands confirm that he
has UPDATE and SELECT privileges on the CARS table.
ij(CONNECTION1)> DISCONNECT;
ij> CONNECT 'jdbc:derby://localhost/testdb;user=user14;
password=14kl14';
ij(CONNECTION1)> SELECT * FROM USER12.CARS;
ERROR 42502: User 'USER14' does not have SELECT permission
on column 'ID' of table 'USER12'.'CARS'.
We disconnect from the database and connect as user14. Trying to execute SELECT
statement leads to an error. The user14 does not have the privileges to SELECT data from the
CARS table.
ij(CONNECTION1)> DISCONNECT;
ij> CONNECT 'jdbc:derby://localhost/testdb;user=user15;
password=35rr++';
ij> SELECT * FROM USER12.CARS;
ID
|NAME
|PRICE
-----------------------------------------------------1
|Audi
|52642
2
|Mercedes
|57127
3
|Skoda
|9000
4
|Volvo
|29000
5
|Bentley
|350000
6
|Citroen
|21000
7
|Hummer
|41400
8
|Volkswagen
|21600
9
|Toyota
|27000
8 rows selected
53
Next we connect as user15. The user can select data from the CARS table.
ij(CONNECTION1)> SELECT * FROM USER12.AUTHORS;
ERROR 42502: User 'USER15' does not have SELECT
permission on column 'ID' of table 'USER12'.'AUTHORS'.
But he cannot select data from the AUTHORS table. Permissions to select data from this
table were not given by the table owner user12.
ij(CONNECTION1)> UPDATE USER12.CARS SET PRICE=27000 WHERE ID=9;
ERROR 25502: An SQL data change is not permitted for a read-only
connection, user or database.
lib/derbynet.jar
First we have to copy the derbyclient.jar, derby.jar and derbynet.jar files into the lib
subdirectory of the Tomcat installation directory.
$ ls webapps
derby.war docs
examples
host-manager
manager
ROOT
Then we have to copy the derby.war file into the webapps subdirectory file of the Tomcat
installation directory. When the Tomcat starts, the file is unpacked and deployed.
$ export JAVA_OPTS=Dderby.system.home=/home/janbodnar/programming/derby/dbs
When we start Derby via Tomcat, the DERBY_OPTS variable is not taken into account. The
derby.system.home must be set prior to starting the Tomcat and Derby servers. We can set
the Derby system directory in the JAVA_OPTS variable.
$ bin/startup.sh
Using CATALINA_BASE:
/home/janbodnar/bin/tomcat
Using CATALINA_HOME:
/home/janbodnar/bin/tomcat
Using CATALINA_TMPDIR: /home/janbodnar/bin/tomcat/temp
54
Using JRE_HOME:
/home/janbodnar/bin/jdk1.6.0_30
Using CLASSPATH:
/home/janbodnar/bin/tomcat/bin/bootstrap.jar:
/home/janbodnar/bin/tomcat/bin/tomcat-juli.jar
We have to navigate to the above mentioned url each time, we start the Tomcat server. To
automatically start Derby, we can add the above line inside the the <servlet> tag of the
web.xml file. The file is located in webapps/derby/WEB-INF directory.
55
Inside the Derby system directory, we have the derby.properties file. In this file, we configure
some options. We set the log severity level to 0 to report all possible problems. This is done
in test environments. We enable authentication. We create four users with corresponding
passwords. Only one of them, user12, has full access priviliges. The others have only
readOnlyAccess.
$ java -Dderby.system.home=/home/janbodnar/programming/derby/dbs \
-Dij.protocol=jdbc:derby: -jar $DERBY_HOME/lib/derbyrun.jar ij
ij version 10.8
ij>
We start the ij command line tool. We will use it to create a database and a table. The Derby
system directory is located at /home/janbodnar/programming/derby/dbs.
ij> CONNECT 'jdbc:derby://localhost:1527/testdb;create=true;
user=user12;password=34klq*';
We create the testdb database and connect to it. We provide user credentials.
ij> run 'cars.sql';
We execute the cars.sql script, which creates a CARS table and fills it with data.
ij> SELECT * FROM CARS;
56
ID
|NAME
|PRICE
-----------------------------------------------------1
|Audi
|52642
2
|Mercedes
|57127
3
|Skoda
|9000
4
|Volvo
|29000
5
|Bentley
|350000
6
|Citroen
|21000
7
|Hummer
|41400
8
|Volkswagen
|21600
8 rows selected
This is our CARS table. Next we will create a Java servlet, which will display these values in
a web browser.
The project
We will create a simple web application, that will connect to the Derby database. One Java
servlet will connect to the Derby and retrieve all data from the CARS table.
$ tree
.
build.xml
context.xml
lib
servlet-api.jar
src
zetcode
SelectAllCars.java
web.xml
3 directories, 5 files
In our current working directory, we have an Ant build.xml file, the context.xml
configuration file, the web.xml deployment descriptor file, and the src and lib subdirectories.
The build.xml file is the Ant build file, which describes the tasks to build, deploy or clean the
project. The web.xml defines the structure of the web application. In the lib directory we have
the servlet-api.jar file, which is needed to compile the source file. (It can be found in the lib
subdirectory of the Tomcat installation directory.) In the src directory, we have the Java
source file.
57
</description>
<display-name>Derby, Tomcat</display-name>
<servlet>
<servlet-name>SelectAllCars</servlet-name>
<servlet-class>zetcode.SelectAllCars</servlet-class>
</servlet>
<servlet-mapping>
<servlet-name>SelectAllCars</servlet-name>
<url-pattern>/SelectAllCars</url-pattern>
</servlet-mapping>
</web-app>
These are the contents of the the web.xml file. In this file, we register the SelectAllCars
servlet.
<?xml version="1.0" encoding="UTF-8"?>
<Context>
<Resource name="jdbc/testdb"
auth="Container"
type="javax.sql.DataSource"
username="user12"
password="34klq*"
driverClassName="org.apache.derby.jdbc.ClientDriver"
url="jdbc:derby://localhost:1527/testdb"
maxActive="10"
maxIdle="4"/>
</Context>
In the context.xml file, we define the JDBC datasource. The context.xml file can be defined
for all web applications or for a single application. The latter is our case.
We will show the Ant build file which will be used to build and deploy our tiny application.
<?xml version="1.0" ?>
<project name="allcars" default="deploy">
<property name="src.dir" value="src"/>
<property name="build.dir" value="build"/>
<property name="dist.dir" value="dist"/>
<property name="deploy.dir"
value="/home/janbodnar/bin/tomcat/webapps"/>
<echo>${ant.project.name}</echo>
<target name="init">
<mkdir dir="${build.dir}/classes" />
<mkdir dir="${dist.dir}"/>
<echo>Directories created.</echo>
</target>
<target name="compile" depends="init">
<javac srcdir="${src.dir}" destdir="${build.dir}/classes"
58
includeantruntime="false">
<classpath path="lib/servlet-api.jar"/>
</javac>
<echo>Source files compiled.</echo>
</target>
<target name="archive" depends="compile">
<war destfile="${dist.dir}/${ant.project.name}.war"
webxml="web.xml">
<classes dir="${build.dir}/classes"/>
<metainf file="context.xml"/>
</war>
<echo>Archive created.</echo>
</target>
<target name="deploy" depends="archive">
<copy file="${dist.dir}/${ant.project.name}.war" todir="$
{deploy.dir}"/>
<echo>Project deployed.</echo>
</target>
<target name="clean">
<delete dir="${dist.dir}"/>
<delete dir="${build.dir}"/>
<echo>Project cleaned.</echo>
</target>
</project>
The build file comprises five tasks. The initialization task will create the necessary
directories. The compile task will compile the source code. The archive task will create a web
archive. The deploy task will deploy the archive to the Tomcat server. Finally, the clean task
will do the cleaning.
java.io.IOException;
java.io.PrintWriter;
java.sql.Connection;
java.sql.ResultSet;
java.sql.SQLException;
java.sql.Statement;
java.util.logging.Level;
java.util.logging.Logger;
javax.naming.Context;
javax.naming.InitialContext;
javax.naming.NamingException;
javax.servlet.ServletException;
javax.servlet.http.HttpServlet;
javax.servlet.http.HttpServletRequest;
javax.servlet.http.HttpServletResponse;
javax.sql.DataSource;
59
60
In the above servlet, we connect to the Derby testdb database and fetch all rows from the
CARS table.
Context ctx = new InitialContext();
DataSource ds = (DataSource) ctx.lookup("java:comp/env/jdbc/testdb");
We use the JNDI naming lookup to obtain the datasource. From the datasource, we create the
connection object.
rs = st.executeQuery("SELECT * FROM CARS");
while (rs.next()) {
out.print(rs.getInt(1));
out.print(" ");
out.print(rs.getString(2));
out.print(" ");
out.print(rs.getString(3));
out.print("<br>");
}
We use the SQL statement to retrieve all data from the CARS table. We print the data from
the result set object.
$ ant
Buildfile: /home/janbodnar/programming/derby/servlet/build.xml
[echo] allcars
init:
[mkdir] Created dir:
/home/janbodnar/programming/derby/servlet/build/classes
[mkdir] Created dir: /home/janbodnar/programming/derby/servlet/dist
[echo] Directories created.
compile:
61
62
Creating a database
First thing to do is to create a new database. We will create testdb database.
63
Database connection
After the database is created, we create a database connection.
64
Netbeans uses these icons for connection objects. The first icon is for disconnected database
connection object, the second for an established database connection object.
The above connection is a Java DB connection created with a Java DB server driver. Note
that when we have created the testdb database, a Java DB server was automatically started
and a connection created. New database connections can be created by right clicking on the
Java DB driver and choosing the Connect Using option.
We are going to create an embedded Java DB database connection. Before creating the
connection, we need to stop the Java DB server, if it is running. Java DB database cannot be
booted by a Java DB server and connected to by an embedded driver at the same time. Note
that we did not have to start the server explicitly. The server could be started behind the
scenes. For example by connecting to the Java DB server connection object or creating a new
database.
65
After succesfully creating the embedded database connection, we see the above icon in the
Netbeans Services window.
Creating a table
The database connection is created. The next thing we do is to create a new database table.
We will create a simple table called FRIENDS with two columns. Id and Name. The Id will
be INTEGER and Name VARCHAR(30).
We expand the Database Connection node and further expand the USER12 schema. We right
click on the Table icon and choose Create Table option.
66
67
The above picture shows the icons to work with the data in the table. The first two icons are
used to insert a new record and delete a selected record, respectively. We can delete multiple
records if we select more rows with a mouse pointer and pressing the Shift key
simultaneously. If the data was modified, the Commit Record(s) icon is enabled. The data is
saved only after we commit it with this action. Except for the SQL statements, we can modify
data with GUI tools. By double clicking on a record a line widget appears. In this widget, we
can change the data. The changes are saved by clicking on the Commit Record(s) action.
In the chapter, we have worked with Java DB inside the Netbeans IDE.
68