Module 2/ unit 4 Poge 178 Module 2 / Unit 4 Using Databases Objectives (On completion of this unit, you will be able to Describe databases and explain the purpose of a database © List the relational methods used by structured databases. List the ways that users and applications can interface with databases. 2 Distinguish application architecture models. Syllabus Objectives and Content Examples This unit covers the following exam domain objectives and content examples: 5 3.4 Explain methods of application architecture and delivery models. Application architecture models (One tier, Two tier, Three tier, retier) 5 5.1 Explain database concepts and the purpose of a database Usage of database (Create, Import/input, Query, Reports) « Flat file vs. database (Multiple concurrent users, Scalability, Speed, Variety of data) + Records + Storage (Data persistence) © 5.2 Compare and contrast various database structures. Structured vs. semi-structured vs. non-structured « Relational databases (Schema, Tables, Rows/records, Fields/columns, Primary key, Foreign key, Constraints) - Non-relational databases (Key/value databases, Document databases) 5.3 Summarize methods used to interface with databases. Relational methods (Data manipulation [Select, Insert, Delete, Update], Data definition (Create, Alter, Drop, Permissions}) - Database access methods (Direct/manual access, Programmatic access User interface/utility access, Query/report builders) - Exportfimport (Database dump, Backup)Database Concepts ee Done Databases are used everywhere in today’s connected world, Every time you make an online purchase with an e-commerce retailer, many databases are accessed by a variety of different applications in order to facilitate your purchase. A database is an organized collection of information, The information is stored in a structured manner for easier access. Typically, a database consists of tables of information, organized into columns and rows, Each row represents a separate record in the database, while each column represents a single field within a record, James Pengelly This user record is backed by a database. Each form field you can see shows and updates a ‘column in a database table while this record is @ row inthe table. Database Usage A database is used both to store information securely and to report on the information it contains. Consequently, database usage involves the following processes and tools: = Creation—this step involves defining what information the database will store, where it will be hosted, and how it will be accessed by clients. = Importinput—once the database has been created, it must be populated with data records. Records can either be input and updated manually, usually using some type of form, or data might be imported from another source, or both Page 179Module 2/ unit 4 Poge 180 = Storage (data persistence} databases are often used with applications. While an application processes variables and other temporary data internally, this information is lost when the application is terminated. A database represents a way for an application to store data persistently and securely, = Queries—it is possible in theory to read the information in each table manually, but in order to view information efficiently, a query is used to extract it. A query allows the user to specify criteria to match values in one or more fields and choose which fields to display in the results so that only information of interest is selected, m= Reports—a query might return a large number of rows and be just as difficult to read as a table, A report is a means of formatting and summarizing the records returned by a query so that the information is easy to read and interpret. Flat File Systems (One common question asked when users are considering a database is, "Why can't | just use a spreadsheet such as Excel?" I's a fair question because Excel enables you to store your information in sheets, which are broadly analogous to tables, with rows and columns of data. This is an example of a flat file data storage and access system rather than a database. Spreadsheets are not the only kind of flat file data store, Another example is a plain text file with delimiters for each column. A Comma Separated Values (CSV) file uses commas to identify the end of a column and a line feed for each row. Comma Separated Values file—the column headers are shown atthe fop with commas separating ‘each. Every line in the file corresponds to a row of data. While flat fle systems are easy to create they have many drawbacks compared to database systems. Databases versus Flat File Systems AA flat file system might be useful for tasks such as simple order or sales databases used by a single person or small workgroup. A flat file is also a ‘good way of exporting and importing information between systems, Dedicated database software has many advantages over flat files though. m= Databases can enforce data types for each column and validate information entered as fields and records. Spreadsheets can mimic some of this functionality but not as robustly. Databases consequently support a wider variety of data formats.m= Databases can manage multiple tables and link the fields in different tables Using Databases to create complex schemas. In a flat fie, all the information is stored within a single table = Databases can support tens, hundreds or thousands, or even millions of users concurrently. A single file-based data storage solution does not offer high enough speed for the volumes of transactions (adding and updating records) on enterprise-level systems. becomes locked to other users. They may be able to view the file but cannot change it (read-only). A database system supports concurrent users without locking the whole database. A particular record may be locked during a data entry transaction though. m= Databases are also more scalable. Scalability means being able to expand usage without increasing costs at the same rate. For example, in a non- scalable system, doubling the number of users would also double the costs of the system. Database architecture means that extra capacity can be added later with much less investment. = Databases provide access controls to protect information from unauthorized disclosure and backup/replication tools to ensure that data can be recovered within seconds of it being committed, Database Structures There are different types of databases and data stores with different levels of structure, Relational Databases A relational database is the type we have been describing so far. A relational database is a highly structured type of database. Information is organized in tables (known as relations). A table is defined with a number of fields, represented by the table columns. Each field can be a particular data type. Each row entered into the table represents a data record. Typically, Relational Database Management Systems (RDBMS) use Structured Query Language (SQL) to maintain and query data in the database. Examples of RDBMS include: m= Microsoft SQL Server. = Oracle Database, = MySQL. = Microsoft Office Access. Page 181Module 2/ unit 4 Poge 162 Primary Key and Foreign Key Attempting to store a complex set of data within a single table is impractical For example, if you want to record who has borrowed books from a library, if you have a single LibraryLoans table, you have to duplicate information about the customer each time a loan record is created. Itis likely that mistakes could be made inputting this information and the customer's details may change. resulting in inconsistencies in your data records, In a relational database, you can have multiple linked tables. If you design the database schema with one Customer table and one Loan table, you can link a single customer record to multiple loan records. If a customer record has to be updated, you can do that once in the Customer table rather than editing lots of records in a monolithic LibraryLoans table. For this to work, in any given table, each record has to be unique in at least one way. This is usually accomplished by designating one column as a primary key. Each row in the table must have a unique value in the primary key field. This primary key is used to define the relationship between one table and another table in the database. When a primary key in one table is referenced in another table, then in the secondary table, that column is referred to as a foreign key. The structure of the database in terms of the fields defined in each table and the relations between primary and foreign keys is referred to as the schema. Relational Database Example To give you an idea about a relational database, think again about a database supporting a lending library. Your database is designed to store and allow retrieval of data concerned with the management of book lending. This database might include a table that stored customer information, another that contained book title information, and finally a table that stored information about the actual lending Relational database table model.You can see that the customer information and book information that appears Using Databases in the Lending table is drawn from the Customer and Book tables respectively. Specifically, the Customer ID is drawn from the Customer table and the Title is drawn from the Book table, The advantage of this approach is that if you must update customer information, you only need to do so in one place—the Customer table. A query can be used to reconstruct the information. For example, if you want to check the customer associated with a particular lending record, your query ‘would select the record from the Lending table then use the join between the Lending and Customer tables to show the values of the name and address fields from the related Customer table record. Constraints ‘One of the functions of an RDBMS is to address the concept of Garbage In, Garbage Out (GIGO). It is very important that the values entered into fields conform are consistent with what information the field is supposed to store. When defining the properties of each field, as well as enforcing a data type, you can impose certain constraints on the values that can be input into each field. A primary key is an example of a constraint. The value entered or changed in a primary key field in any given record must not be the same as any other existing record. Other types of constraints might perform validation on the data that you can enter. For example, you could add constraints to prevent a field from being left blank, to define the format of a telephone number, or to check that the value entered to choose an order delivery date is not in the past. Another example of a constraint is to enter a default value in a field if the user makes no selection. g Constraints can be applied at different levels. As well as applying rules to fields, they can be used at the table and schema levels too. Semi-structured and Unstructured Databases When you store your information in a relational database, itis stored in a structured way. This structure enables you to more easily access the stored information and gives you flexibility over exactly what you access. For example, you can access all fields or only certain fields, Each field has a defined data type, meaning that software that understands the database language (SQL), can parse (interpret) the content of a field easily Unstructured data, on the other hand, provides no rigid formatting of the data Images and text files, Word documents and PowerPoint presentations are examples of unstructured data. Unstructured data is typically much easier to create than structured data, Documents can be added to a store simply and the data store can support a much larger variety of data types than a relational database can. Page 183Module 2/ unit 4 Poge 164 Sitting somewhere between these two is semi-structured data. Strictly speaking, the data lacks the structure of formal database architecture, But in addition to the raw unstructured data, there is associated information called metadata that helps identify the data Document and Key/Value Pair Databases ‘A document database is an example of a semi-structured database. Rather than define tables and fields, the database grows by adding documents to it. The documents can use the same structure or be of different types. The database's query engine must be designed to parse each document type and extract information from it. Documents would very commonly use markup language such as XML (eXtensible Markup Language) to provide structure. Markup languages such as XML were discussed in Unit 2.2. A key/value pair database is a means of storing the properties of objects without predetermining the fields used to define an object. A key/value pair table looks like the following Key Value user01_surname Warren user01_firstname Andy user01_age 2 user02_surname Pengelly user02_town Bigtown user_01_marketingconsent TRUE user_02_marketingconsent FALSE As you can see, not all properties have to be defined for each object. One widely used key/value format is JavaScript Object Notation (JSON). For example, "user01" could be expressed as the following JSON string { "user01_surname" : "Warren", “user01_firstname" andy", “user01_age" : 27, "user01_marketingconsent' TRUE } Document databases and key/value pair databases are non-relational because there are no formal structures to link the different data objects and files. This does not mean that relationships between the data items cannot be found though. Non-relational database systems use searches and queries to ‘summarize and correlate data points. v Database engines dealing with a mixture of structured, unstructured, and semi-structured data are often referred to as NoSQL databases, which can stand either for "No SQL” or for "Not Only SQL.”Relational Methods Database interfaces are the processes used to add/update information to and extract (or view) information from the database. In an RDBMS, the use of Structured Query Language (SQL) relational methods is critical to creating and updating the database. These relational methods can be split into two types; those that define the database structure and those that manipulate information in the database. Data Definition Methods Data Definition Language (DDL) commands refer to SQL commands that add to or modify the structure of the database. Some examples of data definition ‘commands are: m= CREATE—this command can be used to add a new database on the RDBMS server (CREATE DATABASE) or to add a new table within an existing database (CREATE TABLE). The primary key and foreign key can be specified as part of the table definition. m= ALTER TABLE—this allows you to add, remove (drop), and modify table columns (fields), change a primary key and/or foreign key, and configure other constraints. There is also an ALTER DATABASE command, used for modifying properties of the whole database, such as its character set. = DROP—this is the command used to delete a table (DROP TABLE) or database (DROP DATABASE). Obviously, this also deletes any records and data stored in the object. ™ CREATE INDEX—specifying that a column (or combination of columns) is indexed speeds up queries on that column. The tradeoff is that updates are slowed down slightly, or quite a lot if the column is not suitable for indexing ‘The DROP INDEX command can be used to remove an index There are also SQL commands allowing permissions (access controls) to be configured. These are discussed below. Data Manipulation Methods Data Manipulation Language (DML) commands allow you to insert or update records and extract information from records for viewing (a query) INSERT INTO TableName—adds a new row in a table in the database. = UPDATE TableName—changes the value of one or more table columns. This can be used with a WHERE statement to filter the records that will be updated. If no WHERE statement is specified, the command applies to all the records in the table. = DELETE FROM TableName—deletes records from the table. As with UPDATE, this will delete all records unless a WHERE statement is specified. Page 185Module 2/ unit 4 Page 186 m= SELECT—enables you to define a query to retrieve data from a database, The following examples show some of the ways that a SELECT statement can be used to build a query: SELECT * FROM Customers; ..enables you to select all data in the specified "Customers" table. SELECT Name, Town FROM Customers; retrieves the values in the Name and Town fields for all records in the Customers table. SELECT * FROM Customers WHERE Town='Slough' ; retrieves all records from the Customers table where the value in the Town field is equal to "Slough, SELECT * FROM Customers WHERE Town='Slough' ORDER BY Name; as above but also sorts the results in alphabetical order by the contents of the Name field. Permissions SQL supports a secure access control system where specific user accounts can be granted rights over different objects in the database (tables, columns, and views for instance) and the database itself. When an account creates an object, it becomes the owner of that object, with complete control over it. The owner cannot be denied permission over the object. The owner can be changed however, using the ALTER AUTHORIZATION statement Other accounts can be granted specific rights over an object using the statement GRANT permission TO user For example, the following statement grants use of the SELECT statement to the user “james.” GRANT SELECT ON Customers TO james Rights can also be specifically denied a permission using: DENY permission TO user DENY overrides and GRANT permission, but cannot affect the owner, as. mentioned above, Permission statements can be considered as Data Defi (DDL) commands, like CREATE and DROP. jon Languageg ‘SQL also supports a CONTROL permission where the user has all rights to the object by default but can subsequently be denied some rights. Database Access Methods Database access methods are the processes by which a user might run SQL. ‘commands on the database server or update or extract information using a form or application that encapsulates the SQL commands as graphical controls or tools. Direct/Manual Access ‘Administrators might use an administrative tool, such as phpMyAdmin, to connect and sign in to an RDBMS database. Once they have connected, they can run SQL commands to create new databases on the system and interact with stored data. This can be described as direct or manual access. Query/Report Builder There are many users who may need to interact closely with the database but do not want to learn SQL syntax. A query or report builder provides a GUI for users to select actions to perform on the database and converts those selections to the SQL statements that will be executed Programmatic Access A software application can interact with the database either using SQL commands or using SQL commands stored as procedures in the database. Most programming languages include libraries to provide default code for connecting to a database and executing queries. User Interface/Utility Access An application might use a database in the background without the user really being aware of its presence. Alternatively, the application might provide a specific Graphical User Interface (GUI) or form to allow ordinary users to add and search records. Backups and Data Export As with any type of data, itis vital to make secure backups of databases. Most RDBMS provide stored procedures that invoke the BACKUP and RESTORE commands at a database or table level. Poge 187,Module 2/ unit 4 Poge 188 Backing up databases is complicated by the problem of gaining exclusive access to perform a consistent copy. SQL engines have tools to facilitate backup access to an online database. See Unit 4.4 for more information about backup issues. It may also be necessary to export data from the database for use in another database or in another type of program, such as a spreadsheet. A dump is a copy of the database or table schema along with the records expressed as SQL statements. These SQL statements can be executed on another database to import the information. Most database engines support exporting data in tables to other file formats, such as Comma Separated Values (.CSV) or native MS Excel (.XLS). Application Architecture Models A database application can be designed for any sort of business function Customer Relationship Management (CRM) and accounting are typical ‘examples. If the application front-end and processing logic and the database ‘engine are all hosted on the same computer, the application architecture can be described as one-tier or standalone, A two-tier client-server application separates the database engine, or back- ‘end or data layer, from the presentation layer and the application layer, or business logic. The application and presentation layers are part of the client application. The database engine will run on one server (or more likely a cluster of servers), while the presentation and application layers run on the client, Ina three-tier application, the presentation and application layers are also split, The presentation layer provides the client front-end and user interface and runs on the client machine. The application layer runs on a server or server cluster that the client connects to. When the client makes a request, it is checked by the application layer, and if it conforms to whatever access rules have been set up, the application layer executes the query on the data layer which resides on a third tier and returns the result to the client. The client should have no direct communications with the data tier. v In the terminology, a "layer" is a separate logical function, while a “tier” is an independent or compartmentalized processing function. While we describe tiers as "separate servers,” they do not necessarily have to be physically separated machines. They could be implemented as virtual machines for instance. ‘An n-tier application architecture can be used to mean either a two-tier or three-tier application, but another use is an application with a more complex architecture stil. For example, the application may use separate access control or monitoring services.