Introducing Postgres Plus Advanced Server 9.4
4 likes•3,747 views
The document provides an overview of Postgres Plus Advanced Server 9.4, highlighting its features such as resource management, enhanced SQL capabilities, and various tools for database management. It emphasizes innovation in database functionality, improved performance, and security enhancements, including SQL/Protect for preventing injection attacks. Additionally, it discusses partitioning techniques and provides guidelines for when to use them, along with use cases tailored for different customer needs.
Introducing Postgres Plus Advanced Server 9.4
- 1. © 2015 EnterpriseDB Corporation. All rights reserved. 1 Introducing Postgres Plus Advanced Server 9.4 To listen to the recorded presentation please visit EnterpriseDB > Resources > Webcasts > Ondemand Webcasts
- 2. © 2015 EnterpriseDB Corporation. All rights reserved. 2 • EnterpriseDB and Postgres Plus Overview • Postgres Plus Advanced Server 9.4 − Resource Management − Partitioning − Enhanced SQL (Cube/Rollup/Grouping Sets, Connect_By_Root) − UTL_HTTP − SQL/Protect & MTK Enhancements Agenda
- 3. © 2015 EnterpriseDB Corporation. All rights reserved. 3 POSTGRES innovation ENTERPRISE reliability 24/7 support Services & training Enterprise-class features, tools & compatibility Indemnification Product road-map Control Thousands of developers Fast development cycles Low cost No vendor lock-in Advanced features Enabling commercial adoption of Postgres
- 4. © 2015 EnterpriseDB Corporation. All rights reserved. 4 The Database PostgreSQL / Postgres Plus Advanced Server Tools xDB Replication Server Failover Manager Management and Monitoring Back and Recovery Tool Postgres Enterprise Management (PEM) Stack Builder & Update Monitor Cloud Enablement Cloud Database for Amazon Web Services Open Stack Support* Core OR PostgreSQL Postgres Plus Advanced Server Extension Components (PostGIS, pgPool, pgBouncer, SQL/Protect, PL/Perl Python TCL, FDW) Connectors (.Net, JDBC, ODBC,OCL) Utilities (EDBLoader, EDBPlus, ECPGPlus) Migration Tool Kit * Roadmap EnterpriseDB Product Overview APIs to enable the Platform as a Service*
- 5. © 2015 EnterpriseDB Corporation. All rights reserved. 5 EDB is a Gartner Magic Quadrant Leader
- 6. © 2015 EnterpriseDB Corporation. All rights reserved. 6 from PostgreSQL core from EDB Development • 64 bit LOBs up to 4TB in size • Custom background workers • Writable Foreign Data Wrappers v9.1 EDB contributions to PostgreSQL core • No restore In-place version upgrades v9.2 v9.3 v9.0 • Materialized Views • Deferrable unique constraints and Exclusion constraints • Streaming replication • Windows 64 bit Support • Hot standby • Synchronous replication • Serializable Snapshot Isolation • In-memory (unlogged) tables • Writeable Common Table Expressions (WITH) • Cascaded streaming replication • JSON support, Range Types • VARRAY support • SQL Profiler • Index Advisor • Parallel Bulk Data Load • Row Level Security • Declarative Partitioning syntax • Table() function support for nested tables • INSERT APPEND hint • xDB Multi-master replication • Expanded Object Type support • Partition Read Improvements over 75x • Support for 1000s of Partitions • Partition write improvements over 400x • MySQL Foreign Data Wrappers for SQL/MED Postgres Plus Advanced Server Key Feature Development • Index-only scans (covering indexes) • Linear read scalability to 64 cores v9.4 • pg_prewarm • ALTER SYSTEM • Concurrently updatable Materialized Views • Mongo FDW & MySQL FDW • Logical Decoding for Scalability • JSONB Data Type • JSONB Indexing • Expanded JSON functions • Delayed Application of Replication • 3x Faster GIN indexes • Support for Linux Huge Pages • CPU & I/O Resource Management • SQL Aggregation with CUBE, ROLLUP and GROUPING SETS • Comprehensive UTL_HTTP Package • Hash Partitioned Tables • Connect_By_Ro ot Operator for hierarchical queries • SQL/Protect Logging to DB Table • EDB*Loader Improved Error handling
- 7. © 2015 EnterpriseDB Corporation. All rights reserved. 7 CPU & I/O Resource Management Hash Partitioned Tables SQL Aggregation with CUBE, ROLLUP and GROUPING SETS Comprehensive UTL_HTTP Package Connect_By_Root Operator ICU Collation EDB*Loader Improvements SQL/Protect Logging to DB Table Migration Toolkit Enhancements Postgres Plus Advanced Server Postgres Community Feature Highlights for Postgres Plus Advanced Server (PPAS) 9.4 Many new features including: Logical Change Set Extraction JSONB Data Type Time Delayed Standby ALTER SYSTEM pg_prewarm() Materialized View Refresh Concurrently Ordered Set Aggregates and more… http://www.depesz.com/ is a good reference site https://commitfest.postgresql.org/ is the global community site for patches review
- 8. © 2015 EnterpriseDB Corporation. All rights reserved. 8 Postgres Plus Advanced Server Resource Manager (CPU & I/O) Reporting Transactions 80% 20% Run Mixed Workloads More Efficiently with PPAS 9.4 Resource Management • DBA assigns CPU & I/O to job groups • Allocates and prioritizes consumption of resources • Low priority jobs don’t hurt high priority jobs
- 9. © 2015 EnterpriseDB Corporation. All rights reserved. 9 • Create Resource Groups and assign the CPU Rate Limit • Use these resource groups during a psql session Run Mixed Workloads More Efficiently with PPAS 9.4 Resource Management - Statements executed will be limited in CPU or writing to shared_buffers per resource group. - Individual limits are computed based on recent CPU / shared_buffer usage. - Processes sleep when necessary & avoid sleep when holding critical locks. - The limits are adjusted regularly based on current usage. - If multiple processes in the same group are being executed, aggregate usage will be limited CREATE RESOURCE GROUP resgrp_a; CREATE RESOURCE GROUP resgrp_b; ALTER RESOURCE GROUP resgrp_a SET cpu_rate_limit = .25; ALTER RESOURCE GROUP resgrp_a SET dirty_rate_limit = 12288; SET edb_resource_group TO res_grp_a;
- 10. © 2015 EnterpriseDB Corporation. All rights reserved. 10 Chapter 13 of EDB Oracle Compatibility Guide for details on usage One logically large table is broken into smaller physical pieces. • Worthwhile when a table would otherwise be very large. • Exact point to see benefits will depend on the application. When should I Partition? • Good rule of thumb is that the size of the table should exceed the physical memory of the database server. • When most accessed rows are in a single partition or a small number of partitions. • When there is a lot of concurrent inserts or updates (Hash partitioning may benefit) • When a query or update accesses a large percentage of a single partition. • For Bulk Loads / Unloads (ALTER TABLE faster than bulk load, avoids VACUUM overhead from DELETE). • When actively archiving seldom-used data to less expensive storage. Support Larger Tables with Partitioning
- 11. © 2015 EnterpriseDB Corporation. All rights reserved. 11 Use PPAS Syntax To Minimize Partitioning Errors and Complexity CREATE TABLE sales ( dept_no number, part_no varchar2, country varchar2(20), date date, amount number ) PARTITION BY RANGE(date) ( PARTITION q1_2014 VALUES LESS THAN('2014- Apr-01'), PARTITION q2_2014 VALUES LESS THAN('2014- Jul-01'), PARTITION q3_2014 VALUES LESS THAN('2014- Oct-01'), PARTITION q4_2014 VALUES LESS THAN('2015- Jan-01') ); CREATE INDEX sales_date on sales(date); Simple Declarative PARTITION BY and SUBPARTITION BY Single Index command • More SQL syntax provide sophisticated data management techniques − ADD / DROP to augment the partitions − SPLIT to divide the data − EXCHANGE to swap in a new partition − TRUNCATE to remove all data but leave structure in tact − MOVE to shift data to a different tablespace • PPAS Improved performance with Fast Pruning and Constraint Exclusion support • System Catalog Views for Partitions − ALL_PART_TABLES − ALL_TAB_PARTITIONS, ALL_TAB_SUBPARTITIONS − ALL_PART_KEY_COLUMNS, ALL_SUBPART_KEY_COLUMNS
- 12. © 2015 EnterpriseDB Corporation. All rights reserved. 12 List, Range or Hash Partition Rules • Provide the constraints to define where data is stored • For PPAS, also used to support Fast Partition Pruning • Consider how data stored will be queried, include often-queried columns in partitioning rules. • List – Single partitioning key column; based on exact value • Range – One of more partitioning key columns; based on values between two extremes • Hash (New 9.4) – Data divided amongst equal sized partitions based on hash value. * Internal tests have shown hash partitioning can improve performance when many hundred concurrent connections insert/update to the same table* PPAS 9.4 Supports Various Partitioning Rules
- 13. © 2015 EnterpriseDB Corporation. All rights reserved. 13 Advanced Server’s Query Planner uses two optimization techniques to compute an efficient plan: • Constraint exclusion (constraint_exclusion = partition or on) − Provided by PSQL − SELECT w/ WHERE: Query Planner must examine the CHECK constraints defined for each partition before deciding which partition to send query fragments to. • Fast pruning (edb_partition_pruning = on) − Occurs earlier in query planner process. Understands the relationship between partitions in an Oracle style partitioned table. − SELECT w/ WHERE: Query Planner can reason that only a certain partition holds the values without examining the constraints defined for each partition. − Can be used for LIST or single value RANGE Partitions (not usable on subpartitioned tables or multi-value range partitioned tables) − Works with >, >=, =, <=, <, AND, BETWEEN operators in the WHERE Clause PPAS Improves Partitioning Performance
- 14. © 2015 EnterpriseDB Corporation. All rights reserved. 14 • Aggregation Functions (CUBE / ROLLUP / GROUPING SETS) − From the edb-sample.sql, see dept, emp and jobhist tables to aggregate employees based on their locations, departments and jobs. − Chapter 2.2.6 of EDB Database Compatibility Guide for Oracle for more details on GROUP BY ROLLUP / CUBE / GROUPING SETS and the subtotals, groupings and result sets generated. − Your own data for analysis might be involve sales or support related information. You could report on revenues or tickets grouped by geographic regions, assigned sales or support engineer, product utilized, etc. • Connect_By_Root − Support additional hierarchal query needs. With our edb-sample data, we can report on various levels of manager employee hierarchies. − Chapter 2.2.5.6 of EDB Oracle Compatibility Guide for more details on sample usage − Your own data for reporting might look at hierarchies of products or modules Enhanced SQL for Aggregation and Connect_By_Root Syntax
- 15. © 2015 EnterpriseDB Corporation. All rights reserved. 15 • GROUP BY ROLLUP produces subtotals for each hierarchal group based on a left to right listing of items in the expression SELECT loc, dname, job, COUNT(*) AS "employees" FROM emp e, dept d WHERE e.deptno = d.deptno GROUP BY ROLLUP (loc, dname, job) ORDER BY 1, 2, 3; Multidimensional Analysis - Rollup
- 16. © 2015 EnterpriseDB Corporation. All rights reserved. 16 • GROUP BY CUBE produces groupings and subtotals for every permutation of items in the expression SELECT loc, dname, job, COUNT(*) AS "employees" FROM emp e, dept d WHERE e.deptno = d.deptno GROUP BY CUBE (loc, dname, job) ORDER BY 1, 2, 3; Multidimensional Analysis - Cube ……………
- 17. © 2015 EnterpriseDB Corporation. All rights reserved. 17 • GROUP BY GROUPING SETS produces one result set that is a concatenation of multiple result sets based on different groupings (i.e. UNION ALL) SELECT loc, dname, job, COUNT(*) AS "employees" FROM emp e, dept d WHERE e.deptno = d.deptno GROUP BY GROUPING SETS (loc, dname, job) ORDER BY 1, 2, 3; Multidimensional Analysis – Grouping Sets
- 18. © 2015 EnterpriseDB Corporation. All rights reserved. 18 • UTL_HTTP – Built in package that provides a way to use the HTTP(S) protocol to retrieve information found at a URL. Commonly requested functions now provided: − REQUEST, REQUEST_PIECES − GET/SET_FOLLOW_REDIRECT − GET/SET_RESPONSE_ERROR_CHECK − GET/SET_HEADER, GET_HEADER_BY_NAME, GET_HEADER_COUNT − READ_RAW, READ_TEXT − BEGIN/END_REQUEST, GET_RESPONSE • Ch 8.17 of EDB Enterprise Edition / Ch 7.17 of EDB Oracle Compatibility Guide for more details on usage Develop Applications To Retrieve Data With UTL_HTTP Package
- 19. © 2015 EnterpriseDB Corporation. All rights reserved. 19 Preventing attacks is normally the responsibility of the application developer. But with SQL/Protect, DBAs can now provide another layer of protection to prevent corruption or co-opting of the database. Injection Attack Report SQL/ Protect Attacker PREVENTION TECHNIQUES Unauthorized Relations Utility Commands (e.g. DDL) SQL Tautology Unbounded DML EDBSQL/PROTECT DBA Managed with Centralized SQL Injection Protection Threat Deleted New in 9.4 - View edb_sql_protect_queries contains Injection Attack Report information (Ch 4.1.1.2.3 of EDB Enterprise Edition Guide for more information)
- 20. © 2015 EnterpriseDB Corporation. All rights reserved. 20 A More Robust Migration Toolkit New in 9.4 • A detailed log with well defined error codes to allow DBAs to better understand which capabilities of their database applications are migrate-able to PPAS. • See Ch 9 of Migration Toolkit documentation for a list of the error codes provided • Migration of schemas, tables, constraints, indexes, data, views and more • Online (immediate) or offline (DDL scripts) migration • Customize migrations with bulk inserts, row filters, change datatypes inline, subsets of schema objects • Parallel data movement techniques, bypass logging for faster data loads or use native connectivity to source database.
- 21. © 2015 EnterpriseDB Corporation. All rights reserved. 21 1. Customers running mixed workloads. 2. Application Developers who integrate with external Web servers. 3. Customers with large tables where they often search for exact matches or have many concurrent inserts/updates. 4. Users who think they need a NoSQL database. 5. Customers with reporting or data warehousing databases. 6. DBAs who use need to bulk load data. 7. DBA’s concerned with Security and SQL Injection Attacks. Recap: Postgres Plus Advanced Server (PPAS) 9.4 Use Cases
- 22. © 2015 EnterpriseDB Corporation. All rights reserved. 22 How can I learn more? • Download Postgres Plus Advanced Server: • http://www.enterprisedb.com/download-advanced-server • General information feature table: • http://www.enterprisedb.com/postgres-plus-advanced-server • Set up technical call with a Postgres Expert • [email protected]
- 23. © 2015 EnterpriseDB Corporation. All rights reserved. 23