The Danger of Invalid Database Objects

Why they are dangerous.

An awareness Paper for IT-Managers.

http://www.IT-Checklists.com

Copyright © 2005-2006 Mercury Consulting Ltd. http://www.Mercury-Consulting-Ltd.com 1

Copyright © 2005 Mercury Consulting
Published in June 2005

Oracle® and Oracle-based trademarks and logos are trademarks or registered trademarks of Oracle Corporation, Inc.
in the United States and other countries.

Mercury Consulting is independent of those companies.

Conventions
The following typographical conventions are used in this book:

Constant width
Used for source code
Constant width bold
user input
[] optional elements
{} mandatory choice
| separates choices

Copyright © 2005-2006 Mercury Consulting Ltd. http://www.Mercury-Consulting-Ltd.com 2

Table of Contents
Summary.......................................................................................................................................................... 4
Audience.......................................................................................................................................................... 4
Introduction.......................................................................................................................................................4
Impact Analysis................................................................................................................................................ 5
Real world case study...................................................................................................................................... 6
Important Housekeeping: Clean the database from invalid objects.................................................................7
Even if you have no time for "cleaning" databases from invalid objects, then you must ................................ 8
Prevention for new problems of this type......................................................................................................... 8
How Mercury Consulting Ltd. can help you .....................................................................................................9

Copyright © 2005-2006 Mercury Consulting Ltd. http://www.Mercury-Consulting-Ltd.com 3

Summary
This document shows the risks when operating a database with invalid objects. This document is written for
members of application support operations managers. The operations managers should understand the
risk associated when operating databases with invalid objects and he should solve the dispute
between application support and DBA's who both argument that this is not "his" but "the others" problem.
An invalid database object (Synonym, Package, Procedure, View,..) could indicate a problem, but it might
be also harmless. Nevertheless operating a database with "harmless" invalid objects makes the detection
of a new invalid object indicating a serious problem impossible or much more difficult. Combined with
missing controls in other areas severe malfunction might not be detected. In case of problems the
troubleshooting process might require more time because investigators might also research all invalid
objects.
Effort for an initial cleaning (getting rid of invalid objects) of all databases pays off, as all involved persons
get awareness about risks and importance so that new systems and changes will not introduce new invalid
objects. Thus you will increase the maturity level of your IT.

Audience
• Operations Manager
• Testing Manager
• Environment Manager
• Application Support Staff
• System-DBA, Application-DBA, Development-DBA
• Quality Assurance Manager

Introduction
In an Oracle Database an object (Synonym, Package, Procedure, View, ...) becomes invalid if one of its
underlying objects is changed. If the change on the base object does not cause an error in the referencing
object, a simple recompiling will make the object valid again. This recompile is automatically triggered if an
invalid object is accessed. But if the change on the underlying, referenced object causes an error in the
calling object, this calling object CANNOT be recompiled, and all programs calling this now invalid objects
will fail.

There are two types of invalid objects:

1) Objects which become valid when recompiling them
2) Objects which cannot be successfully recompiled

Invalid objects of type 1) are not critical but they make monitoring and troubleshooting more difficult.
Therefore it should be investigated why they became invalid (and implement a mechanism to recompile
them after this event). As objects should be recompiled at next access those objects might be obsolete if
they stay invalid for a long time. Requesting a patch (or at least written approval) from application vendor to
remove those obsolete objects is the most effective way to get the database clean.
Invalid objects of type 2) could be either obsolete objects (e.g. a new version removed a table or procedure
but not the public synonym) or indicate a problem.
Before drilling down to details the next chapter "Impact Analysis" should demonstrate the risks and issues
caused by invalid objects.

Copyright © 2005-2006 Mercury Consulting Ltd. http://www.Mercury-Consulting-Ltd.com 4

Impact Analysis

If you do have invalid objects in your database, then answer following questions:

Question
1) How do you monitor your A) Our monitor system shows permanent a red flag for "invalid objects" but we ignore it because
system for invalid objects ? we know that we have invalid objects. We are aware that we would not detect a new invalid
object.
B) We disabled this control in our monitoring system because we know that we have invalid
objects. We are aware that we would not detect a new invalid object.
C) We customized our monitoring system to ignore the known invalid objects, but we would get
an alarm if there is a new invalid object.
D) Our monitoring system does not this control on invalid objects.
E) We don't have database monitoring at all.
2) In case of implementing a ..A) detect new invalid objects because we compare during the post implementation check the
change on that database we... invalid objects with the list of documented (known) invalid objects.
.. B) will not detect a new invalid object because we don't check for invalid objects at all.
.. C) will not detect a new invalid objects because we don't know in detail which objects have
been invalid before applying the change.

Answers to question 1:
C You did a great job, but you surely know which costs the customization did cost.
A, B, D, E We hope that you have excellent controls on many other places to detect malfunctions.

Answer to question 2:
A) You are doing a great job, but you are aware about the additional effort compared to a clean (free of invalid
objects) database.
B), C) You hopefully have a good post implementation check to identify that after implementing this change something
does not work. Take care! - The NEW module you installed or changed might work, but ANOTHER,
UNCHANGED module might suddenly fail!

Copyright © 2005-2006 Mercury Consulting Ltd. http://www.Mercury-Consulting-Ltd.com 5

Real world case study
You should not spend too much time on discussing the different arguments in table "Blamestorming" - just
take this unlucky incident as warning to keep your system (not only the database) as clean as possible and
use each slightest indication of a problem as warning and investigate it.

procedure JOB1 (1) Initial Situation

IS
....
....
BEGIN
....
....
INSERT INTO LOG_TABLE values (sysdate, 'JOB1'', 'START', 0);
....
....
END;

Table LOG_TABLE
EDATE date
PROCESS_NAME varchar2(30)
STATUS varchar2(5)
RESULTCODE number(3,0)

(2) Changerequest
New process "JOB2"
Additional column "PROCESS_STEP" added to
table "LOG_TABLE"

(3) New Situation after

new
Change procedure JOB2
IS
procedure JOB1 unchanged ....
BEGIN
IS but cannot be ....
.... recompiled! ....
.... iNSERT INTO LOG_TABLE
BEGIN (EDATE, PROCESS_NAME, PROCESS_STEP, STATUS, RESULTCODE)
.... values
.... (sysdate, 'JOB2'','INIT', 'START', 0);
INSERT INTO LOG_TABLE values (sysdate, 'JOB1'', 'START', 0); ....
.... ....
.... END;
END;

Table LOG_TABLE
EDATE date
PROCESS_NAME varchar2(30)
STATUS varchar2(5)
RESULTCODE number(3,0) changed
PROCESS_STEP varchar2(30)

(a) Due to change at table LOG_TABLE

(adding a column)
procedure JOB1 becomes invalid (documented
behaviour),

(b) but due to the (bad) coded "INSERT

Statement" the additional column causes the
statement to fail, thus the procedure CANNOT
be recompiled to become valid!

=> NOTE: Although this procedure was not

changed, suddenly this old, unchanged (and
therefore not tested) job does not run
anymore!

Copyright © 2005-2006 Mercury Consulting Ltd. http://www.Mercury-Consulting-Ltd.com 6

The Blamestorming
Blame Response
It's a development problem. You never should The developer arguments that no coding standard. And there was no code-
code an insert-statement without explicitly review.
listing the column-names.
The testing team failed, because it should have a) Testing team did not look at all
detected the invalid procedure "JOB1" already
on the testing environment. b) There are such many invalid objects that nobody can identify a new invalid
object.
c) This could not be detected, as the new change changing the LOG_TABLE
and introducing procedure JOB2 was tested on test environment T2, and
procedure JOB1 was installed only on test environment T1 but NOT on T2;
therefore JOB1 could not become invalid. But that is a problem of "environment
maintenance".
The testing team failed because it did not a We don't have enough time to re-test all applications after each small change.
complete re-testing of all modules
There is no documentation like a "Function / Entity" Matrix showing the
dependencies, so we cannot identify dependent modules.
The post release check failed, because the The System DBA arguments that database objects in the application schema
invalid object was not identified and are under the responsibility of the application support team.
investigated after implementing the change.
Application Support members argument that they are not database experts and
don't understand what an invalid database object is.
Fact: There is a problem in the release process and in the CLEAR,
unambiguously assignment of responsibilities. To address this quite typical
problem Mercury Consulting Ltd. created product "DBA and Application
Support: Job Description and Self-Assessment".
The problem that the old process JOB1 did This issue is addressed in Mercury Consulting Ltd.'s free whitepaper "The
start but immediately fail with error message importance of application level monitoring" and in the product "Non
added to logfile for several days is a mistake of functional Requirements", a requirements template with 150 non functional
application support, because there was neither requirements, among those many about application level monitoring.
a monitoring for errors in the log file nor an
independent monitoring of successful job
completion.
Conclusion:
(1) Big problems are very seldom the result of one single mistake; therefore you should try to eliminate as
many small mistakes as possible.
(2) The problem could have been identified in many places - take each chance to detect an anomaly.
Keeping your systems clean will help you to detect new anomalies.

Important Housekeeping: Clean the database from invalid objects

Statement Comments
We have no time You definitely need time to document all invalid objects with an explanation why there is
no problem (including explanations from Vendors or developers).
If you have no time to write this documentation now, you can for sure plan that time as
follow-up action after the next external IS audit.
And you will need additional time to modify your monitoring system to ignore exactly
those known and documented invalid objects.
Disabling the existing monitor event is about the same as removing the red light bulb in
your car's dashboard indicating a problem in the break system or oil level.
And the author has absolutely no idea how to explain an external auditor why an
standard alarm was disabled.
We don't know whom to ask because 1) You definitely have a problem in your change management- and configuration
we don't know if those invalid objects management process. And you should fix that before the next external IS audit.
are part of the application delivery or
were created by someone else. 2) Check the vendor's documentation

Copyright © 2005-2006 Mercury Consulting Ltd. http://www.Mercury-Consulting-Ltd.com 7

Even if you have no time for "cleaning" databases from invalid
objects, then you must ...
The author understands that in many cases the existence of invalid objects is not the DBA's fault and not
the application support member's fault, they have been introduced by others.
But nevertheless application support (if object is in the application schema), System-DBA (if object is in one
of the system-schemas as "SYS", "SYSTEM", "OUTLN", ...) must document this problem and all steps
done to mitigate it.

Nr. Action
1) Immediately add a "Problem" to your "Problem Management System". (This is the ITIL1 term, in your company you
might use another term for the system documenting, tracking and managing problems).
2) Document all steps which you tried to solve the problem. If it's a 3rd party application you should at least open a
Anomaly, TAR, Problem at the vendor's support. The vendor might provide a patch (script) to remove the obsolete
objects, or approve the removal or he will tell you that this is OK. Although the last response is not acceptable, you at
least will have an official statement from the application vendor which you can show to the auditor who might list the
invalid objects as defect.
3a) If you got rid of invalid objects, then you can close the problem.

3b) If you could not, but got official statements from the application vendor, then you could change the status to "Known
Problem".
But additionally you must customize the monitoring system to ignore the DOCUMENTED, KNOWN invalid objects but to
insure to detect new invalid objects.

Prevention for new problems of this type

As you have enough problems in living with your existing "dirty" databases you should definitely avoid that
new dirty databases will be introduced. Therefore you need to add the requirement that the delivered
system or change must be free of any invalid object as mandatory requirement to your requirements list or
acceptance criteria.

If you do have this requirement documented but those requirements are not communicated to vendors and
developers you do have a problem in the change management process.

If you use our requirements template "Non Functional Requirements", then this requirement is one of 150
other requirements.

1 ITIL® = "IT Infrastructure Library" is not an official standard but the worldwide most accepted collection of best practices for Service Delivery
and Service Support. ITIL is a registered trademark of UK's OGC , Office of Government Commerce.

Copyright © 2005-2006 Mercury Consulting Ltd. http://www.Mercury-Consulting-Ltd.com 8

How Mercury Consulting Ltd. can help you
Mercury Consulting Limited (MCL) is a professional consultancy providing experience, support and training
in IS/IT operations for companies during high-time-pressure startup phase and following consolidation
phase, especially in the Telecom-market.

Our Products
you can purchase online and immediately download at our
eBook-Shop
Our Checklists and Templates will help you to ensure that good or best practice is not only known but
consistently applied!

Database Independent Products

Product Benefit
150 Non functional Requirements. Requirements Template with 150 non functional requirements for
selecting or developing robust, reliable, stable and manageable
applications to meet the Service Level Agreements (SLA's).
For external RFP's (Request for Proposal) and for internal development.
Checklist for Data Migration 65 important questions to identify and address or exclude typical
migration pitfalls in an early phase of the project, thus ensuring the
confidence for keeping the time plan.
Template: Systems- and Operations Template to establish that documentation auditors like to see!
Handbook
Interface Checklist Those questions which you need to ask before starting the development!
Requirements, Checklist and Template for Planning, Defining and
Documenting Application Interfaces.
Checklist for Production Release and Checklist for small and medium projects focusing on non-functional
System Handover aspects for operations team.
Simple but effective!
Business Requirements for Archiving Template with Business Requirements for Archiving & Restore & Purging.
and Purging Not removing old customer data can cause conflicts with privacy laws.
Business must act and clearly specify what to purge and what to
archive!
Application Health Check: Using this template to check your systems - and DOCUMENT the findings
Stability Assessment might show you even more potential issues beyond invalid objects.
Technology Selection for Standby database using transfer of transaction log / archive logs or
Disaster Recovery (DR) replicating the database files ?
Host-based, Storage based or Switch/Fabric based replication ?
This document evaluates the different techologies and their advantages
and constraints from an operational point of view.
Backup SLA / OLA: Operations Level A template which supports the creation of an operational level
Agreement with the Backup-Team agreement (OLA) between application support, database administrators
and backup team.

Copyright © 2005-2006 Mercury Consulting Ltd. http://www.Mercury-Consulting-Ltd.com 9

Database Specific Products
Product Benefit
Template for Database Operations Level If a Service Level Manager needs to offer to Business a Service Level
Agreement (OLA) / SLA Agreement (SLA) for an End-to-End IS/IT-Service, he sign this SLA
only after he arranged within IS/IT for each system or component
used to provide this service an Operations Level Agreement (OLA)
with the providing team or department.
This document provides a template for such an Operations Level
Agreement (OLA) for Databases containing the agreed values and for
QA-purpose also the measures implemented to reduce the likelihood
of violations of those agreed values.
It does not only deal with availability, but contains also
comprehensive service catalogue of advanced DBA services and a
template for the Service Level Reporting (SLR).
Database Health Check - Stability Assessment of your Database.
Part 1: Stability Assessment Most Application- and Database-Crashes can be avoided when
detecting early indicators and reacting to them.
Be Proactive - Check Now!
DBA and Application Support: Checklist to ensure that all 60 DBA-duties are assigned:
Job Description and Self Assessment System DBA, Development DBA and Application DBA versus
Application Support
If you have just a job role "DBA", but not a dedicated job role
"Application DBA" those 19 duties must be explicitly assigned to either
"Application Support" or to the "DBA" - otherwise they might not be
executed!
Detect unassigned tasks before an auditor reports them !
This product addresses disputes between System DBA and Application
Support (or, if existing, dedicated Application DBA) regarding the
responsibility for the application's database objects.

Copyright © 2005-2006 Mercury Consulting Ltd. http://www.Mercury-Consulting-Ltd.com 10

Our free Whitepapers
http://www.mercury-consulting-ltd.com/wp/whitepapers.html

Whitepaper Benefit and Description

The Importance of Application Level Monitoring. Keeping your applications free from invalid objects is an important task, but
does not guarantee error free operations. This free whitepaper explains the
difference between "Application Level Monitoring" to "Database Monitoring"
and "System/Server/OS"-Monitoring.
Important Facts about Redolog- and Archivelog / Understanding the topic “archivelog volume” can avoid unexpected troubles
Transaction Logs in Databases for Change when applying changes or conducting upgrades.
Managers and Application Support Staff This awareness paper explains why Change Managers must ask questions
about the archivelog volume created during changes and upgrades and why
application support staff – and not only the DBA - must understand this topic.
The Danger of Invalid Database Objects An awareness paper for Operations Managers and Application Support
describing the problems and potential risks caused by invalid objects in
Databases.
Private or Public Synonyms A decision support paper visualizing the pro's and contras on a single page in
tabular form and evaluating the arguments.
– or no Synonyms at all ?
Daily, weekly or monthly partitions? This document evaluates the different factors impacting the decision if daily,
weekly or monthly partitions should be used. Legal- , performance- and
maintenance aspects are evaluated.

Our Services
http://www.Mercury-Consulting-Ltd.com/services.htm

About the Author

The Author of this white paper is an

With 12 years experience as DBA and 6 years experience in Telecommunications Companies.

Copyright © 2005-2006 Mercury Consulting Ltd. http://www.Mercury-Consulting-Ltd.com 11