Oracle Autonomous Health Framework AHF

Autonomous Health Framework (AHF) provides a number of diagnostic tools in a

single bundle, making it easy to gather diagnostic information about the Oracle
database and clusterware, which in turn helps with problem resolution when
dealing with Oracle Support. AHF) combines the existing Trace File Analyzer
(TFA) and ORAchk/EXAchk, along with a bunch of other stuff, into a single download
and setup. For any scenario where you would have installed one of these
individual tools, you should now install AHF.

Autonomous Health Framework (AHF) will be shipped with new releases of the
database, but you should always download the latest version. Single installer for
Trace File Analyzer (TFA) and ORAchk/EXAchk
Autonomous Health Framework (AHF) - Including TFA and ORAchk/EXAChk (Doc ID
2550798.1)
If possible you should install as root. This will give you the richest capabilities.. Unzip
the
software and run the ahf_setup command. Answer the questions when prompted. 
Download the appropriate installer from above, copy to a target machine and unzip
Run the TFA & ORAchk/EXAchk install command 

./ahf_setup [-ahf_loc install_dir] [-data_dir data_dir]

# cd /u01/software

# unzip -oq AHF-LINUX_v20.4.0.zip

# ./ahf_setup

AHF Installation Log : /tmp/ahf_install_7860_2020_12_31-11_11_11.log

Starting Autonomous Health Framework (AHF) Installation

AHF Version: 204000 Build Date: 202012317860

Default AHF Location : /opt/oracle.ahf

Do you want to change AHF Location (/opt/oracle.ahf) ? Y|[N] :

AHF Location : /opt/oracle.ahf

AHF Data Directory stores diagnostic collections and metadata.

AHF Data Directory requires at least 5GB (Recommended 10GB) of free space.

Choose Data Directory from below options :

1. /opt/oracle.ahf [Free Space : 1007860 MB]

2. Enter a different Location

Choose Option [1 - 2] : 1

AHF Data Directory : /opt/oracle.ahf/data

Do you want to add AHF Notification Email IDs ? [Y]|N :

Enter Email IDs separated by space : [email protected]

Extracting AHF to /opt/oracle.ahf

Configuring TFA Services

Discovering Nodes and Oracle Resources

Starting TFA Services

Created symlink from /etc/systemd/system/multi-user.target.wants/oracle-tf

Created symlink from /etc/systemd/system/graphical.target.wants/oracle-tfa

.-------------------------------------------------------------------------
| Host | Status of TFA | PID | Port | Version | Build ID
+-----------+---------------+------+-------+------------+-----------------
| localhost | RUNNING | 7860 | 24035 | 20.4.0.0.0 | 2040002020123110
'-----------+---------------+------+-------+------------+-----------------

Running TFA Inventory...

Adding default users to TFA Access list...

.-------------------------------------------------------.

| Summary of AHF Configuration |

+-----------------+-------------------------------------+

| Parameter | Value |

+-----------------+-------------------------------------+

| AHF Location | /opt/oracle.ahf |

| TFA Location | /opt/oracle.ahf/tfa |

| Orachk Location | /opt/oracle.ahf/orachk |

| Data Directory | /opt/oracle.ahf/data |

| Repository | /opt/oracle.ahf/data/repository |

| Diag Directory | /opt/oracle.ahf/data/localhost/diag |

'-----------------+-------------------------------------'

AHF binaries are available in /opt/oracle.ahf/bin

AHF is successfully installed

Moving /tmp/ahf_install_7860_2020_12_31-11_11_11.log to /opt/oracle.ahf/da

As shown above, the tfactl and orachk commands are available from the

following directory.

/opt/oracle.ahf/bin

[root@ed-ol-rac-lin01::~]$ exachk -v

EXACHK  VERSION: 20.4.0_20201214

As part of installation 'oracle-tfa.service' will be enabled and started.

# systemctl status oracle-tfa.service

● oracle-tfa.service - Oracle Trace File Analyzer

   Loaded: loaded (/etc/systemd/system/oracle-tfa.service; enabled; vendor preset: disabled)

   Active: active (running) since Wed 2020-12-22 11:11:11 NZDT; 1 weeks 2 days ago

 Main PID: 1007860 (init.tfa)

   CGroup: /system.slice/oracle-tfa.service

           ├─ 1007860 /bin/sh /etc/init.d/init.tfa run >/dev/null 2>&1 </dev/null

           ├─ 12345 /opt/oracle.ahf/jre/bin/java -server -Xms256m -Xmx512m -Djava.awt.headless=true -

Ddisable.checkForUpdate=true -XX:HeapDump...

           └─284747 /bin/sleep 30

Dec 31 09:39:54 ed-ol-raclin01 su[195700]: (to oracle) root on none

Dec 31 09:39:59 ed-ol-raclin01 su[196070]: (to root) root on none

Dec 31 09:39:59 ed-ol-raclin01 su[196075]: (to root) root on none

Dec 31 09:39:59 ed-ol-raclin01 su[196079]: (to oracle) root on none

Dec 31 09:39:59 ed-ol-raclin01 su[196086]: (to oracle) root on none

Dec 31 09:39:59 ed-ol-raclin01 su[196091]: (to oracle) root on none

Dec 31 09:40:00 ed-ol-raclin01 su[196265]: (to oracle) root on none

Dec 31 09:40:00 ed-ol-raclin01 su[196268]: (to oracle) root on none

Dec 31 09:40:00 ed-ol-raclin01 su[196271]: (to oracle) root on none

Dec 31 09:40:00 ed-ol-raclin01 su[196276]: (to oracle) root on none

Warning: oracle-tfa.service changed on disk. Run 'systemctl daemon-reload' to reload units.

Use 'systemctl daemon-reload' command to reload units

#systemctl daemon-reload

tfactl and orachk commands are available from the following directory.
$ORACLE_HOME/ahf/oracle.ahf/bin
With the installation complete we can use the orachk command to check the services running on
the server. This can be done on-demand, or run as a background task, which will send an email to
the notification email addresses.

# as root

cd /opt/oracle.ahf/bin

# as non-root

cd $ORACLE_HOME/ahf/oracle.ahf/bin

# on-demand

./orachk

# background

./orachk -autostart

Once this is complete the oracle-orachkscheduler.service service will be enabled and running.

# systemctl status oracle-orachkscheduler.service

Run a TFA Collection

With the installation complete we can use the tfactl command to perform a
number of collections, including TFA Service Request Data Collections (SRDC).
There are a large number of SRDC collection types, with each gathering different
information, as described here.

Here are a few examples.

# root

cd /opt/oracle.ahf/bin

# non-root

cd $ORACLE_HOME/ahf/oracle.ahf/bin

# Gather information about errors. You are prompted to select a specific i

./tfactl diagcollect -srdc ORA-00600

./tfactl diagcollect -srdc ORA-07445

# Collect data for all components for a specific time period.

./tfactl diagcollect -from "2020-12-15 11:11:11" -to "2020-12-31 11:11:11"

# Collect data for all components for the last 12 hours.

./tfactl diagcollect

Each TFA collection produces a single zip file that can be uploaded to My Oracle
Support (MOS), as described below.

Upload Files to My Oracle Support (MOS)

You can manually upload a zip file produced by TFA or ORAchk by attaching it to
your SR through the MOS website in the normal way. An alternative is to get TFA
to upload it and attach it to your Service Request (SR). To do this you must
provide your MOS credentials. This can be done as part of the command line, or
they can be stored in a secure wallet by running the following command as the
"root" user.

# root

cd /opt/oracle.ahf/bin

# non-root

cd $ORACLE_HOME/ahf/oracle.ahf/bin

# ./tfactl setupmos

Enter User Id: [email protected]

Enter Password:

SUCCESS - CERTIMPORT - Successfully imported certificate

The result of a TFA collection can be uploaded directly to MOS by including the "-
sr" option in the command to specify the target SR number.

# MOS credentials supplied by wallet.

./tfactl diagcollect -srdc ORA-00600 -sr 1-12345678910

# MOS credentials supplied on command line. You are prompted for the passw
./tfactl diagcollect -srdc ORA-00600 -sr 1-12345678910 -user sami.malik@or
You can upload one or more files (both TFA generated and other files) as a
separate action using the following commands.

# MOS credentials supplied by wallet.

./tfactl upload -sr 1-12345678910 -wallet file1.zip file2.zip file3.zip

# MOS credentials supplied on command line. You are prompted for the passw
./tfactl upload -sr 1-12345678910 -user [email protected] file1.zip fi

The Autonomous Health Framework (AHF) installer includes:

TFA is the Primary diagnostic collection tool, with Database Support Tools Bundle

Memory can be limited at either the system level using:

tfactl setresourcelimit -resource kmem

or combined system and swap memory, using:

tfactl setresourcelimit -resource swmem

For example:

To limit the memory usage to only 1GB of system memory run:

tfactl setresourcelimit -resource kmem -value 1024

Alternatively to limit the combined total of system memory and the swap memory to 2GB use:

tfactl setresourcelimit -resource swmem -value 2048

ORAchk / EXAchk
Oracle Stack Compliance Checks

procwatcher
Automate & capture database performance diagnostics & session level hangs
See 459694.1 for more details.

events
Reports warnings and errors seen in the logs

managelogs
Shows disk space usage and purges ADR log and trace files

alertsummary
Provides summary of events for one or more database or ASM alert files from all nodes

ls / dir
Lists all files TFA knows about for a given file name pattern across all nodes

summary
High level summary of the configuration
 vi / notepad
Open alert or trace files for viewing a given database and file name pattern in the vi editor

tail
Run a tail on an alert or trace files for a given database and file name pattern

param
Show all database and OS parameters that match a specified pattern

oswatcher
Collect and archive OS metrics, useful for instance / node evictions & performance Issues.
See 301137.1 for more details.

oratop
Near real-time database monitoring
See 1500864.1 for more details.

pstack
Generate process stack for specified processes across all nodes

grep / findstr
Search alert or trace files with a given database and file name pattern, for a search string

dbglevel
Set and unset multiple CRS trace levels with one command

history
Show the shell history for the tfactl shell

changes
Report any noted changes in the system setup over a given time period. This includes database a
parameters, OS parameters, patches applied etc

calog
Reports major events from the Cluster Event log

ps / tasklist
Finds processes

triage
Summarize oswatcher/exawatcher data