Blogs

Oracle Trace File Analyzer (TFA) Upgrade to Address Log4j Vulnerability

As you are all aware, all organizations are allocating more time to address this Log4j vulnerability and every DBA/Infrastructure engineer is working on mitigating the Log4j issue. As for the oracle updates database side, we need to upgrade the TFA (Trace File Analyzer) utility to mitigate the Log4j vulnerability. 

The link below provides really good insight into how this Log4j works. We think it’s really important to get an understanding of this process before upgrading the vulnerable utilities and products.

https://socradar.io/what-do-you-know-about-the-log4j-critical-vulnerability-and-what-can-we-do/

How does Log4j vulnerability work?

How the Log4j processor handles the log messages is the root cause of the vulnerability. An attacker can remotely execute codes by sending a custom message that may include malicious code like the following:

This code insertion results in loading an external code class or message lookup and the execution of that code.

I hope that the link below will be useful to download the latest patches for OCT 2021. These patches come with addressing CVE-2021-44228 log4j vulnerability.

Main update link for CVE-2021-44228:

Oracle Security Alert Advisory – CVE-2021-44228

Quarterly patches including CVE-2021-44228:

Document 2796575.1 (oracle.com)

 

While upgrading the TFA on the database environment we faced a few unexpected issues. In this article, I will cover the TFA upgrade steps and solutions for TFA installation issues.

After downloading the stage, the patch run mentions the command to verify the patch. 

 

Verification

[root@ecl-odabase-0 AHF-LINUX_v21.3.4]# openssl dgst -sha256 -verify ./oracle-tfa.pub -signature ./ahf_setup.dat ./ahf_setup
Verified OK

 

Error:

[root@ecl-odabase-0 AHF-LINUX_v21.3.4]# ./ahf_setup

AHF Installer for Platform Linux Architecture x86_64

AHF Installation Log : /tmp/ahf_install_213400_6537_2021_12_16-12_01_12.log

Starting Autonomous Health Framework (AHF) Installation

AHF Version: 21.3.4 Build Date: 202112151432

[ERROR] : AHF-00099: Invalid Existing AHF on ODA VM Installation detected

[ERROR] : Please visit https://blogs.oracle.com/oda/using-orachk-with-the-oracle-database-appliance for advice

Note: The best option is to address this issue by performing a fresh installation.

 

First, make sure to uninstall TFA on all the nodes on the cluster.

[root@ecl-odabase-0 AHF-LINUX_v21.3.4]#  /opt/oracle/dcs/oracle.ahf/tfa/bin/tfactl uninstall
Starting AHF Uninstall
NOTE : Uninstalling does not return all the space used by the AHF repository
AHF will be uninstalled on:
ecl-odabase-0


Do you want to continue with AHF uninstall ? [Y]|N : Y

Stopping AHF service on local node ecl-odabase-0...
Stopping TFA Support Tools...


TFA-00002 Oracle Trace File Analyzer (TFA) is not running
Removing AHF setup on ecl-odabase-0:
Removing /etc/rc.d/rc0.d/K17init.tfa
Removing /etc/rc.d/rc1.d/K17init.tfa
Removing /etc/rc.d/rc2.d/K17init.tfa
Removing /etc/rc.d/rc4.d/K17init.tfa
Removing /etc/rc.d/rc6.d/K17init.tfa
Removing /etc/init.d/init.tfa...
Removing /opt/oracle/dcs/oracle.ahf/jre
Removing /opt/oracle/dcs/oracle.ahf/common
Removing /opt/oracle/dcs/oracle.ahf/bin
Removing /opt/oracle/dcs/oracle.ahf/python
Removing /opt/oracle/dcs/oracle.ahf/analyzer
Removing /opt/oracle/dcs/oracle.ahf/tfa
Removing /opt/oracle/dcs/oracle.ahf/orachk
Removing /opt/oracle/dcs/oracle.ahf/ahf
Removing /opt/oracle/dcs/oracle.ahf/data/ecl-odabase-0

Removing /opt/oracle/dcs/oracle.ahf/data/work
Removing /opt/oracle/dcs/oracle.ahf/install.properties

 

Verify the installation is completely uninstall.

[root@ecl-odabase-0 AHF-LINUX_v21.3.4]# /opt/oracle/dcs/oracle.ahf/tfa/bin/tfactl status
-bash: /opt/oracle/dcs/oracle.ahf/tfa/bin/tfactl: No such file or directory

Before starting the installation, remove the previously installed directory on both nodes.

##### remove folder before the installation

drwxr-xr-x 3 root root 4.0K Dec 16 12:04 oracle.ahf
[root@ecl-odabase-0 dcs]# rm -fr oracle.ahf
[root@ecl-odabase-0 dcs]# cd /u01/AHF/
[root@ecl-odabase-0 AHF]# ls -lrth

 

Installation

root@ecl-odabase-0 AHF-LINUX_v21.3.4]# ./ahf_setup

AHF Installer for Platform Linux Architecture x86_64

AHF Installation Log : /tmp/ahf_install_213400_71649_2021_12_16-12_12_48.log

Starting Autonomous Health Framework (AHF) Installation

AHF Version: 21.3.4 Build Date: 202112151432
Default AHF Location : /opt/oracle.ahf

Do you want to install AHF at [/opt/oracle.ahf] ? [Y]|N : Y

AHF Location : /opt/oracle.ahf

AHF Data Directory stores diagnostic collections and metadata.
AHF Data Directory requires at least 5GB (Recommended 10GB) of free space.

Choose Data Directory from below options :

1. /u01/app/grid [Free Space : 0 MB]
2. Enter a different Location

Choose Option [1 - 2] : 2

Please Enter AHF Data Directory : /opt/oracle.ahf

AHF Data Directory : /opt/oracle.ahf/data

Do you want to add AHF Notification Email IDs ? [Y]|N : N

AHF will also be installed/upgraded on these Cluster Nodes :

1. ecl-odabase-1

The AHF Location and AHF Data Directory must exist on the above nodes
AHF Location : /opt/oracle.ahf
AHF Data Directory : /opt/oracle.ahf/data

Do you want to install/upgrade AHF on Cluster Nodes ? [Y]|N : Y

Extracting AHF to /opt/oracle.ahf

Configuring TFA Services

Discovering Nodes and Oracle Resources

Not generating certificates as GI discovered

Starting TFA Services

.----------------------------------------------------------------------------------.
| Host          | Status of TFA | PID   | Port | Version    | Build ID             |
+---------------+---------------+-------+------+------------+----------------------+
| ecl-odabase-0 | RUNNING       | 95921 | 5000 | 21.3.4.0.0 | 21340020211215143236 |
'---------------+---------------+-------+------+------------+----------------------'

Running TFA Inventory...

Adding default users to TFA Access list...

.-----------------------------------------------------------.
|                Summary of AHF Configuration               |
+-----------------+-----------------------------------------+
| Parameter       | Value                                   |
+-----------------+-----------------------------------------+
| AHF Location    | /opt/oracle.ahf                         |
| TFA Location    | /opt/oracle.ahf/tfa                     |
| Orachk Location | /opt/oracle.ahf/orachk                  |
| Data Directory  | /opt/oracle.ahf/data                    |
| Repository      | /opt/oracle.ahf/data/repository         |
| Diag Directory  | /opt/oracle.ahf/data/ecl-odabase-0/diag |
'-----------------+-----------------------------------------'


Starting orachk scheduler from AHF ...

AHF install completed on ecl-odabase-0

Installing AHF on Remote Nodes :

AHF will be installed on ecl-odabase-1, Please wait.

Installing AHF on ecl-odabase-1 :

[ecl-odabase-1] Copying AHF Installer

[ecl-odabase-1] Running AHF Installer

[ERROR] : [ecl-odabase-1] Failed to Install AHF. Exit Status : 99

Adding rpm Metadata to rpm database on ODA system

RPM File /opt/oracle.ahf/rpms/oracle-ahf-213400-20211215143236.x86_64.rpm
Preparing...                ########################################### [100%]
Using Dummy RPM Installer for oracle-ahf
Tool Install Base /opt/oracle.ahf

   1:oracle-ahf             ########################################### [100%]
Upgrading oracle-ahf
warning:    erase unlink of /oracle-ahf-193000.zip failed: No such file or directory
warning:    erase unlink of /opt/oracle/dcs/oracle.ahf failed: No such file or directory

AHF binaries are available in /opt/oracle.ahf/bin

AHF is successfully installed

Do you want AHF to store your My Oracle Support Credentials for Automatic Upload ? Y|[N] : N

Moving /tmp/ahf_install_213400_71649_2021_12_16-12_12_48.log to /opt/oracle.ahf/data/ecl-odabase-0/diag/ahf/

You have new mail in /var/spool/mail/root
[root@ecl-odabase-0 AHF-LINUX_v21.3.4]#

 

Sync Issue

After installation, tfactl status shows only one node when we need to sync up the nodes to make this correct.

[root@ecl-odabase-1 ~]# /opt/oracle/dcs/oracle.ahf/tfa/bin/tfactl status
WARNING - TFA Software is older than 180 days. Please consider upgrading TFA to the latest version.

.-----------------------------------------------------------------------------------------------------.
| Host          | Status of TFA | PID   | Port | Version    | Build ID             | Inventory Status |
+---------------+---------------+-------+------+------------+----------------------+------------------+
| ecl-odabase-1 | RUNNING       | 10456 | 5000 | 19.3.0.0.0 | 19300020200108023845 | COMPLETE         |
'---------------+---------------+-------+------+------------+----------------------+------------------'

 

Solution: Execute sync nodes command

Execute /usr/bin/tfactl syncnodes command to represent both nodes

root@ecl-odabase-0 AHF]# /usr/bin/tfactl syncnodes

TFA has not yet generated any certificates on this Node.

Do you want to generate new certificates to synchronize across the nodes? [Y]|N: Y

Generating new TFA Certificates...

Successfully generated certificates.

Restarting TFA on ecl-odabase-0...
Shutting down TFA
oracle-tfa stop/waiting
Successfully shutdown TFA..

Starting TFA..
oracle-tfa start/running, process 87162
Waiting up to 100 seconds for TFA to be started..
. . . . .
Successfully started TFA Process..
. . . . .
TFA Started and listening for commands

Current Node List in TFA :
1. ecl-odabase-0

Node List in Cluster :
1. ecl-odabase-0
2. ecl-odabase-1

Node List to sync TFA Certificates :
     1  ecl-odabase-1
Do you want to update this node list? Y|[N]: N

Syncing TFA Certificates on ecl-odabase-1 :

TFA_HOME on ecl-odabase-1 : /opt/oracle.ahf/tfa

DATA_DIR on ecl-odabase-1 : /opt/oracle.ahf/data/ecl-odabase-1/tfa

Shutting down TFA on ecl-odabase-1...
Copying TFA Certificates to ecl-odabase-1...
Copying SSL Properties to ecl-odabase-1...
Sleeping for 5 seconds...
Starting TFA on ecl-odabase-1...


.-----------------------------------------------------------------------------------------------------.
| Host          | Status of TFA | PID   | Port | Version    | Build ID             | Inventory Status |
+---------------+---------------+-------+------+------------+----------------------+------------------+
| ecl-odabase-0 | RUNNING       | 87437 | 5000 | 21.3.4.0.0 | 21340020211215143236 | COMPLETE         |
| ecl-odabase-1 | RUNNING       | 30305 | 5000 | 21.3.4.0.0 | 21340020211215143236 | COMPLETE         |
'---------------+---------------+-------+------+------------+----------------------+------------------'

[root@ecl-odabase-0 AHF]#

 

 

More from this Author