MySQL: Updating the Configuration of a MySQL DB System on OCI
Overview: A MySQL configuration is a collection of variables that define the operation of a MySQL DB system. It is analogous to ... Read More
Learn more about why Eclipsys has been named the 2023 Best Workplaces in Technology and Ontario, Certified as a Great Place to Work in Canada and named Canada’s Top SME Employer!
Learn more!Starting with Oracle 11gR2, it is difficult to lose your OCR file or voting disk as they are stored in an ASM disk group with normal or high redundancy.
The scenario below will corrupt the header of oracleasm disks that belong to disk group that has an OCR file, a voting disk, and ASM Password file.
Then we will show the steps and commands to recover from losing those files by
Unfortunately, there is NO other option to remove OCR file. We can’t manually delete OCR file from ASM, because the file is being accessed.
For example,
ASMCMD> rm -rf +DATA/rac-cluster/OCRFILE/REGISTRY.255.966255313 ORA-15032: not all alterations performed ORA-15028: ASM file '+DATA/rac-cluster/OCRFILE/REGISTRY.255.966255313' not dropped; currently being accessed (DBD ERROR: OCIStmtExecute)
Also, when cluster or CRS is down, we are not able to get to the OCR file as it is not available.
For example,
[oracle@rac1 ~]$ asmcmd Connected to an idle instance. ASMCMD> ls ASMCMD-8102: no connection to Oracle ASM; command requires Oracle ASM to run ASMCMD>
Assumptions
Scenario preparation | |
---|---|
Step# | Description |
1 | Take a backup for ASM SPFILE file.
— grid account |
2 | Take a backup for ASM metadata.
— grid account |
3 | Take a backup for the last OCR automatic backup.
— grid account |
4 | Run below commands to show OCR & voting disk info and save the output as a reference.
— grid account |
5 | Query oracleasm disk info for REDO disks and save the output as a reference to be used later when we recreate the disks again.
— root account |
6 | This is an optional as long as databases files are stored on another disk groups – Take a backup for your databases using RMAN and store backup on file system. |
Corrupt REDO disks header and cleanup all oracle processes | |
Step# | Description |
1 | Corrupt REDO disks header using DD command from one node only.
— root account |
2 | Most of CRS resources will crash and servers may reboot. We need to cleanup oracle processes on all cluster nodes.
— root account |
3 | Cleanup oracleasm disks on all cluster nodes.
— root account |
Recover from losing OCR disk group | |
Step# | Description |
1 | Create oracleasm disks for REDO disk group. Run from one node only then scan disks on another nodes.
— root account — On remaining nodes as root account $ oracleasm scandisks |
2 | Start ASM instance on one node only in exclusive mode.
— root account |
3 | Check CRSD is down and stop it if it is not (STATE=OFFLINE).
— root account |
4 | Query voting disks and drop them if they exist.
— root account |
5 | Connect to ASM instance and drop REDO disk group if it still exists.
— grid account |
6 | Connect to ASM instance and set ASM parameter asm_diskstring.
— grid account |
7 | Create REDO disk group using the metadata backup script we created earlier.
Copy SQL commands, which are related to REDO disk group from backup script, connect to ASM instance, then run the commands. — grid account |
8 | Restore the OCR from backup done before.
— root account |
9 | Add voting disks.
— root account |
10 | Connect to ASM instance and create ASM SPFILE
— grid account |
11 | CRS is required to be running in non-exclusive mode to recreate the ASM password file in the disk group. This means some ‘application’ cluster resources will be started and available now.
— root account |
12 | Create ASM password file on REDO disk group.
— grid account 12.1. Create password file ASMCMD> pwget –asm 12.2. Grant sysasm to SYS ASMCMD> orapwusr –grant sysasm SYS 12.3. Add ASMSNMP user and grant sysasm & sysdba ASMCMD> orapwusr –add ASMSNMP è enter the same password used to create password file 12.4. Add user CRSUSER__ASM_001 to ASM password file 12.4.1. Dump OCR file and search dump file for “CRSUSER__ASM_001” to get user “ORATEXT” $ $GRID_HOME/bin/ocrdump /tmp/OCR.dmp 12.4.2. Get user “CRSUSER__ASM_001” password $ /prod01/home/dba/grid/12.2.0.1/bin/crsctl get credmaint -path /ASM/Self/a332f4dbe3da5f4bbf3a71895a466319 -credtype userpass -id 0 -attr user -local 12.4.3. Add user CRSUSER__ASM_001 to password file ASMCMD> orapwusr –add CRSUSER__ASM_001 è Use password you got it from previous command |
13 | Stop and start CRS normally. All resources should start normally.
— root account |
14 | Start CRS on the remaining cluster node(s). All resources should start normally.
— root account |
15 | Run a cluster verification of all cluster nodes. Run from one node only.
— grid account |
16 | Check the status of cluster resources. Resources should be ONLINE on all nodes.
— grid account |
17 | Stop database(s) then startup mount to recreate redo log groups on REDO disk group.
— oracle account |
18 | Stop database(s) then startup.
— oracle account |
Overview: A MySQL configuration is a collection of variables that define the operation of a MySQL DB system. It is analogous to ... Read More
Overview: Oracle 23c introduces the use of SUM and AVG functions with INTERVAL datatype This enhancement makes it easier to calculate totals and ... Read More