Tag Archives: RMAN-03002

Delete Old SBT_TAPE Backups that are No Longer Available in RMAN

Posted on by Standard 6

When in the situation of having ‘SBT_TAPE’ backups, that are still known by the catalog or control file, but the device type ‘SBT_TAPE’ is no longer available.  If you try to delete the backups you get the following error message:

RMAN> list backup summary;

List of Backups
===============
Key TY LV S Device Type Completion Time #Pieces #Copies Compressed Tag
------- -- -- - ----------- --------------- ------- ------- ---------- ---
11208 B 0 A SBT_TAPE 20-JUL-17 1 1 YES TAG20170720T200232
11209 B 0 A SBT_TAPE 20-JUL-17 1 1 YES TAG20170720T200232
11210 B 0 A SBT_TAPE 20-JUL-17 1 1 YES TAG20170720T200232
...
11356 B 0 A SBT_TAPE 28-JUL-17 1 1 YES TAG20170728T200202
11357 B 0 A SBT_TAPE 28-JUL-17 1 1 YES TAG20170728T200202
11358 B A A SBT_TAPE 28-JUL-17 1 1 YES TAG20170728T201725

RMAN> delete backup;

RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-03002: failure of delete command at 08/10/2017 12:49:55
ORA-19554: error allocating device, device type: SBT_TAPE, device name:
ORA-27211: Failed to load Media Management Library
Additional information: 2

 

This is because the backups are of device type ‘SBT_TAPE’, which is no longer available.  To remove, you need to allocate channel for maintenance of device type ‘SBT’ but dummy to disk as explained in MOS Note:

Need Help To Remove Old Tape Backups From Catalog (Doc ID 342284.1)

“GOAL

How to delete obsolete TAPE backups via RMAN, when the Media Management Layer (MML Software not installed) is NOT available anymore.

So how to delete OLD backups which have been made to tape, while the current backup strategy is to disk and NO Media Manager is available.

SOLUTION

Allocate a maintenance channel with the dummy sbt API and run the DELETE FORCE OBSOLETE.

RMAN> allocate channel for maintenance device type sbt
parms 'SBT_LIBRARY=oracle.disksbt, ENV=(BACKUP_DIR=/tmp)';


RMAN> delete force obsolete;

The DUMMY API (oracle.disksbt) is simulating the callout to the Media Management Layer (MML).  This way RMAN thinks an actual MML is available and can perform the maintenance.”

My scenario:

RMAN> allocate channel for maintenance device type sbt
2> parms 'SBT_LIBRARY=oracle.disksbt, ENV=(BACKUP_DIR=/tmp)';

allocated channel: ORA_MAINT_SBT_TAPE_1
channel ORA_MAINT_SBT_TAPE_1: SID=808 device type=SBT_TAPE
channel ORA_MAINT_SBT_TAPE_1: WARNING: Oracle Test Disk API

RMAN> delete force obsolete;

RMAN retention policy will be applied to the command
RMAN retention policy is set to recovery window of 8 days
Deleting the following obsolete backups and copies:
Type Key Completion Time Filename/Handle
-------------------- ------ ------------------ --------------------
Backup Set 11217 20-JUL-17
 Backup Piece 11217 20-JUL-17 vks9rkqo_1_1
Backup Set 11214 20-JUL-17
 Backup Piece 11214 20-JUL-17 vls9rkqo_1_1
Backup Set 11216 20-JUL-17
 Backup Piece 11216 20-JUL-17 vms9rkqo_1_1
...
 Backup Piece 11341 28-JUL-17 3tsagnmq_1_1
Backup Set 11346 28-JUL-17
 Backup Piece 11346 28-JUL-17 3usagnmq_1_1
Backup Set 11340 28-JUL-17
 Backup Piece 11340 28-JUL-17 3vsagnmq_1_1

Do you really want to delete the above objects (enter YES or NO)? yes
deleted backup piece
backup piece handle=vks9rkqo_1_1 RECID=11217 STAMP=949867353
deleted backup piece
backup piece handle=vls9rkqo_1_1 RECID=11214 STAMP=949867353
deleted backup piece
backup piece handle=vms9rkqo_1_1 RECID=11216 STAMP=949867353
...
deleted backup piece
backup piece handle=3tsagnmq_1_1 RECID=11341 STAMP=950558426
deleted backup piece
backup piece handle=3usagnmq_1_1 RECID=11346 STAMP=950558426
deleted backup piece
backup piece handle=3vsagnmq_1_1 RECID=11340 STAMP=950558426
Deleted 132 objects

RMAN>

 

You may still have some ‘SBT_TAPE’ backups remaining due to your retention policy and hence the backups are not obsolete, therefore you need to crosscheck as follows:

RMAN> list backup summary;

List of Backups
===============
Key TY LV S Device Type Completion Time #Pieces #Copies Compressed Tag
------- -- -- - ----------- --------------- ------- ------- ---------- ---
11348 B 0 A SBT_TAPE 28-JUL-17 1 1 YES TAG20170728T200202
11349 B 0 A SBT_TAPE 28-JUL-17 1 1 YES TAG20170728T200202
11350 B 0 A SBT_TAPE 28-JUL-17 1 1 YES TAG20170728T200202
11351 B 0 A SBT_TAPE 28-JUL-17 1 1 YES TAG20170728T200202
11352 B 0 A SBT_TAPE 28-JUL-17 1 1 YES TAG20170728T200202
11353 B 0 A SBT_TAPE 28-JUL-17 1 1 YES TAG20170728T200202
11354 B 0 A SBT_TAPE 28-JUL-17 1 1 YES TAG20170728T200202
11355 B 0 A SBT_TAPE 28-JUL-17 1 1 YES TAG20170728T200202
11356 B 0 A SBT_TAPE 28-JUL-17 1 1 YES TAG20170728T200202
11357 B 0 A SBT_TAPE 28-JUL-17 1 1 YES TAG20170728T200202
11358 B A A SBT_TAPE 28-JUL-17 1 1 YES TAG20170728T201725

RMAN> allocate channel for maintenance device type sbt
2> parms 'SBT_LIBRARY=oracle.disksbt, ENV=(BACKUP_DIR=/tmp)';

allocated channel: ORA_MAINT_SBT_TAPE_1
channel ORA_MAINT_SBT_TAPE_1: SID=17 device type=SBT_TAPE
channel ORA_MAINT_SBT_TAPE_1: WARNING: Oracle Test Disk API

RMAN> delete backup;

List of Backup Pieces
BP Key BS Key Pc# Cp# Status Device Type Piece Name
------- ------- --- --- ----------- ----------- ----------
11348 11348 1 1 AVAILABLE SBT_TAPE 46sagnpr_1_1
11349 11349 1 1 AVAILABLE SBT_TAPE 47sagnpr_1_1
11350 11350 1 1 AVAILABLE SBT_TAPE 44sagnpr_1_1
11351 11351 1 1 AVAILABLE SBT_TAPE 48sagnqb_1_1
11352 11352 1 1 AVAILABLE SBT_TAPE 49sagnqb_1_1
11353 11353 1 1 AVAILABLE SBT_TAPE 45sagnpr_1_1
11354 11354 1 1 AVAILABLE SBT_TAPE 41sagnpr_1_1
11355 11355 1 1 AVAILABLE SBT_TAPE 43sagnpr_1_1
11356 11356 1 1 AVAILABLE SBT_TAPE 42sagnpr_1_1
11357 11357 1 1 AVAILABLE SBT_TAPE 40sagnpr_1_1
11358 11358 1 1 AVAILABLE SBT_TAPE 4asagoml_1_1

Do you really want to delete the above objects (enter YES or NO)? yes

RMAN-06207: WARNING: 11 objects could not be deleted for SBT_TAPE channel(s) due
RMAN-06208: to mismatched status. Use CROSSCHECK command to fix status
RMAN-06210: List of Mismatched objects
RMAN-06211: ==========================
RMAN-06212: Object Type Filename/Handle
RMAN-06213: --------------- ---------------------------------------------------
RMAN-06214: Backup Piece 46sagnpr_1_1
RMAN-06214: Backup Piece 47sagnpr_1_1
RMAN-06214: Backup Piece 44sagnpr_1_1
RMAN-06214: Backup Piece 48sagnqb_1_1
RMAN-06214: Backup Piece 49sagnqb_1_1
RMAN-06214: Backup Piece 45sagnpr_1_1
RMAN-06214: Backup Piece 41sagnpr_1_1
RMAN-06214: Backup Piece 43sagnpr_1_1
RMAN-06214: Backup Piece 42sagnpr_1_1
RMAN-06214: Backup Piece 40sagnpr_1_1
RMAN-06214: Backup Piece 4asagoml_1_1

RMAN> crosscheck backup;

crosschecked backup piece: found to be 'EXPIRED'
backup piece handle=46sagnpr_1_1 RECID=11348 STAMP=950558523
crosschecked backup piece: found to be 'EXPIRED'
backup piece handle=47sagnpr_1_1 RECID=11349 STAMP=950558524
crosschecked backup piece: found to be 'EXPIRED'
backup piece handle=44sagnpr_1_1 RECID=11350 STAMP=950558523
crosschecked backup piece: found to be 'EXPIRED'
backup piece handle=48sagnqb_1_1 RECID=11351 STAMP=950558539
crosschecked backup piece: found to be 'EXPIRED'
backup piece handle=49sagnqb_1_1 RECID=11352 STAMP=950558539
crosschecked backup piece: found to be 'EXPIRED'
backup piece handle=45sagnpr_1_1 RECID=11353 STAMP=950558523
crosschecked backup piece: found to be 'EXPIRED'
backup piece handle=41sagnpr_1_1 RECID=11354 STAMP=950558523
crosschecked backup piece: found to be 'EXPIRED'
backup piece handle=43sagnpr_1_1 RECID=11355 STAMP=950558523
crosschecked backup piece: found to be 'EXPIRED'
backup piece handle=42sagnpr_1_1 RECID=11356 STAMP=950558523
crosschecked backup piece: found to be 'EXPIRED'
backup piece handle=40sagnpr_1_1 RECID=11357 STAMP=950558523
crosschecked backup piece: found to be 'EXPIRED'
backup piece handle=4asagoml_1_1 RECID=11358 STAMP=950559445
Crosschecked 11 objects

RMAN> delete expired backup;

List of Backup Pieces
BP Key BS Key Pc# Cp# Status Device Type Piece Name
------- ------- --- --- ----------- ----------- ----------
11348 11348 1 1 EXPIRED SBT_TAPE 46sagnpr_1_1
11349 11349 1 1 EXPIRED SBT_TAPE 47sagnpr_1_1
11350 11350 1 1 EXPIRED SBT_TAPE 44sagnpr_1_1
11351 11351 1 1 EXPIRED SBT_TAPE 48sagnqb_1_1
11352 11352 1 1 EXPIRED SBT_TAPE 49sagnqb_1_1
11353 11353 1 1 EXPIRED SBT_TAPE 45sagnpr_1_1
11354 11354 1 1 EXPIRED SBT_TAPE 41sagnpr_1_1
11355 11355 1 1 EXPIRED SBT_TAPE 43sagnpr_1_1
11356 11356 1 1 EXPIRED SBT_TAPE 42sagnpr_1_1
11357 11357 1 1 EXPIRED SBT_TAPE 40sagnpr_1_1
11358 11358 1 1 EXPIRED SBT_TAPE 4asagoml_1_1

Do you really want to delete the above objects (enter YES or NO)? yes
deleted backup piece
backup piece handle=46sagnpr_1_1 RECID=11348 STAMP=950558523
deleted backup piece
backup piece handle=47sagnpr_1_1 RECID=11349 STAMP=950558524
deleted backup piece
backup piece handle=44sagnpr_1_1 RECID=11350 STAMP=950558523
deleted backup piece
backup piece handle=48sagnqb_1_1 RECID=11351 STAMP=950558539
deleted backup piece
backup piece handle=49sagnqb_1_1 RECID=11352 STAMP=950558539
deleted backup piece
backup piece handle=45sagnpr_1_1 RECID=11353 STAMP=950558523
deleted backup piece
backup piece handle=41sagnpr_1_1 RECID=11354 STAMP=950558523
deleted backup piece
backup piece handle=43sagnpr_1_1 RECID=11355 STAMP=950558523
deleted backup piece
backup piece handle=42sagnpr_1_1 RECID=11356 STAMP=950558523
deleted backup piece
backup piece handle=40sagnpr_1_1 RECID=11357 STAMP=950558523
deleted backup piece
backup piece handle=4asagoml_1_1 RECID=11358 STAMP=950559445
Deleted 11 EXPIRED objects

RMAN>

Now you can continue to use device type ‘DISK’ or configure a new ‘SBT_TAPE’ 🙂

 

If you found this blog post useful, please like as well as follow me through my various Social Media avenues available on the sidebar and/or subscribe to this oracle blog via WordPress/e-mail.

Thanks

Zed DBA (Zahid Anwar)

RMAN-06403: could not obtain a fully authorized session

Posted on by Standard Reply

An automated process that refreshes a Development database from Production by doing a RMAN duplicate failed with ‘RMAN-06403: could not obtain a fully authorized session’:

Starting Duplicate Db at 25-JUL-2017 17:44
RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-03002: failure of Duplicate Db command at 07/25/2017 17:44:46
RMAN-05501: aborting duplication of target database
RMAN-06403: could not obtain a fully authorized session
RMAN-04006: error from auxiliary database: ORA-01034: ORACLE not available
ORA-27101: shared memory realm does not exist
Linux-x86_64 Error: 2: No such file or directory
Additional information: 3651
Additional information: 1054940735

 

This was initially puzzling as nothing appeared to have changed.  A ‘My Oracle Support’ (MOS) search, revealed the following MOS note:

Duplicating database using RMAN fails with errors RMAN-06403, RMAN-04006, ORA-01034, ORA-27101 (Doc ID 274233.1)

“CAUSE

The errors are occurring because the auxiliary database is not started in nomount state.

SOLUTION

The auxiliary instance must be in a nomount state during duplication of database using RMAN.

Startup the auxiliary instance in nomount state.  Then try to duplicate the database again.

SQL> connect / as sysdba
startup nomount;

Also you have to verify the following:

1. Ensure that the ORACLE_SID and ORACLE_HOME were set correctly when starting the auxiliary instance.  Especially any additional backslash at the end of the ORACLE_HOME can cause mismatches

2. Verify that the ORACLE_HOME directory is not a linked directory.
Make a hard copy of the init.ora in the ORACLE_HOME/dbs directory.

3. Double check the ‘listener.ora’ and ‘tnsnames.ora’ for correctness.”

 

The database was in nomount state started with a basic pfile:

*.db_name=V1DEV
*.db_unique_name=V1DEV

 

But upon checking point 3, I noticed the static listener registration that is required as the database is in nomount and thus doesn’t register with the listener had the wrong home:

SID_LIST_LISTENER =
 (SID_LIST =
 (SID_DESC =
 (SID_NAME = V1DEV1)
 (GLOBAL_DBNAME = V1DEV)
 (ORACLE_HOME = /u01/app/oracle/product/12.1.0.2/dbhome_1)
 )
 )

 

I recently changed the Oracle Home to use the dbhome_2 Oracle Home, so that the Development databases were not locked in with the DR databases using the dbhome_1 Oracle Home.  This allows for independent patching between the Oracle Homes, which allows soak testing of patches in Development before applying to Production:

Cloning an Oracle Home

 

Once this was changed:

SID_LIST_LISTENER =
 (SID_LIST =
 (SID_DESC =
 (SID_NAME = V1DEV1)
 (GLOBAL_DBNAME = V1DEV)
 (ORACLE_HOME = /u01/app/oracle/product/12.1.0.2/dbhome_2)
 )
 )

 

It started working again 🙂 :

Starting Duplicate Db at 26-JUL-2017 11:38
allocated channel: ORA_AUX_DISK_1
channel ORA_AUX_DISK_1: SID=515 device type=DISK
allocated channel: ORA_AUX_DISK_2
channel ORA_AUX_DISK_2: SID=1475 device type=DISK
allocated channel: ORA_AUX_DISK_3
channel ORA_AUX_DISK_3: SID=643 device type=DISK
allocated channel: ORA_AUX_DISK_4
channel ORA_AUX_DISK_4: SID=835 device type=DISK
current log archived

 

If you found this blog post useful, please like as well as follow me through my various Social Media avenues available on the sidebar and/or subscribe to this oracle blog via WordPress/e-mail.

Thanks

Zed DBA (Zahid Anwar)