Print zombie processes

Print zombie processes
ps aux | awk '"[Zz]" ~ $8 { system(sprintf("ps -p %d", $2)); }'

Fix ASM diskgroup discovery string to ORCL:*

Issue
$ srvctl config asm
ASM home: /u01/app/oracle/product/11.2.0/grid_1
ASM listener: LISTENER
Spfile: +DATA/asm/asmparameterfile/registry.253.811613369
ASM diskgroup discovery string: /dev/oracleasm/disks/*

$ sqlplus "/as sysasm"

SQL> show parameter asm_diskstring;

asm_diskstring                       string      /dev/oracleasm/disks/*

Solution
1. Install for OEL 6.x
wget http://download.oracle.com/otn_software/asmlib/oracleasmlib-2.0.12-1.el6.x86_64.rpm

rpm -ihv oracleasmlib-2.0.12-1.el6.x86_64.rpm

2. Check visibility from asm user (grid)

oracleasm-discover

Using ASMLib from /opt/oracle/extapi/64/asm/orcl/1/libasm.so

[ASM Library - Generic Linux, version 2.0.12 (KABI_V2)]

Discovered disk: ORCL:DISK1 [2933593688 blocks (1501999968256 bytes), maxio 512, integrity none]

Discovered disk: ORCL:DISK2 [838858752 blocks (429495681024 bytes), maxio 512, integrity none]

Discovered disk: ORCL:DISK3 [2936005222 blocks (1503234673664 bytes), maxio 512, integrity none]

3. Set asm instance parameter for discover

sqlplus "/as sysasm"

ALTER SYSTEM SET asm_diskstring='ORCL:*' SCOPE=SPFILE;

4. Restart asm instance

srvctl stop asm -f

srvctl start asm

5. Check

srvctl config asm

ASM diskgroup discovery string: ORCL:*

Delete filesystem from OVM

If you want to delete physical disk, but it have filesystem and not visible from GUI.
You can get error:

OVMRU_002026E Cannot use or delete physical disk: FreeBSD (1), it already contains a file system: 
[0004fb0000050000b4ed2daa9d83c7a8]. 

Solution:
1. Connect to OVM CLI


[root@ovm]# ssh -l admin localhost -p 10000

2. Delete filesystem

OVM> delete FileSystem id=0004fb0000050000b4ed2daa9d83c7a8

Oracle vm manager issue

the VM manager version is 3.3.1.1065

Unable to get object 0004fb00001200006c57e05c9aae76d1.img  [db02-disk2] of class class com.oracle.ovm.appfw.generatedpojos.VirtualDiskPOJO: ManagedObjImpl.getObjectFromDb - Unable to get object of type VirtualDiskPOJO using id of 0004fb00001200006c57e05c9aae76d1.img

Unable to get object 0004fb0000120000d4f1d482f1ffa666.img  [db02-disk1] of class class com.oracle.ovm.appfw.generatedpojos.VirtualDiskPOJO: ManagedObjImpl.getObjectFromDb - Unable to get object of type VirtualDiskPOJO using id of 0004fb0000120000d4f1d482f1ffa666.img

Switchover Oracle 10g from Primary to Standby

Validate Primary and standby for swichover.
Verify Managed Recovery is Running (non-broker) on the standby

SQL> SELECT PROCESS
FROM V$MANAGED_STANDBY
WHERE PROCESS LIKE 'MRP%';

Cancel apply delay for the target standby using SQL
On the target physical standby database capture the current delay value

SQL> SELECT DELAY_MINS
FROM V$MANAGED_STANDBY
WHERE PROCESS = 'MRP0';

On the target physical standby database turn off delay if > 0

SQL> RECOVER MANAGED STANDBY DATABASE CANCEL

SQL> ALTER DATABASE RECOVER MANAGED STANDBY DATABASE NODELAY USING CURRENT LOGFILE DISCONNECT FROM SESSION;

Ensure Online Redo Log Files on the Target Physical Standby have been cleared

On the target physical standby run the following query to determine if the online redo logs have not been cleared:

SQL> SELECT DISTINCT L.GROUP#
FROM V$LOG L, V$LOGFILE LF
WHERE L.GROUP# = LF.GROUP#
AND L.STATUS NOT IN (‘UNUSED’, ‘CLEARING’,’CLEARING_CURRENT’);

If the above query returns rows, on the target physical standby issue the following statement for each GROUP# returned:

SQL> ALTER DATABASE CLEAR LOGFILE GROUP ;

Check for Previously Disabled Redo Threads
To determine if this situation exists, on your primary database, first run the following query to determine if there are any threads with a SEQUENCE# greater than 0:

SQL> SELECT THREAD#
FROM V$THREAD
WHERE SEQUENCE# > 0;

On the primary database, determine the current database redo branch:

SQL> SELECT TO_CHAR(RESETLOGS_CHANGE#)
FROM V$DATABASE_INCARNATION
WHERE STATUS = 'CURRENT';

Any of the threads identified by the first query are disabled provided there are no archive log or log history records in the control file of the target physical standby database on the current branch of redo on the primary.

To determine this, substitute the resetlogs_change# from the primary database (found in the second query) into the query below and execute it on the target physical standby database for each thread reported from the first query above.

SQL> SELECT 'CONDITION MET'
FROM SYS.DUAL
WHERE NOT EXISTS (SELECT 1
FROM V$ARCHIVED_LOG
WHERE THREAD# =
AND RESETLOGS_CHANGE# = )
AND NOT EXISTS (SELECT 1
FROM V$LOG_HISTORY
WHERE THREAD# =
AND RESETLOGS_CHANGE# = );

If this query returns a row for any of the threads, you have a disabled thread with a non-zero SEQUENCE# that can prevent a switchover from the primary database to the physical standby database. In this case, you must apply the latest patchset or use one of the following workarounds:

Check if the standby has ever been open read-only
On the target physical standby database run this query:

SQL> SELECT VALUE
FROM V$DATAGUARD_STATS
WHERE NAME='standby has been open';

Verify there are no large GAPS.
Identify the current sequence number for each thread on the primary database

SQL> SELECT THREAD#, SEQUENCE# FROM V$THREAD;

Verify the target physical standby database has applied up to, but not including the logs from the primary query. On the standby the following query should be no more than 1-2 less than the primary query result.

SQL> SELECT THREAD#, MAX(SEQUENCE#)
FROM V$ARCHIVED_LOG
WHERE APPLIED = 'YES'
AND RESETLOGS_CHANGE# = (SELECT RESETLOGS_CHANGE#
FROM V$DATABASE_INCARNATION
WHERE STATUS = 'CURRENT')
GROUP BY THREAD#;

 Verify Primary and Standby TEMP Files Match
On the standby for each temporary tablespace, verify that temporary files associated with that tablespace on the primary database also exist on the standby database. Temp files added after initial standby creation are not propagated to the standby. Run this query on both the primary and target physical standby databases and verify that they match.

SQL> SELECT TMP.NAME FILENAME, BYTES, TS.NAME TABLESPACE
FROM V$TEMPFILE TMP, V$TABLESPACE TS
WHERE TMP.TS#=TS.TS#;

If the queries do not match then you can correct the mismatch now or immediately after the open of the new primary.

To correct now: add or delete a tempfile now requires managed recovery to be stopped and the standby to be open read only. Opening the standby read-only will require a database close and open before becoming the new primary, see “Open the new primary database”.

To correct post-primary-open: see “Correct any tempfile mismatch” step of Switchover

Verify that there is no issue with V$LOG_HISTORY on the Standby

Determine threads that have been active at some point on the primary database:

SQL> SELECT THREAD#, SEQUENCE#
FROM V$THREAD
WHERE SEQUENCE# > 0;

Get the RESETLOGS_CHANGE# from the primary database:
SQL> SELECT RESETLOGS_CHANGE#
FROM V$DATABASE_INCARNATION
WHERE STATUS = 'CURRENT';

On the target physical standby database, get the maximum sequence numbers for each thread from V$LOG_HISTORY:
SQL> SELECT THREAD#, MAX(SEQUENCE#)
FROM V$LOG_HISTORY
WHERE RESETLOGS_CHANGE#=< resetlogs_change# from the primary V$DATABASE_INCARNATION.RESETLOGS_CHANGE# >
GROUP BY THREAD#;

The last SEQUENCE# for each THREAD# from V$LOG_HISTORY on the target physical standby database should be close (the difference in log sequences < 3) to the SEQUENCE# for each THREAD# from V$THREAD on the primary database. If the difference in log sequences is greater than 3 or no row is returned for the thread, you have encountered this problem and should recreate the standby controlfile. See Note 459411.1. If backups are being done on the standby without an RMAN Catalog then backup history will be lost. It is highly recommended to use an RMAN Catalog for all backups.

Verify no old partial Standby Redo Logs on the Standby

Get the RESETLOGS_CHANGE# from the primary database:
SQL> SELECT RESETLOGS_CHANGE#
FROM V$DATABASE_INCARNATION
WHERE STATUS = 'CURRENT';

On the target physical standby database, identify any active standby redo logs (SRL’s)
SQL> SELECT GROUP#, THREAD#, SEQUENCE#
FROM V$STANDBY_LOG
WHERE STATUS = 'ACTIVE'
ORDER BY THREAD#,SEQUENCE#;

On the target physical standby database, identify maximum applied sequence number(s).
SQL> SELECT THREAD#, MAX(SEQUENCE#)
FROM V$LOG_HISTORY
WHERE RESETLOGS_CHANGE#=< resetlogs_change# from the primary V$DATABASE_INCARNATION.RESETLOGS_CHANGE# >
GROUP BY THREAD#;

If there are any active SRL's that have a thread#/sequence# less than the thread#/sequence# returned from the V$LOG_HISTORY (meaning the recovery has progressed beyond the active SRL) query, clear them on the target physical standby.

SQL> RECOVER MANAGED STANDBY DATABASE CANCEL;

SQL> ALTER DATABASE CLEAR LOGFILE GROUP ;

Swichover to DR.
Clear Potential Blocking Parameters & Jobs

Capture current job state on the primary

SQL> SELECT *
FROM DBA_JOBS_RUNNING; [depending on what the running job is, be ready to terminate]

SQL> SELECT OWNER, JOB_NAME, START_DATE, END_DATE, ENABLED
FROM DBA_SCHEDULER_JOBS
WHERE ENABLED='TRUE'
AND OWNER <> 'SYS';

SQL> SHOW PARAMETER job_queue_processes

Note: cron job candidates to be disabled among others: oracle text sync and optimizer, RMAN backups, application garbage collectors, application background agents.

Block further job submission

SQL> ALTER SYSTEM SET job_queue_processes=0 SCOPE=BOTH SID='*';

SQL> EXECUTE DBMS_SCHEDULER.DISABLE( );

Disable any cron jobs that may interfere.

Create Guaranteed Restore Points

Create a guaranteed restore point on the primary
Verify if flashback database is on or a guaranteed restore point already exists

SQL> SELECT FLASHBACK_ON FROM V$DATABASE;

If this query returns “YES” (flashback database is on) or “RESTORE POINT ONLY” (Flashback is on but one can only flashback to an existing guaranteed restore point) then proceed to creating the guaranteed restore point.

NOTE: Unless you have a backport for Bug 7568556, “ACTIVE APPLY RATE SEEN FROM 63MB/S TO 544KB/S AFTER RESTORE POINT ENABLED”, you should not have just a guaranteed restore point only (V$DATABASE.FLASHBACK_ON=”RESTORE POINT ONLY”) and ensure that flashback database is also on (V$DATABASE.FLASHBACK_ON=”YES”) when creating a guaranteed restore point.

If this query returns “NO” then you need to turn on flashback database before creating the guaranteed restore point. This requires the database to be mounted.

See Enabling Logging for Flashback Database for those steps.

Create the guaranteed restore point

SQL> CREATE RESTORE POINT SWITCHOVER_START_GRP GUARANTEE FLASHBACK DATABASE;
Create a guaranteed restore point on the standby
Verify if flashback database is on or a guaranteed restore point already exists

SQL> SELECT FLASHBACK_ON FROM V$DATABASE;

If this query returns “YES” (flashback database is on) or “RESTORE POINT ONLY” (Flashback is on but one can only flashback to an existing guaranteed restore point) then proceed to creating the guaranteed restore point.

If this query returns “NO” then you need to turn on flashback database before creating the guaranteed restore point. This requires being in the MOUNT state.

See Enabling Logging for Flashback Database for those steps.

Create the guaranteed restore point

SQL> ALTER DATABASE RECOVER MANAGED STANDBY DATABASE CANCEL;
SQL> CREATE RESTORE POINT SWITCHOVER_START_GRP GUARANTEE FLASHBACK DATABASE;

SQL> ALTER DATABASE RECOVER MANAGED STANDBY DATABASE USING CURRENT LOGFILE DISCONNECT;

Verify that the primary database can be switched to the standby role

Query the SWITCHOVER_STATUS column of the V$DATABASE view on the primary database. For example:
SQL> SELECT SWITCHOVER_STATUS
FROM V$DATABASE;

SWITCHOVER_STATUS
-----------------
TO STANDBY

A value of TO STANDBY or SESSIONS ACTIVE (requires the WITH SESSION SHUTDOWN clause on the switchover command) indicates that the primary database can be switched to the standby role. If neither of these values is returned, a switchover is not possible because redo transport is either mis-configured or is not functioning properly

Switchover the primary to a standby database

SQL> ALTER DATABASE COMMIT TO SWITCHOVER TO STANDBY WITH SESSION SHUTDOWN;

If an ORA-16139 is encountered, as long as V$DATABASE.DATABASE_ROLE=’PHYSICAL STANDBY’, then you can proceed. A common case where this can occur is when there are a large number of data files, greater than 1,000, the apply of the EOR log will timeout.. Once managed recovery is started on the new standby it will recover.

If the role was not changed then you need to cancel the switchover and review the alert logs and trace files further.

Verify the standby has received the end-of-redo (EOR) log(s)

MRP0: Media Recovery Complete: End-Of-REDO (sfs_stby1)
Resetting standby activation ID 2821924805 (0xa83327c5)
Mon Nov 3 06:53:21 2008
MRP0: Background Media Recovery process shutdown (sfs_stby1)
Mon Nov 3 06:53:22 2008
SUCCESS: diskgroup REGR was dismounted
Mon Nov 3 06:53:22 2008
Switchover: Complete - Database shutdown required (sfs_stby1)
Mon Nov 3 06:53:22 2008
Completed: ALTER DATABASE COMMIT TO SWITCHOVER TO STANDBY WITH SESSION SHUTDOWN

And correspondingly in the standby alert log file you should see messages like this:

Mon Nov 3 06:53:17 2008
Media Recovery Log +REGR2/sfs/archivelog/2008_11_03/thread_1_seq_21.3819.669797593
Identified End-Of-Redo for thread 1 sequence 21
Mon Nov 3 06:53:17 2008
Media Recovery End-Of-Redo indicator encountered
Mon Nov 3 06:53:17 2008
Media Recovery Applied until change 8338654
Mon Nov 3 06:53:17 2008
MRP0: Media Recovery Complete: End-Of-REDO (sfs1)
Resetting standby activation ID 2821924805 (0xa83327c5)
Mon Nov 3 06:53:19 2008
MRP0: Background Media Recovery process shutdown (sfs1)

Verify that the standby database can be switched to the primary role

Query the SWITCHOVER_STATUS column of the V$DATABASE view on the standby database. For example:
SQL> SELECT SWITCHOVER_STATUS
FROM V$DATABASE;

SWITCHOVER_STATUS
-----------------
TO PRIMARY

A value of TO PRIMARY or SESSIONS ACTIVE indicates that the standby database is ready to be switched to the primary role. If neither of these values is returned, verify that redo apply is active and that redo transport is configured and working properly. Continue to query this column until the value returned is either TO PRIMARY or SESSIONS ACTIVE.

Switchover the standby database to a primary

SQL> ALTER DATABASE COMMIT TO SWITCHOVER TO PRIMARY;

Open the new primary database:

SQL> ALTER DATABASE OPEN;

!!!!!!!!After Switchover need delete restore points on both databases!!!!!!
DROP RESTORE POINT SWITCHOVER_START_GRP;

How to Remove non-exist Storage LUNs from Oracle VM Manager

First Try:
1. Execute below command on all Oracle VM Servers which had been assigned with the non-exist Storage LUN:
# scsi-rescan
2. Execute below command on all Oracle VM Servers to flush all unused multipath device maps:
# multipath -F
3. Log into Oracle VM Manager BUI again, under Storage tab, select the SAN Server under which the non-exist Storage LUN is listed, Click button "Refresh SAN Server".

if above method could not fix the problem try:
REGENERATE YOUR DATABASE

# service ovmm stop

# cd /u01/app/oracle/ovm-manager-3/ovm_tools/bin/

# ./BackupDatabase.py

you backup have stored in /u01/app/oracle/mysql/dbbackup

The values for some of the command line switches for the ovm_upgrade.sh command can be found in the /u01/app/oracle/ovm-manager-3/.config file

# cd /u01/app/oracle/ovm-manager-3/bin

# ./ovm_upgrade.sh --deletedb --dbhost=localhost --dbport=49500 --dbsid=ovs --dbuser=ovs --dbpass=xxxx

to prevent error: AUTH_000002:Connection to manager failed: Certificate authentication failed

execute

# ./configure_client_cert_login.sh

# service ovmm start

use tools from Doc ID 1555915.1 to assist in populating the friendly object names

# ./ovm_reporestore -u admin -p Wltd0www -h localhost -F /u01/app/oracle/mysql/dbbackup/OVMModelExport-YYYYMMDD_xxxxxx.xml -a -R

Links:
How to Remove non-exist Storage LUNs from Oracle VM Manager (Doc ID 1680025.1)
Oracle VM Manager 3.x: How to Recover from "ObjectNotFoundException" and "ONF Exception" Errors (Doc ID 1555915.1)

Oracle VM Manager 3.3.* vmpinfo

[root@OracleVMmgmt ~]# cd /u01/app/oracle/ovm-manager-3/ovm_tools/support/

[root@OracleVMmgmt support]# ./vmpinfo3.sh --username=admin servers=oraclevm2,oraclevm1
Enter OVM Manager Password:

Change ip network infrastructure on ORACLE RAC

How to Modify Public Network Information including VIP in Oracle Clusterware (Doc ID 276434.1)
How to update the IP address of the SCAN VIP resources (ora.scan{n}.vip) (Doc ID 952903.1)
How to Modify Private Network Information in Oracle Clusterware (Doc ID 283684.1)

Procedure to Change the Hostname of the Virtual Machine Oracle E-Business Suite, Release 12.2.4

1. Log in as the oracle user.

2. Source the runtime environment:
$ source  /u01/install/VISION/EBSapps.env Run

3. Execute the following command:
$ $INST_TOP/admin/scripts/adadminsrvctl.sh start

4. Execute the following command:
$ $INST_TOP/admin/scripts/adpreclone.pl appsTier

5. Source the patch environment in the patch file system:
$ source /u01/install/VISION/EBSapps.env Patch

6. Execute the following command:

$ $INST_TOP/admin/scripts/adadminsrvctl.sh start forcepatchfs

7. Execute the following command:
$ $INST_TOP/admin/scripts/adpreclone.pl appsTier

8. Execute the following command to stop the Oracle E-Business Suite application tier:
$ /u01/install/VISION/scripts/stopvisionapps.sh

9. Execute the following command to stop the Oracle E-Business Suite database tier:
$ /u01/install/VISION/scripts/stopvisiondb.sh

10. Execute either one of the following commands as the root user:
# /u01/install/scripts/configstatic.sh
Or
# /u01/install/scripts/configdhcp.sh

11. Execute the following command as the root user:
# /u01/install/VISION/scripts/visiondbconfig.sh

12. Execute the following command as the root user:
# /u01/install/VISION/scripts/visionappsconfig.sh

ps:

weblogic password - welcome1

apps password - apps

pss:

i got error

Target System Port Pool [0-99] : RC-00201: Error: Not a valid port pool number

Spacewalk tips

SERVER
Install the Spacewalk repository
rpm -Uvh http://yum.spacewalkproject.org/2.2/RHEL/6/x86_64/spacewalk-repo-2.2-1.el6.noarch.rpm
Install the following required additional repositories
cat > /etc/yum.repos.d/jpackage-generic.repo << EOF
[jpackage-generic]
name=JPackage generic
#baseurl=http://mirrors.dotsrc.org/pub/jpackage/5.0/generic/free/
mirrorlist=http://www.jpackage.org/mirrorlist.php?dist=generic&type=free&release=5.0
enabled=1
gpgcheck=1
gpgkey=http://www.jpackage.org/jpackage.asc
EOF

Install EPEL

rpm -Uvh http://dl.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm

Install spacewalk for postgresql

yum install spacewalk-setup-postgresql

yum install spacewalk-postgresql -y

CLIENT

Install the Spacewalk client yum repository

rpm -Uvh http://yum.spacewalkproject.org/2.2-client/RHEL/6/x86_64/spacewalk-client-repo-2.2-1.el6.noarch.rpm

Install client packages

yum install rhn-client-tools rhn-check rhn-setup rhnsd m2crypto yum-rhn-plugin

Register your CentOS or Red Hat Enterprise Linux system to Spacewalk using the activation key you created earlier

 rhnreg_ks --serverUrl=http://spacewalk.core.kran/XMLRPC --activationkey=1-a7f734ebf3911f03e34f834376b34ab4

yum install osad rhncfg rhncfg-actions rhncfg-client rhncfg-managemen

Unregister
rm -f /etc/sysconfig/rhn/systemid
Oracle spacewalk client
http://public-yum.oracle.com/repo/OracleLinux/OL6/spacewalk20/client/x86_64/getPackage/spacewalk-client-repo-2.0-3.el6.noarch.rpm

Redhat spacewalk client
rpm -ihv http://yum.spacewalkproject.org/2.2-client/RHEL/6/x86_64/spacewalk-client-repo-2.2-1.el6.noarch.rpm

##########
service osad start
Starting osad: Invalid Cert Error:

ERROR: unhandled exception occurred: (Expired certificate /usr/share/rhn/RHNS-CA-CERT).

                                                           [FAILED]

you need to grab your spacewalk server's ssl cert from the pub location (http://<your-server>/pub ) and put it under /usr/share/rhn on your client. Then modify the sslCACert=/usr/share/rhn/RHNS-CA-CERT field in /etc/sysconfig/rhn/up2date file to /usr/share/rhn/RHN-ORG-TRUSTED-SSL-CERT.
https://www.redhat.com/archives/spacewalk-list/2008-July/msg00271.html

Windows 8.1 x64 and Oracle Client x86

Before install client.
Install http://www.microsoft.com/en-US/download/details.aspx?id=26999