Netbackup daemons/programs

Netbackup daemons/programs Daemon/Program Description bp Character based menu drive interface for user directed backups, archives, and restores.  Started by: /usr/openv/netbackup/bin/bp Stopped by: Exiting the interface. Activity log: /usr/openv/netbackup/logs/bp bpadm Character based menu driven interface for configuring and managing Netbackup. Started by: /usr/openv/netbackup/bin/bpadm Stopped by: Exiting the interface. Activity log: /usr/openv/netbackup/logs/admin/admin.log on the server

Oracle Web Logic init scrips

Simple script to start WLS.

There is one difficulty with console return after calling startWebLogic.sh and startManagedWebLogic.sh. Regarding to it I had to run it in background. If startup script runs in background it returns code which means successful completion of "&" operation. If you need to make sure services started correctly code you have two ways:

1. do it in WLST
2. try to catch "<Notice> <WebLogicServer> <BEA-000360> <Server started in RUNNING mode>

This script installed at development server. As it is not a production variant and have limited number of domains I do not have to spend much time WLST script coding (which is actually python). Lately if it would be necessary I will certainly do advanced script to automatically start/stop WLS domains. There are some simple samples.

#!/bin/sh

# using of LOOPs is regarded to future improovements. May well you will
# have several Admin servers and Several Managed Servers. Just change
# the code to work with arrays.

# The only problem is to check successfull start of Java instances which is in background.

LOG_ADM="/tmp/ora_wls-admin"
LOG_MNGR="/tmp/ora_wls-nmgr"
MW_HOME="/u01/oracle/mw"
WLS_DOMAINS_ADM="base_domain"
WLS_DOMAINS_MNGD="osb_server1 Server-0"
#for compatibility with sh
ADNAME="basedomain"
MDNAME="osbserver1"

# write svn information to a header

version() {

Log " category System scripts"
Log " author V.Bychkov"
Log " changedby $Author: bes $"
Log " version SVN: $Id: oracle.wls-middleware 1002 2011-07-04 04:26:20Z bes $"
Log " revision SVN: $Revision: 1002 $"

Log " link       $HeadURL: file:///opt/svn/projects/scripts/oracle/init.d/oracle.wls-middleware $"
Log "# date       $Date: 2011-07-04 14:26:20 +1000 (Mon, 04 Jul 2011) $"

}

Delete domain in Oracle WebLogic Server

There is no special tool for that. If we are talking about 11g, just delete domain directory.

For example in my configuration it is:
rm -rf /u01/oracle/mw/user_projects/domains/base_domain

it is also necessary to remove deleted domain from domains list:

[oracle@db02-app01 zone:app01]/u01/oracle/mw/wls_10.3$ cat common/nodemanager/nodemanager.domains
#
#Thu Jun 30 15:53:24 MSD 2011
wl_server=/u01/oracle/mw/wls_10.3/samples/domains/wl_server
medrec-spring=/u01/oracle/mw/wls_10.3/samples/domains/medrec-spring
base_domain=/u01/oracle/mw/user_projects/domains/base_domain
servicebus=/u01/oracle/mw/osb_11.1/../user_projects/samples/domains/servicebus
medrec=/u01/oracle/mw/wls_10.3/samples/domains/medrec

It is a good practice for me to make backup of file before modification.

Start Oracle Service Bus 11g

To start OSB firstly need to start a WLS with correct memory settings and configure WLS Servers/Domains.

Execute /opt/bea/wlserver_103/common/bin/config.sh to configure domain.

1. choose "Oracle Service Bus for Developers - 11.1.1.4 [Oracle_OSB1]" option during configuration process. After completing this step you will get configured base_domain domain with osb_server1 server instance. For the first time I decided to accept default names.

2. start WL Server with Administration Console
export USER_MEM_ARGS="-Xms1280m -Xmx1280m -XX:PermSize=128m -XX:MaxPermSize=512m"
/u01/oracle/mw/user_projects/domains/base_domain$ ./bin/startWebLogic.sh

3. Start Managed Server osb_server1
/u01/oracle/mw/user_projects/domains/base_domain$ ./bin/startManagedWebLogic.sh osb_server1

Working example of Oracle Service Bus 11g silent installation.

I am administering some clients remotely, so it is vital to be able to install and configure with no GUI because I currently have ping time=391 ms to my client.

All that we need:
1. prepare
2. install Oracle WebLogic Server
3. install Oracle Service Bus then.

let's go.

1. some preparation:

#create users and groups
groupadd -g 100 oinstall
groupadd -g 110 dba
useradd -u 100 -g oinstall -G dba oracle
passwd oracle

# create /var/opt/oracle/oraInst.loc file
echo "inventory_loc=/u01/oracle
inst_group=oinstall" > /var/opt/oracle/oraInst.loc


2. Install Oracle WebLogic Server 11g
(all output included)

# create silent xml file:
<bea-installer>
     <input-fields>

       <data-value name="BEAHOME" value="/u01/oracle/mw" />
       <data-value name="WLS_INSTALL_DIR" value="/u01/oracle/mw/wls_10.3" />
       <data-value name="COMPONENT_PATHS" value="WebLogic Server" />
       <data-value name="INSTALL_NODE_MANAGER_SERVICE" value="yes" />
       <data-value name="NODEMGR_PORT" value="5559" />
       <data-value name="INSTALL_SHORTCUT_IN_ALL_USERS_FOLDER" value="yes"/>

</input-fields>
</bea-installer>

liboraInstaller.so: wrong ELF class: ELFCLASS64 (Possible cause: architecture word width mismatch)

Machine Info: SunOS XXX 5.10 Generic_142909-17 sun4u sparc SUNW,Sun-Fire-V440

I run it on Sun Fire which is 64 bit.

[oracle@XXX]/u01/app/oracle/product/wls/oms11g/OPatch$ opatch -invPtrLoc /u01/app/oracle/product/wls/oms11g/oraInst.loc -local -oh /u01/app/oracle/product/wls/oms11g/../oracle_common /u01/app/oracle/product/wls/oms11g/install/oneoffs/111010/9329357
Invoking OPatch 11.1.0.8.0
java.lang.UnsatisfiedLinkError: /u01/app/oracle/product/wls/oracle_common/oui/lib/solaris/liboraInstaller.so: ld.so.1: java: fatal: /u01/app/oracle/product/wls/oracle_common/oui/lib/solaris/liboraInstaller.so: wrong ELF class: ELFCLASS64 (Possible cause: architecture word width mismatch)
        at java.lang.ClassLoader$NativeLibrary.load(Native Method)
        at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1803)
        at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1699)
        at java.lang.Runtime.load0(Runtime.java:770)
        at java.lang.System.load(System.java:1003)
        at oracle.sysman.oii.oiip.osd.unix.OiipuUnixOps.loadNativeLib(OiipuUnixOps.java:402)
        at oracle.sysman.oii.oiip.osd.unix.OiipuUnixOps.(OiipuUnixOps.java:125)
        at oracle.sysman.oii.oiip.oiipg.OiipgEnvironment.ssgetUidux(OiipgEnvironment.java:528)
        at oracle.sysman.oii.oiix.OiixEnvironmentOps.ssgetUidux(OiixEnvironmentOps.java:159)
        at oracle.opatch.OUIReplacer.isRootAccess(OUIReplacer.java:716)
        at oracle.opatch.OPatch.main(OPatch.java:445)
Exception in thread "main" java.lang.UnsatisfiedLinkError: no oraInstaller in java.library.path
        at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1734)
        at java.lang.Runtime.loadLibrary0(Runtime.java:823)
        at java.lang.System.loadLibrary(System.java:1028)
        at oracle.sysman.oii.oiip.osd.unix.OiipuUnixOps.loadNativeLib(OiipuUnixOps.java:420)
        at oracle.sysman.oii.oiip.osd.unix.OiipuUnixOps.(OiipuUnixOps.java:125)
        at oracle.sysman.oii.oiip.oiipg.OiipgEnvironment.ssgetUidux(OiipgEnvironment.java:528)
        at oracle.sysman.oii.oiix.OiixEnvironmentOps.ssgetUidux(OiixEnvironmentOps.java:159)
        at oracle.opatch.OUIReplacer.isRootAccess(OUIReplacer.java:716)
        at oracle.opatch.OPatch.main(OPatch.java:445)

OPatch failed with error code 1

Explore:

ORA-27102: out of memory when DBCA -createDatabase

[oracle@xxx zone:xx]~/Install/solaris.sparc64_11gR2_database/database$ $ORACLE_HOME/bin/dbca -silent -createDatabase -templateName General_Purpose.dbc -gdbName $SID -sid $SID -sysPassword $PASSWORD -systemPassword $PASSWORD -sysmanPassword $PASSWORD -dbsnmpPassword $PASSWORD -emConfiguration LOCAL -storageType FS -datafileJarLocation $ORACLE_HOME/assistants/dbca/templates -characterset WE8ISO8859P1 -obfuscatedPasswords false -sampleSchema false -oratabLocation /var/opt/oracle/oratab
Copying database files
1% complete
2% complete
4% complete
100% complete
Look at the log file "/u01/app/oracle/cfgtoollogs/dbca/XX01/XX01.log" for further details.
[oracle@xxx zone:xx]~/Install/solaris.sparc64_11gR2_database/database$ less /u01/app/oracle/cfgtoollogs/dbca/XX01/XX01.log
Copying database files
DBCA_PROGRESS : 1%
DBCA_PROGRESS : 2%
ORA-27102: out of memory

DBCA_PROGRESS : 4%
ORA-01034: ORACLE not available

ORA-01034: ORACLE not available

DBCA_PROGRESS : 100%

solution

WebLogic Server installation, The local BEA product registry is corrupted error

[oracle@em.e-soft.ru zone:em]~/Install$ ./wls1034_solaris32.bin -mode=silent -silent_xml=./silent.xml -Djava.security.egd=file:/dev/./urandom
Extracting 0%....................................................................................................100%
The local BEA product registry is corrupted. Please select another Middleware Home or contact Oracle Support
** Error during execution, error code = 65280.

solution:

no backup or copy of datafile XX found to restore when backup actually exists

Attempting to restore catalogued backup on another host gives following error:

executing command: SET NEWNAME

executing command: SET NEWNAME

Starting restore at 20-FEB-11

released channel: ch01
released channel: ch02
RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-03002: failure of restore command at 02/20/2011 20:35:40
RMAN-06026: some targets not found - aborting restore
RMAN-06023: no backup or copy of datafile XX1 found to restore
RMAN-06023: no backup or copy of datafile XX2 found to restore

but when I did list backup, it shown backup exist.

Erorr was the consequence of  catalog command. There was attempt to catalog backupset with dificult name like /mnt/share/backup/oracle/DBNAME/local-lvl-0-db_3673504914-DB01-20110220-38400-1-1 which catalog was not able to compile.
solution:

silent database creation

$ dbca -silent -createDatabase -templateName Transaction_Processing.dbc -gdbname rcat -sid rcat1 -datafileJarLocation /export/home/oracle/oracle/product/10.2.0/db/assistants/dbca/templates -datafileDestination /export/home/oracle/oracle/product/10.2.0/db/oradata -responseFile NO_VALUE -characterset CL8MSWIN1251
Copying database files
1% complete
3% complete

delete/add Oracle Home from Central Inventory

some times it happens...:

$ ./OPatch/opatch lsinv
Invoking OPatch 10.2.0.4.3

Oracle Interim Patch Installer version 10.2.0.4.3
Copyright (c) 2007, Oracle Corporation.  All rights reserved.


Oracle Home       : /export/home/oracle/oracle/product/10.2.0/db
Central Inventory : /u01/oracle/oraInventory
   from           : /var/opt/oracle/oraInst.loc
OPatch version    : 10.2.0.4.3
OUI version       : 10.2.0.4.0
OUI location      : /export/home/oracle/oracle/product/10.2.0/db/oui
Log file location : /export/home/oracle/oracle/product/10.2.0/db/cfgtoollogs/opatch/opatch2011-02-02_14-30-01PM.log

List of Homes on this system:

  Home name= OraDb10g_home2, Location= "/d03/oracle/product/10.2.0/Db1"
Inventory load failed... OPatch cannot load inventory for the given Oracle Home.
Possible causes are:
   Oracle Home dir. path does not exist in Central Inventory
   Oracle Home is a symbolic link
   Oracle Home inventory is corrupted
LsInventorySession failed: OracleHomeInventory gets null oracleHomeInfo

OPatch failed with error code 73

Solution:

alter database open resetlogs error ORA-00349

After restore & recovery I got error:

alter database open resetlogs
*
ERROR at line 1:
ORA-00349: failure obtaining block size for
'+ASMDG01/XXX/onlinelog/redo_g1_a'

in alert.log:

Sun Jan 30 06:01:07 2011

Errors in file /oracle/admin/XXX/udump/XXX_ora_3115.trc:

ORA-00313: open failed for members of log group 4 of thread 1

ORA-00312: online log 4 thread 1: '+DG01/XXX/redo_g4_b'

ORA-17503: ksfdopn:2 Failed to open file +DG01/XXX/redo_g4_b

ORA-15173: entry 'redo_g4_b' does not exist in directory 'XXX'

ORA-00312: online log 4 thread 1: '+DG01/XXX/redo_g4_a'

ORA-17503: ksfdopn:2 Failed to open file +DG01/XXX/redo_g4_a

ORA-15173: entry 'redo_g4_a' does not exist in directory 'XXX'

Solution:

This database instance failed to turn on flashback database

RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-03002: failure of recover command at 01/29/2011 00:40:41
ORA-00283: recovery session canceled due to errors
RMAN-11003: failure during parse/execution of SQL statement: alter database recover if needed
 start until cancel using backup controlfile
ORA-00283: recovery session canceled due to errors
ORA-38760: This database instance failed to turn on flashback database

Solution:

ORA-29701: unable to connect to Cluster Manager

SQL> startup
ORA-29701: unable to connect to Cluster Manager

[oracle@xxx]/oracle/product/10.2/bin$ ./localconfig delete

You must be logged in as root to run ./localconfig.

Log in as root and restart ./localconfig execution.

exit from oracle account or execute su/sudo

bash-3.00# cd /oracle/product/10.2/bin/

bash-3.00# ./localconfig delete

Stopping CSSD.

Unable to communicate with the CSS daemon.

Shutdown has begun. The daemons should exit soon.

bash-3.00# ./localconfig add

Successfully accumulated necessary OCR keys.

Creating OCR keys for user 'root', privgrp 'root'..

Operation successful.

Configuration for local CSS has been initialized

Adding to inittab

Startup will be queued to init within 30 seconds.

Checking the status of new Oracle init process...

Expecting the CRS daemons to be up within 600 seconds.

Giving up: Oracle CSS stack appears NOT to be running.

Oracle CSS service would not start as installed

Automatic Storage Management(ASM) cannot be used until Oracle CSS service is started

2009.04.27: Oracle in Russia, RAC days.

Thread 1 cannot allocate new log, sequence...

I sometimes expecting this message in alert.log file:
Thread 1 cannot allocate new log, sequence nnnnn

It can be skipped because it is not harmful but I decided to explore.

SQL> alter system set log_checkpoints_to_alert=true;

SQL> alter system checkpoint;

let's get some outcome at first.

"Error: ORA-16541: site is not enabled" after ENABLE CONFIGURATION command in Data Guard Brocker

If you see something like that:

DGMGRL> connect /
Connected.
DGMGRL> DISABLE CONFIGURATION;
Disabled.
DGMGRL> edit database "dbXX" set property 'LocalListenerAddress'='(ADDRESS=(PROTOCOL=tcp)(HOST=XXX.XX.au)(PORT=1521))';
Error: ORA-16541: site is not enabled

Configuration details cannot be determined by DGMGRL
DGMGRL> ENABLE CONFIGURATION;
Error: ORA-16541: site is not enabled

Configuration details cannot be determined by DGMGRL
DGMGRL> exit

It may well happen after switchover operation.

ORA-16570: operation requires restart of database ""

After successfull switchover I got  the message
ORA-16570: operation requires restart of database "".

DGMGRL> SWITCHOVER TO dbXXsb;
Performing switchover NOW, please wait...
Operation requires shutdown of instance "dbXX" on database "dbXX"
Shutting down instance "dbXX"...
ORA-01031: insufficient privileges

You are no longer connected to ORACLE
Please connect again.
Unable to shut down instance "dbXX"
You must shut down instance "dbXX" manually
Operation requires shutdown of instance "dbXX" on database "dbXXsb"
You must shut down instance "dbXX" manually
Operation requires startup of instance "dbXX" on database "dbXX"
You must start instance "dbXX" manually
Operation requires startup of instance "dbXX" on database "db07sb"
You must start instance "dbXX" manually
Switchover succeeded, new primary is "dbXXsb"
DGMGRL> SHOW CONFIGURATION;
not logged on
DGMGRL> connect /
Connected.
Error:
ORA-16570: operation requires restart of database ""
ORA-06512: at "SYS.X$DBMS_DRS", line 180
ORA-06512: at line 1

DGMGRL>
DGMGRL> exit


shutdown immediate;

startup mount;
DG Brocker starts all others.

Warning: ORA-16610: command 'Broker automatic health check' in progress

Continuous getting  "Warning: ORA-16610: command 'Broker automatic health check' in progress" message after successful switchover operation.

DGMGRL> show configuration;

Configuration
Name: dbXX.rtcomm.ru
Enabled: YES
Protection Mode: MaxPerformance
Fast-Start Failover: DISABLED
Databases:
dbXX - Physical standby database
dbXXsb - Primary database

Current status for "dbXX.rtcomm.ru":
Warning: ORA-16610: command 'Broker automatic health check' in progress


the only way was to restart DG Brocker
ALTER SYSTEM SET DG_BROKER_START=FALSE;
ALTER SYSTEM SET DG_BROKER_START=TRUE;

The message disappeared.

Parallelism analyzing

set :

set timi on
set autot on

to chose the difference betwee:

select count(*) from all_users au;

 and

select /*+ parallel (au,10) */ count(*) from all_users au;

some useful information:

select * from V$PX_SESSION;

SELECT NAME, VALUE FROM GV$SYSSTAT
WHERE UPPER (NAME) LIKE '%PARALLEL OPERATIONS%'
OR UPPER (NAME) LIKE '%PARALLELIZED%' OR UPPER (NAME) LIKE '%PX%';

SELECT * FROM V$PX_PROCESS;

SELECT * FROM V$PX_PROCESS_SYSSTAT;

parameter to reduse/increase parallelism in Oracle is PARALLEL_MAX_SERVERS.