Forgot your password?    
+ Reply to Thread
Results 1 to 1 of 1

Thread: Add Node to a Cluster

  1. #1
    Expert Oracle Administrator
    Join Date
    Oct 2011
    New Delhi, India

    Add Node to a Cluster

    Step 1: Account for Dependencies and Prerequisites

    The new node should have the

    - same version of the operating system as the existing nodes, including all patches required for Oracle.
    - maintain the current naming convention.
    - have the same required rpms installed
    - have the kernel parameters set to the same values as the other nodes
    - have the same oracle user and group
    - have the GID of user oracle are identical to that of the other RAC nodes.
    - have the same ulimits for the oracle user as the other nodes

    1) Add the entries for the new host (public ip,private ip and vip) in the /etc/hosts for all the servers

    2) Setup the user equivalence with SSH.

    When adding nodes to the cluster, Oracle copies files from the node where the installation was originally performed to the new node in the cluster. Such a copy process is performed either by using ssh protocol where available or by using the remote copy ( rcp). In order for the copy operation to be successful, the oracle user on the RAC node must be able to login to the new RAC node without having to provide a password or passphrase.

    ensure passwordless command execution

    ssh rac1 hostname
    ssh rac3 hostname
    ssh rac1-priv hostname
    ssh rac3-priv hostname

    Note: When performing these tests for the first time, the operating system will display a key and request the user to accept or decline. Enter ‘Yes’ to accept and register the key. Tests should be performed on all other nodes across all interfaces in the cluster except for the VIP.

    Step 2: Install Oracle Clusterware

    Oracle Clusterware is already installed on the cluster; the task here is to add the new node to the clustered configuration. This task is performed by executing the Oracle provided utility called addnode located in the Clusterware’s home oui/bin directory. Oracle Clusterware has two files (Oracle cluster repository, OCR; and Oracle Cluster Synchronization service, CSS, voting disk) that contain information concerning the cluster and the applications managed by the Oracle Clusterware. These files need to be updated with the information concerning the new node. The first step in the clusterware installation process is to verify if the new node is ready for the install.

    Cluster Verification. In Oracle Database 10g Release 2, Oracle introduced a new utility called Cluster Verification Utility (CVU)as part of the clusterware software. Executing the utility using the appropriate parameters determines the status of the cluster. At this stage, before beginning installation of the Oracle Clusterware, you should perform two verifications:

    If the hardware and operating system configuration is complete:

    a) cluvfy stage -post hwos -n rac1,rac3

    b) cluvfy stage -pre crsinst -n rac1,rac3

    Step 3: Configure Oracle Clusterware

    Using the OUI requires that the terminal from where this installer is ran be X-windows compatible. If not, an appropriate X-windows emulator should be installed and the emulator invoked using the DISPLAY command using the following syntax.

    export DISPLAY=<client IP address>:0.0

    The next step is to configure the Clusterware on the new node rac3.
    For this, as mentioned earlier, Oracle has provided a new executable called located in the <Clusterware Home>/oui/bin directory.

    Execute the script <Clusterware Home>/oui/bin/

    Welcome - click on Next.

    Specify Cluster Nodes to Add to Installation - In this screen, OUI lists existing nodes in the cluster and in the bottom half of the screen lists the new node(s) information to be added in the appropriate columns. Once the information is entered click on Next.

    Public Node Name Private Node Name Virtual Host Name
    rac3 rac3-priv rac3-vip

    Cluster Node Addition Summary - Verify the new node is listed under the “New Nodes” drilldown and click on Install.
    Once all required clusterware components are copied from rac1 to rac3, OUI prompts to execute three files:
    /usr/app/oracle/oraInventory/ on node rac3

    [root@rac3 oraInventory]# ./
    Changing permissions of /u01/app/oracle/oraInventory to 770.
    Changing groupname of /u01/app/oracle/oraInventory to dba.
    The execution of the script is complete

    Execute $ORA_CRS_HOME/install/ on node rac1.
    (The file will add the new node information to the OCR using the srvctl utility. Note the srvctl command with the nodeapps parameter at the end of the script output below.)

    [root@rac1 install]# ./

    $ORA_CRS_HOME/ on node rac3.

    [root@rac3]# ./

    invoke VIPCA also as root. (VIPCA will also configure GSD and ONS resources on the new node.)

    Welcome - click on Next.
    On completion of the Oracle Clusterware installation, the following files are created in their respective directories.

    Clusterware files:

    [root@rac3 root]# ls -ltr /etc/init.d/init.*

    The operating system provided inittab file is updated with the following entries.

    [root@rac3 root]# tail -5 /etc/inittab

    Click on OK after all the listed scripts have run on all nodes.
    End of Installation – Click on Exit.
    Verify if the Clusterware has all the nodes registered using the olsnodes command.
    [oracle@rac1 oracle]$ olsnodes

    [oracle@rac1 oracle]$
    Verify if the cluster services is started, using the crs_stat command.
    [oracle@rac1 oracle]$ crs_stat -t
    Name Type Target State Host

    Verify if the VIP services are configured at the OS level. The virtual IP address is configured and added to the OS network configuration and the network services are started. The VIP configuration could be verified using the ifconfig command at the OS level.

    [oracle@rac3 oracle]$ ifconfig -a

    Note: eth0:1 indicates that it is a VIP address for the basic host eth0. When the node fails eth0:1 will be moved to a surviving node in the cluster. The new identifier for the VIP on the failed over server will be indicated by eth0:2 or higher, depending on what other nodes have failed in the cluster and the VIP has migrated.

    Step 4: Install Oracle Software

    The next step is to install the Oracle software on the new node. As mentioned earlier, Oracle has provided a new executable called located in the $ORACLE_HOME/oui/bin directory.

    Execute the script $ORACLE_HOME/oui/in/
    Welcome - click on Next.
    Specify Cluster Nodes to Add to Installation - In this screen OUI lists existing nodes in the cluster and in the bottom half of the screen lists the new node(s). Select the node rac3. Once the information is entered, click on Next.

    Cluster Node Addition Summary - Verify if the new node is listed under the “New Nodes” drilldown and click on the Install button.
    When copy of Oracle software to node rac3 is complete, OUI will prompt you to execute /u01/app/oracle/product/10.2.0/db_1/ script from another window as the root user on the new node(s) in the cluster.

    [root@rac3 db_1]# ./
    Running Oracle10 script...

    Step 5: Add New Instance(s)

    DBCA has all the required options to add additional instances to the cluster.

    Make a full cold backup of the database before commencing the upgrade process.
    Oracle Clusterware should be running on all nodes.
    Welcome screen - select Oracle Real Application Cluster database and click on Next.

    Step 1 of 7: Operations - A list of all operations that can be performed using the DBCA are listed. Select Instance Management and click on Next.

    Step 2 of 7: Instance Management - A list of instance management operations that can be performed are listed. Select Add an Instance and click on Next.

    Step 3 of 7: List of cluster databases - A list of clustered databases running on the node are listed. In this case the database running on node rac1 is devdb; select this database. In the bottom part of the screen, DBCA requests you to “Specify a user with SYSDBA system privileges”:
    Username: sys
    Password: < > and click on Next.

    Step 4 of 7: List of cluster database instances - The DBCA lists all the instances currently available on the cluster. Verify if all instances are listed and click on Next.

    Step 5 of 7: Instance naming and node selection - DBCA lists the next instance name in the series and requests the node on which to add the instance. In our example the next instance name is devdb3 and the node name is rac3. Click on Next after making the appropriate selection. At this stage there is a small pause before the next screen appears as DBCA determines the current state of the new node and what services are configured on the existing nodes.

    Step 6 of 7: Database Services - If the current configuration has any database services configured, this screen will appear (else is skipped). Make the appropriate selections and click on Next when ready.

    Step 7 of 7: Instance Storage - In this screen, DBCA will list the instances specific files such as undo tablespaces, redo log groups, and so on. Verify if all required files are listed and click on Finish.

    Database Configuration Assistant: Summary - After verifying the summary, click on OK to begin the software installation.
    DBCA verifies the new node rac3, and as the database is configured to use ASM, prompts with the message “ASM is present on the cluster but needs to be extended to the following nodes: [rac3]. Do you want ASM to be extended?” Click on Yes to add ASM to the new instance.

    In order to create and start the ASM instances on the new node, Oracle requires the Listener to be present and started. DBCA prompts with requesting permission to configure the listener using port 1521 and listener name LISTENER_rac3. Click on Yes if the default port is good, else click on No and manually execute NetCA on rac3 to create the listener using a different port.

    Database Configuration Assistant progress screen - Once instance management is complete, the user is prompted with the message "Do you want to perform another operation?” Click on No to end.

    At this stage, the following is true:

    The clusterware has been installed on node rac3 and is now part of the cluster.

    The Oracle software has been installed on node rac3.

    The ASM5 and new Oracle instance devdb3 has been created and configured on rac3.

    Verify that the upgrade is successful.

    Verify that all instances in the cluster are started using the V$ACTIVE_INSTANCES view from any of the participating instances. For example:

    SQL> select * from v$active_instances;



    [oracle@rac1 oracle]$ srvctl status database -d devdb

    Instance devdb3 is running on node rac3

    [oracle@rac1 oracle]$ srvctl status service -d devdb

    Step 6: Set the Environment and the TNS entries

    For easy administration and navigation, you should define several different environment variables in the login profile.
    [oracle@rac3 oracle]$ more .bash_profile

    export ORACLE_BASE=...
    export ORACLE_HOME=..
    export ORA_CRS_HOME=..

    export PATH=..
    export ORACLE_ADMIN=..
    export TNS_ADMIN=..

    Also add the new network address to the clinet tnsnames.ora file to the appropriate connect descriptors.

    <conn_identifier> =
    (ADDRESS = (PROTOCOL = TCP)(HOST = rac1-vip)(PORT = 1521))
    (ADDRESS = (PROTOCOL = TCP)(HOST = rac2-vip)(PORT = 1521))
    (ADDRESS = (PROTOCOL = TCP)(HOST = rac3-vip)(PORT = 1521))
    (LOAD_BALANCE = yes)
    (SERVICE_NAME = ..)

    If the servers are configured to use FAN features, add the new server address to the onsctl files on all database servers.

    The ons.config file is located in
    [oracle@oradb4 oracle]$ more $ORACLE_HOME/opmn/conf/ons.config
    useocr=on,,,onsclient2.dbalounge .com:6200
    Last edited by Hemant; 01-09-2012 at 01:58 AM.
    Hemant Fulfagar
    Oracle Administration Architect
    Skype Name: hemant.fulfagar
    DBALounge Provides Oracle DBA Training Services. This includes Online Training, Classroom based Training at Noida (Delhi NCR).
    Students come from Delhi, Gurgaon, Noida, Ghaziabad, Faridabad, Greater Noida .
    Trainings : Oracle Database,10G,11G,RAC,Apps,11i,R12,Fusion Middleware,WebLogic,OID,OBIEE,OAM,OIM,Directory Services,Oracle VM.

+ Reply to Thread

Similar Threads

  1. Replies: 0
    Last Post: 09-22-2012, 08:57 PM
  2. Oracle 11g ACFS - ASM Cluster File Systems
    By Hemant in forum RAC Performance,LoadBalancing,TAF,FAN,Troubleshooting
    Replies: 0
    Last Post: 08-18-2012, 01:45 AM
    By vipul in forum Oracle Apps Installation,Migration and Upgrades
    Replies: 2
    Last Post: 04-27-2012, 01:37 AM
  4. Cluvfy - the cluster verification tool in Oracle RAC
    By Hemant in forum RAC Performance,LoadBalancing,TAF,FAN,Troubleshooting
    Replies: 1
    Last Post: 12-15-2011, 08:49 PM
  5. Cluvfy - the cluster verification tool in Oracle RAC
    By Hemant in forum RAC Concepts and Administration
    Replies: 1
    Last Post: 12-15-2011, 08:49 PM

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts

DBA Lounge (P) Ltd. deals in Oracle Technologies on Consulting, Resourcing, Corporate Training

Online and corporate training available on Oracle Database 11g, Oracle 11g Real Application Cluster (RAC), Oracle Applications 11i/R12, Oracle Fusion Middleware 11g, Oracle Identity Management-OIM, Oracle Internet Directory 11g-OID, Oracle Business Intelligence Enterprise Edition-OBIEE, Oracle Golden Gate, Oracle Access Management-OAM, Oracle Internet Directory-ODS, Oracle Identity Analytics Architecture-OIA Statistics