There maybe times when you are required to fully shutdown an Oracle Exadata Machine, for example for maintenance.

Below is the My Oracle Support note used to carry out the shutdown:
Steps To Shutdown/Startup The Exadata & RDBMS Services and Cell/Compute Nodes On An Exadata Configuration (Doc ID 1093890.1)

1. Pre-requisites

Ensure you have all the compute nodes and storage cells ILOM addresses and correct passwords.  Otherwise you will not be able to remotely power back on and will require a physical power on using the power button on the front panels.

2. Disable clusterware autostart

First we need to stop clusterware restarting up on reboot.  So logon to your first compute node and disable via dcli (more info on dcli can be found in this blog post) using your correct crs home:

login as: root
root@x.x.x.x's password:
Last login: Wed Jun 10 08:45:30 IST 2020 from x.x.x.x on pts/0
Last login: Wed Jun 10 09:07:17 2020 from x.x.x.x
[root@v1ex1dbadm01 ~]# dcli -g /opt/oracle.SupportTools/onecommand/dbs_group -l root /u01/app/12.1.0.2/grid/bin/crsctl config crs
v1ex1dbadm01: CRS-4622: Oracle High Availability Services autostart is enabled.
v1ex1dbadm02: CRS-4622: Oracle High Availability Services autostart is enabled.
[root@v1ex1dbadm01 ~]# dcli -g /opt/oracle.SupportTools/onecommand/dbs_group -l root /u01/app/12.1.0.2/grid/bin/crsctl disable crs
v1ex1dbadm01: CRS-4621: Oracle High Availability Services autostart is disabled.
v1ex1dbadm02: CRS-4621: Oracle High Availability Services autostart is disabled.
[root@v1ex1dbadm01 ~]#

3. Stop Grid Infrastructure on the cluster

Next we stop clusterware cluster-wide gracefully:

[root@v1ex1dbadm01 ~]# . oraenv
ORACLE_SID = [root] ? +ASM1
The Oracle base has been set to /u01/app/oracle
[root@v1ex1dbadm01 ~]# dcli -g /opt/oracle.SupportTools/onecommand/dbs_group -l root /u01/app/12.1.0.2/grid/bin/crsctl check crs
v1ex1dbadm01: CRS-4638: Oracle High Availability Services is online
v1ex1dbadm01: CRS-4537: Cluster Ready Services is online
v1ex1dbadm01: CRS-4529: Cluster Synchronization Services is online
v1ex1dbadm01: CRS-4533: Event Manager is online
v1ex1dbadm02: CRS-4638: Oracle High Availability Services is online
v1ex1dbadm02: CRS-4537: Cluster Ready Services is online
v1ex1dbadm02: CRS-4529: Cluster Synchronization Services is online
v1ex1dbadm02: CRS-4533: Event Manager is online
[root@v1ex1dbadm01 ~]#

[root@v1ex1dbadm01 ~]# crsctl stat res -t
--------------------------------------------------------------------------------
Name Target State Server State details
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.DATAC1.dg
ONLINE ONLINE v1ex1dbadm01 STABLE
ONLINE ONLINE v1ex1dbadm02 STABLE
...
ora.RECOC1.dg
ONLINE ONLINE v1ex1dbadm01 STABLE
ONLINE ONLINE v1ex1dbadm02 STABLE
ora.asm
ONLINE ONLINE v1ex1dbadm01 Started,STABLE
ONLINE ONLINE v1ex1dbadm02 Started,STABLE
...
ora.net1.network
ONLINE ONLINE v1ex1dbadm01 STABLE
ONLINE ONLINE v1ex1dbadm02 STABLE
ora.ons
ONLINE ONLINE v1ex1dbadm01 STABLE
ONLINE ONLINE v1ex1dbadm02 STABLE
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.LISTENER_SCAN1.lsnr
1 ONLINE ONLINE v1ex1dbadm02 STABLE
ora.LISTENER_SCAN2.lsnr
1 ONLINE ONLINE v1ex1dbadm01 STABLE
ora.LISTENER_SCAN3.lsnr
1 ONLINE ONLINE v1ex1dbadm01 STABLE
...
1 ONLINE ONLINE v1ex1dbadm01 STABLE
ora.scan1.vip
1 ONLINE ONLINE v1ex1dbadm02 STABLE
ora.scan2.vip
1 ONLINE ONLINE v1ex1dbadm01 STABLE
ora.scan3.vip
1 ONLINE ONLINE v1ex1dbadm01 STABLE
--------------------------------------------------------------------------------
[root@v1ex1dbadm01 ~]#

[root@v1ex1dbadm01 ~]# crsctl stop cluster -all
CRS-2673: Attempting to stop 'ora.crsd' on 'v1ex1dbadm01'
CRS-2790: Starting shutdown of Cluster Ready Services-managed resources on 'v1ex1dbadm01'
...
CRS-2677: Stop of 'ora.ons' on 'v1ex1dbadm01' succeeded 
CRS-2673: Attempting to stop 'ora.net1.network' on 'v1ex1dbadm01' 
CRS-2677: Stop of 'ora.net1.network' on 'v1ex1dbadm01' succeeded 
CRS-2792: Shutdown of Cluster Ready Services-managed resources on 'v1ex1dbadm01' has completed 
CRS-2677: Stop of 'ora.crsd' on 'v1ex1dbadm02' succeeded 
CRS-2673: Attempting to stop 'ora.ctssd' on 'v1ex1dbadm02' 
CRS-2673: Attempting to stop 'ora.evmd' on 'v1ex1dbadm02' 
CRS-2673: Attempting to stop 'ora.storage' on 'v1ex1dbadm02' 
CRS-2677: Stop of 'ora.storage' on 'v1ex1dbadm02' succeeded 
CRS-2673: Attempting to stop 'ora.asm' on 'v1ex1dbadm02' 
CRS-2677: Stop of 'ora.crsd' on 'v1ex1dbadm01' succeeded 
CRS-2673: Attempting to stop 'ora.ctssd' on 'v1ex1dbadm01' 
CRS-2673: Attempting to stop 'ora.evmd' on 'v1ex1dbadm01' 
CRS-2673: Attempting to stop 'ora.storage' on 'v1ex1dbadm01' 
CRS-2677: Stop of 'ora.storage' on 'v1ex1dbadm01' succeeded 
CRS-2673: Attempting to stop 'ora.asm' on 'v1ex1dbadm01' 
CRS-2677: Stop of 'ora.ctssd' on 'v1ex1dbadm02' succeeded 
CRS-2677: Stop of 'ora.evmd' on 'v1ex1dbadm02' succeeded 
CRS-2677: Stop of 'ora.evmd' on 'v1ex1dbadm01' succeeded 
CRS-2677: Stop of 'ora.ctssd' on 'v1ex1dbadm01' succeeded 
CRS-2677: Stop of 'ora.asm' on 'v1ex1dbadm02' succeeded 
CRS-2673: Attempting to stop 'ora.cluster_interconnect.haip' on 'v1ex1dbadm02' 
CRS-2677: Stop of 'ora.cluster_interconnect.haip' on 'v1ex1dbadm02' succeeded 
CRS-2673: Attempting to stop 'ora.cssd' on 'v1ex1dbadm02' 
CRS-2677: Stop of 'ora.cssd' on 'v1ex1dbadm02' succeeded 
CRS-2673: Attempting to stop 'ora.diskmon' on 'v1ex1dbadm02' 
CRS-2677: Stop of 'ora.diskmon' on 'v1ex1dbadm02' succeeded
CRS-2677: Stop of 'ora.asm' on 'v1ex1dbadm01' succeeded 
CRS-2673: Attempting to stop 'ora.cluster_interconnect.haip' on 'v1ex1dbadm01' 
CRS-2677: Stop of 'ora.cluster_interconnect.haip' on 'v1ex1dbadm01' succeeded 
CRS-2673: Attempting to stop 'ora.cssd' on 'v1ex1dbadm01'
CRS-2677: Stop of 'ora.cssd' on 'v1ex1dbadm01' succeeded
CRS-2673: Attempting to stop 'ora.diskmon' on 'v1ex1dbadm01'
CRS-2677: Stop of 'ora.diskmon' on 'v1ex1dbadm01' succeeded
[root@v1ex1dbadm01 ~]#

[root@v1ex1dbadm01 ~]# dcli -g /opt/oracle.SupportTools/onecommand/dbs_group -l root /u01/app/12.1.0.2/grid/bin/crsctl stat res -t
v1ex1dbadm01: CRS-4535: Cannot communicate with Cluster Ready Services
v1ex1dbadm01: CRS-4000: Command Status failed, or completed with errors.
v1ex1dbadm02: CRS-4535: Cannot communicate with Cluster Ready Services
v1ex1dbadm02: CRS-4000: Command Status failed, or completed with errors.
[root@v1ex1dbadm01 ~]#

4. Power off Storage Cells

Now that clusterware is down including ASM, we can power down the storage cells by first shutting down the cell services:

[root@v1ex1dbadm01 ~]# dcli -g /opt/oracle.SupportTools/onecommand/cell_group -l root "cellcli -e alter cell shutdown services all"
v1ex1celadm01:
v1ex1celadm01: Stopping the RS, CELLSRV, and MS services...
v1ex1celadm01: The SHUTDOWN of services was successful.
v1ex1celadm02:
v1ex1celadm02: Stopping the RS, CELLSRV, and MS services...
v1ex1celadm02: The SHUTDOWN of services was successful.
v1ex1celadm03:
v1ex1celadm03: Stopping the RS, CELLSRV, and MS services...
v1ex1celadm03: The SHUTDOWN of services was successful.
[root@v1ex1dbadm01 ~]#

Now the storage cells are shutdown, we can power them off:

[root@v1ex1dbadm01 ~]# dcli -g /opt/oracle.SupportTools/onecommand/cell_group -l root poweroff

v1ex1celadm02:Connection to v1ex1celadm02 closed by remote host.

v1ex1celadm01:Connection to v1ex1celadm01 closed by remote host.

v1ex1celadm03:Connection to v1ex1celadm03 closed by remote host.

[root@v1ex1dbadm01 ~]#

5. Power off Compute Nodes

As we are on the first compute node, we can power this off as shown below:

[root@v1ex1dbadm01 ~]# poweroff

Now we power off the remaining compute node by logging on via ssh:

login as: root
root@x.x.x.x's password:
Last login: Wed Jun 10 08:45:25 IST 2020 from x.x.x.x on ssh
Last login: Wed Jun 10 09:03:41 2020 from x.x.x.x
[root@v1ex1dbadm02 ~]# poweroff

If you have a half or full rack and wish to power off all compute nodes, you can use:

[root@v1ex1dbadm01 ~]# dcli -g /opt/oracle.SupportTools/onecommand/dbs_group_all_but_first -l root poweroff

Then power off the first node:

[root@v1ex1dbadm01 ~]# poweroff

Carry out your maintenance and when completed, you can restart the Oracle Exadata Machine by follow my blog post:
How to Startup an Oracle Exadata Machine

If you found this blog post useful, please like as well as follow me through my various Social Media avenues available on the sidebar and/or subscribe to this oracle blog via WordPress/e-mail.

Thanks

Zed DBA (Zahid Anwar)