You may encounter a scenario where the subnet manager is down on your InfiniBand on your Exadata. To restart is very simple:

[root@v1ex1dbadm01 ~]# ssh v1ex1sw-ibb01
You are now logged in to the root shell.
It is recommended to use ILOM shell instead of root shell.
All usage should be restricted to documented commands and documented
config files.
To view the list of documented commands, use "help" at linux prompt.
[root@v1ex1sw-ibb01 ~]# enablesm
Starting IB Subnet Manager. [ OK ]
Starting partitiond-daemon [ OK ]
[root@v1ex1sw-ibb01 ~]# ps -ef | grep opensm
root 8075 1 0 09:59 ? 00:00:00 /usr/sbin/opensm --daemon
root 8224 7909 0 09:59 pts/0 00:00:00 grep opensm
[root@v1ex1sw-ibb01 ~]#

More information can be found here:
Sun Datacenter InfiniBand Switch 36 – Enable the Subnet Manager

If you found this blog post useful, please like as well as follow me through my various Social Media avenues available on the sidebar and/or subscribe to this oracle blog via WordPress/e-mail.

Thanks

Zed DBA (Zahid Anwar)