VMware Expert Database Workshop Program Oracle Edition – Day 3

Day 3 kicked off again with another early start at 7am, yawn 🙂

Again, it was a very intense day, with lots of presentations and then ending with video interviews of each attendee:

  • Dean Bolton from VLSS, talked to us about “License Fortress from VLSS”
    • Very interesting product, where VLSS can advise you how to run your Oracle on VMware, then protect you from Oracle with the License Fortress guarantee, that has lawyers ready to defend you in case of license compliance backed by an insurance policy if required.  So the customer is never at risk when they take out the “License Fortress from VLSS”.
    • Plug for my current employer who also offer License Audit Consulting.
  • Chris Rohan from VMware, talked to us about “VMware vSphere Core & SDDC – Networking – NSX & VCNS”
  • Marcus Thordal from Brocade, talked to us about “Brocade and VMware Technology and the VMware Solutions Lab”
    • Interestingly Brocade have worked out a way to tag network packets, so you can identify which VM guest is causing network traffic
    • Also had good example of how NVMe is causing need to higher network bandwidth
  • Simon Guyennet from VMware, talked to us about “Emerging Products” & “VMware Integrated Containers and Oracle”
    • Lot of NDA stuff, so those interest in this area, keep look out, some interesting stuff coming soon 🙂
  • Mike Adams from VMware, talked to us about “The CPBU, vSphere and Friends, and the Experts Program”
    • Key take away that was not NDA, is VMware on AWS that is currently available to select customer and will be Generally Available soon 🙂
  • Somu Rajarathinam and Ron Ekins from Pure Storage gave a Technical Session
  • Feidhlim O’Leary from VMware, talked to us about “High Availability and Disaster Recovery in the SDDC”
  • Alain Geenrits from Blue Medora, talked to us about “Management & Monitoring – Blue Medora and Oracle on vSphere”
  • Daniel Hesselink from License Consulting, talked to us about “License Audit with License Consulting”
  • The duo Sudhir Balasubramanian and Mohan Potheri, talk to us about “vSphere HA or Oracle RAC, SRM or Data Guard, they are all complimentary when Oracle is run in the SDDC”
    • I enjoyed the labs from this duo, with their “good cop, bad cop” style 🙂

The workshop ended with a short video interview, where we were each asked to introduce ourselves, answer a few questions about the workshop and Pure Storage.  I’m not the best at this sort of things, so I don’t think I’ll end up in the marketing video of the event, but time will tell 🙂

My OCM buddy Yvonne Murphy, then gave a few of us an extended tour of the Global Support Services (GSS) whist we waited for the shuttle back to the hotel.

Then it was a quick chauffeured ride to the airport and a short flight back home to Manchester.

Another great day that concluded the workshop, increased my knowledge of VMware and gave me great opportunity to network with Oracle Database Experts from around Europe 🙂

Many thanks to VMware and Pure Storage for organising this workshop and allowing me to be a part of it 🙂

My tweets for the day can be seen here.

The VMware Expert Database Workshop Program hashtag is #VMWORA

My related Blog Posts

VMware Expert Database Workshop Program Oracle Edition
VMware Expert Database Workshop Program Oracle Edition – Day 1
VMware Expert Database Workshop Program Oracle Edition – Day 2

Other related Blog Posts

Tim Hall (Oracle Base) – VMware Expert Database Workshop Program Oracle Edition
Tim Hall (Oracle Base) – VMware Workshop – The Journey Begins
Tim Hall (Oracle Base) – VMware Workshop – Day 1
Tim Hall (Oracle Base) – VMware Workshop – Day 2
Tim Hall (Oracle Base) – VMware Workshop – Day 3
Michael Corey (Columnist) – VMware Experts Program Oracle Edition
Michael Corey (Columnist) – Day 1 VMware Experts Program Oracle Edition
Michael Corey (Columnist) – Day 2 VMware Experts Program Oracle Edition

If you found this blog post useful, please like as well as follow me through my various Social Media avenues available on the sidebar and/or subscribe to this oracle blog via WordPress/e-mail.

Thanks

Zed DBA (Zahid Anwar)

VMware Expert Database Workshop Program Oracle Edition – Day 2

Day 2 kicked off again with another early start at 7am, coupled with the late night, I was a bit tired to say the least, but was all worth it 🙂

Again, it was a very intense day, with lots of presentations and technical deep dives, ending with a lab session:

  • My OCM buddy Yvonne Murphy, kicked the day off by talking to us about “The Best Oracle Support Team on Earth – Global Support Services Oracle Team”
  • Dave Welch, House of Brick, talked to us about “Oracle on vSphere Licensing”
    • Some interesting Oracle license cases of which the main one discussed is available here
  • Jad El-Zein, talked to us about “vRealizeAutomation and Oracle”
  • Andreas Scherr, talked to us about “vSphere Core Storage Fundamentals” & “Modern Converged Storage, & vSAN & vVols”
    • I was impressed with the vSAN concept of using spare drive bays in an ESXi host to put a Hard Disks or SSD Drives in and/or using spare PCIe slots to put a Flash Cards in (including NVMe 🙂 )
    • Then these server attached storage devices are pooled together to provide a shared Datastore that has resilience built in using software rather then hardware 🙂
    • Can be all flash or hybird but each ESXi needs a cache device i.e. SSD Drive or Flash Card
    • More info can be found here
  • Valentin Bondzio, gave us an very enjoyable deep dive on CPU usage in Virtual Environment and how to troubleshoot 🙂
  • We then finished the day with a lab session with Sudhir Bala and Mohan Potheri, where we got to for example:
    • Create a Virtual Hard Drive
    • Attach an Existing Virtual Hard Drive (useful for RAC clusters 🙂 )
    • Then we played a game to stress test a Pure Storage, however I could only get 2Gb a second using 10 sessions running a parallel query of 10 on the largest object in the database.  This is not because the Pure Storage but because of I/O queue in the VMware stack, which we didn’t get time to change but highlight the point 🙂

Another great day of the 3 days workshop, in which I got to gain even more new knowledge in regards to VMware, in particular CPU usage and vSAN 🙂

The day ended with a meal and drinks at The Oliver Plunkett, more socialising with Johannes AhrendsRon EkinsFrits HooglandTim Hall and Mohan Potheri.  Then a nice walk back to the hotel with Mohan PotheriJohannes Ahrends and Martin Klier.  It’s a small world as I found Martin Klier and I had overlap with some customers and people 🙂  The socialising then continued in the reception lounge with Carl Dudley, Ron EkinsFrits Hoogland and Tim Hall.

Many thanks to VMware and Pure Storage, I’m looking forward to the rest of the workshop 🙂

My tweets for the day can be seen here.

The VMware Expert Database Workshop Program hashtag is #VMWORA

My related Blog Posts

VMware Expert Database Workshop Program Oracle Edition
VMware Expert Database Workshop Program Oracle Edition – Day 1

Other related Blog Posts

Tim Hall (Oracle Base) – VMware Expert Database Workshop Program Oracle Edition
Tim Hall (Oracle Base) – VMware Workshop – The Journey Begins
Tim Hall (Oracle Base) – VMware Workshop – Day 1
Tim Hall (Oracle Base) – VMware Workshop – Day 2
Michael Corey (Columnist) – VMware Experts Program Oracle Edition
Michael Corey (Columnist) – Day 1 VMware Experts Program Oracle Edition
Michael Corey (Columnist) – Day 2 VMware Experts Program Oracle Edition

If you found this blog post useful, please like as well as follow me through my various Social Media avenues available on the sidebar and/or subscribe to this oracle blog via WordPress/e-mail.

Thanks

Zed DBA (Zahid Anwar)

VMware Expert Database Workshop Program Oracle Edition – Day 1

The journey began with a 9pm flight from Manchester to Cork, I was then picked up by a chauffeur in an S Class Mercedes along with Jan-Peter courtesy of VMware 🙂

We arrived at the posh The Kingsley Hotel at 11:00pm, where Carl Dudley, Ron EkinsFrits Hoogland and Tim Hall were still in the reception lounge having discussions about Cloud, Storage amongst other topics.  So Jan-Peter and myself joined in until midnight.

The next morning was an early start with a 7am private shuttle ride to VMware Campus in Ballincollig.

It was a very intense day, with lots of presentations, technical deep dives and labs:

  • Mike Corey recapped ‘The Basics of vSphere’
  • Karen Egan VP Support, gave an overview of Global Support Center (GSC) EMEA
  • Then a few presentations from the sponsor Pure Storage regarding their Flash offerings from Carl McQuillanSomu Rajarathinam and Alejandro Infanzon 🙂
  • Valentin Bondzio gave a deep dive of VMware vSphere Core in particular vMotion, which really impressed me 🙂
  • My OCM buddy Yvonne Murphy, talked about Oracle on VMware “One Woman’s Journey”, which was about her true experience of real life customer problems working within Global Support Center EMEA.  It was reassuring to know VMware has 1,000 support engineers in GSC EMEA in Cork alone and most Support Requests are resolved in 24-58 hours.  We then got the privilege of having a tour of the GSC 🙂
  • We then finished the day with labs session with Sudhir Bala and Mohan Potheri, where we got to for example:
    • Create a Template from a VM guest
    • Create a VM guest from a Template
    • Move a VM guest whilst powered on from one physical host to another using vMotion
    • Increase CPUs and Memory of a VM guest on the fly
    • And more…

It was a great 1st day of a 3 days workshop, in which I got to gain new knowledge in regards to VMware, which I can see help in particular circumstances and requirements 🙂

The day ended with a meal and drinks at SoHo Rooftop Bar & Restaurant, where I got the opportunity to catch up with my OCM buddy Yvonne Murphy, network with new people including: Dean Bolton from VLSS, Valentin Bondzio and Don Sullivan, both from VMware.

It was then the short journey back to the hotel, where I caught up with Tim Hall.  It was then a gather in the hotel reception where conversations continued until late 🙂

Many thanks to VMware and Pure Storage, I’m looking forward to the rest of the workshop 🙂

My tweets for the day can be seen here.

The VMware Expert Database Workshop Program hashtag is #VMWORA

My related Blog Posts

VMware Expert Database Workshop Program Oracle Edition

Other related Blog Posts

Tim Hall (Oracle Base) – VMware Expert Database Workshop Program Oracle Edition
Tim Hall (Oracle Base) – VMware Workshop – The Journey Begins
Tim Hall (Oracle Base) – VMware Workshop – Day 1
Michael Corey (Columnist) – VMware Experts Program Oracle Edition
Michael Corey (Columnist) – Day 1 VMware Experts Program Oracle Edition

If you found this blog post useful, please like as well as follow me through my various Social Media avenues available on the sidebar and/or subscribe to this oracle blog via WordPress/e-mail.

Thanks

Zed DBA (Zahid Anwar)

VMware Expert Database Workshop Program Oracle Edition

The VMware Expert Database Workshop Program started back in 2013 and has been predominately ran in Palo Alto, California.  Last year was the first time it was ran outside Palo Alto, in Sofia, Bulgaria and this year there is an additional workshop in Cork, Ireland being ran on the 18th to 20th of July 2017.

The program brings together selected Database and Application Experts from around the world to meet with VMware senior executives, architects and engineers.  The program entails assortment of open discussions and lab exercises for an intense learning environment with the intention of gaining extensive knowledge of VMware virtualisation infrastructure.

I’m honoured to have been selected by VMware to join the likes of Heli Helskyaho, Carl Dudley, Johannes Ahrends, Jan Timmerman, Niki Christopher, Mikhail Kirpichev, Yann Allandit, Kamran Aghayev, Frits Hoogland and Tim Hall 🙂

I’m really looking forward to the workshop next week, so I can meet new people and to build on my VMware knowledge 🙂

The VMware Expert Database Workshop Program hashtag is #VMWORA, if you wish to follow.

My related Blog Posts

VMware Expert Database Workshop Program Oracle Edition – Day 1

Other related Blog Posts

Tim Hall (Oracle Base) – VMware Expert Database Workshop Program Oracle Edition
Michael Corey (Columnist) – VMware Experts Program Oracle Edition

Related Links

VMware Experts Database Workshop – YouTube Playlist
VMware Experts Database Workshop – Oracle Edition April 2017 – Palo Alto
VMware Experts Database Workshop – SQL workshop – Nov 2016 – Sofia Bulgaria
VMware Experts Database Workshop – Oracle Edition April 2016 – Palo Alto
VMware Experts Database Workshop – SQL Workshop – April 2015 – Palo Alto
VMware Experts Database Workshop – SQL Workshop – Sept 2014 – Palo Alto

If you found this blog post useful, please like as well as follow me through my various Social Media avenues available on the sidebar and/or subscribe to this oracle blog via WordPress/e-mail.

Thanks

Zed DBA (Zahid Anwar)

How to find the Oldest and Newest Interval Partition

I was required to find the oldest and newest interval partition to be able to do some automated maintenance, but found query USER_TAB_PARTITIONS, ALL_TAB_PARTITIONS and DBA_TAB_PARTITIONS was a bit tricky as the HIGH_VALUE is a LONG data type 😦

A quick search on the web and I found a perfect solution from my good friend Tim Hall (Oracle Base), who created a function to convert the LONG into a DATE by executing the HIGH VALUE 🙂

Script: part_hv_to_date.sql

For ease of access, script below (please check Tim’s blog for updates and always ask for permission if you’re going to quote 🙂 ):

CREATE OR REPLACE FUNCTION part_hv_to_date (p_table_owner IN VARCHAR2,
 p_table_name IN VARCHAR2,
 p_partition_name IN VARCHAR2)
 RETURN DATE
-- -----------------------------------------------------------------------------------
-- File Name : https://oracle-base.com/dba/miscellaneous/part_hv_to_date.sql
-- Author : Tim Hall
-- Description : Create a function to turn partition HIGH_VALUE column to a date.
-- Call Syntax : @part_hv_to_date
-- Last Modified: 19/01/2012
-- Notes : Has to re-select the value from the view as LONG cannot be passed as a parameter.
-- Example call:
--
-- SELECT a.partition_name, 
-- part_hv_to_date(a.table_owner, a.table_name, a.partition_name) as high_value
-- FROM all_tab_partitions a;
--
-- Does no error handling. 
-- -----------------------------------------------------------------------------------
AS
 l_high_value VARCHAR2(32767);
 l_date DATE;
BEGIN
 SELECT high_value
 INTO l_high_value
 FROM all_tab_partitions
 WHERE table_owner = p_table_owner
 AND table_name = p_table_name
 AND partition_name = p_partition_name;
 
 EXECUTE IMMEDIATE 'SELECT ' || l_high_value || ' FROM dual' INTO l_date;
 RETURN l_date;
END;
/

 

With this you can get the oldest interval partition by running the following SQL:

SELECT PARTITION_NAME
FROM DBA_TAB_PARTITIONS
WHERE TABLE_OWNER = 'ZEDDBA'
AND TABLE_NAME = 'EXAMPLE'
AND PART_HV_TO_DATE(TABLE_OWNER, TABLE_NAME, PARTITION_NAME) = (
SELECT MIN(PART_HV_TO_DATE(TABLE_OWNER, TABLE_NAME, PARTITION_NAME))
FROM DBA_TAB_PARTITIONS
WHERE TABLE_OWNER = 'ZEDDBA'
AND TABLE_NAME = 'EXAMPLE');

Output:

SQL> SELECT PARTITION_NAME
   2 FROM DBA_TAB_PARTITIONS
   3 WHERE TABLE_OWNER = 'ZEDDBA'
   4 AND TABLE_NAME = 'EXAMPLE'
   5 AND PART_HV_TO_DATE(TABLE_OWNER, TABLE_NAME, PARTITION_NAME) = (
   6 SELECT MIN(PART_HV_TO_DATE(TABLE_OWNER, TABLE_NAME, PARTITION_NAME))
   7 FROM DBA_TAB_PARTITIONS
   8 WHERE TABLE_OWNER = 'ZEDDBA'
   9 AND TABLE_NAME = 'EXAMPLE');

PARTITION_NAME
--------------------
SYS_P10590

SQL>

 

And the newest interval partition by running the following SQL:

SELECT PARTITION_NAME
FROM DBA_TAB_PARTITIONS
WHERE TABLE_OWNER = 'ZEDDBA'
AND TABLE_NAME = 'EXAMPLE'
AND PART_HV_TO_DATE(TABLE_OWNER, TABLE_NAME, PARTITION_NAME) = (
SELECT MAX(PART_HV_TO_DATE(TABLE_OWNER, TABLE_NAME, PARTITION_NAME))
FROM DBA_TAB_PARTITIONS
WHERE TABLE_OWNER = 'ZEDDBA'
AND TABLE_NAME = 'EXAMPLE');

Output:

SQL> SELECT PARTITION_NAME
   2 FROM DBA_TAB_PARTITIONS
   3 WHERE TABLE_OWNER = 'ZEDDBA'
   4 AND TABLE_NAME = 'EXAMPLE'
   5 AND PART_HV_TO_DATE(TABLE_OWNER, TABLE_NAME, PARTITION_NAME) = (
   6 SELECT MAX(PART_HV_TO_DATE(TABLE_OWNER, TABLE_NAME, PARTITION_NAME))
   7 FROM DBA_TAB_PARTITIONS
   8 WHERE TABLE_OWNER = 'ZEDDBA'
   9 AND TABLE_NAME = 'EXAMPLE');

PARTITION_NAME
--------------------
SYS_P17718

SQL>

 

If you found this blog post useful, please like as well as follow me through my various Social Media avenues available on the sidebar and/or subscribe to this oracle blog via WordPress/e-mail.

Thanks

Zed DBA (Zahid Anwar)

 

VSS oravssw.exe Memory Leak on Windows

So we have a 12c RMAN Recovery Catalog in a database called rman12c running on a Windows 2012 R2 server.  We noticed the rman12c database was reporting “ORA-04030: out of process memory”:

Wed Jul 05 22:47:45 2017

Errors in file E:\APP\ORACLE\diag\rdbms\rman12c\rman12c\trace\rman12c_m000_9188.trc  (incident=76891):

ORA-04030: out of process memory when trying to allocate 8048 bytes (pga heap,KGLSPF_P ARRAY)

Incident details in: E:\APP\ORACLE\diag\rdbms\rman12c\rman12c\incident\incdir_76891\rman12c_m000_9188_i76891.trc

Use ADRCI or Support Workbench to package the incident.

See Note 411.1 at My Oracle Support for error and packaging details.

Process m000 died, see its trace file

This was rather strange, seems as though the server has 8Gb of RAM and 24Gb Virtual Memory.  But from Task Manager we could see 30.5Gb of 32Gb was committed:

VSS_Server_Memory_before_restart

Investigating further using Resource Monitor we can see the process oravssw.exe has nearly 26Gb committed memory! 😐 :

VSS_oravssw_leak_before_restart

Even when Task Manager is only showing 77.1Mb memory used:

VSS_oravssw_leak_process

A search in My Oracle Support (MOS – formally Metalink), we found the following MOS notes:
Bug 19835650 – Windows: VSS oravssw.exe memory leak (Doc ID 19835650.8)

“Description

VSS Writer leaks memory when performing volume based shadow copy.

Work Around:
Periodically stop/restart the VSS service to free the allocated memory.

Packaging Issue:
Technically and in theory this fix is included in the source of the 11.2.0.4.16 Windows Bundle.  Due build issues the 11.2.0.4.18 Windows Bundle is needed to use VSS without the memory leak.

In 12c the fix of this bug is included in 12.1.0.2.160419 Windows Bundle through the fix of bug 21634629.

Affects:

Product (Component):
Oracle Server (Rdbms)

Range of versions believed to be affected:
Versions BELOW 12.2

Versions confirmed as being affected:
12.1.0.1 (Base Release)
11.2.0.4

Platforms affected:
Windows 64bit (AMD64/EM64T)

Fixed:

The fix for 19835650 is first included in:
12.2.0.1 (Base Release)
12.1.0.2.160419 (Apr 2016) Bundle Patch for Windows Platforms
12.1.0.1 Patch 20 on Windows Platforms
11.2.0.4 Patch 16 on Windows Platforms”

Bug 19835650 : VSS ORAVSSW.EXE MEMORY LEAK

“DETAILED PROBLEM DESCRIPTION
===============================
Using Oracle VSS Writer service to backup to EMC VSS backup.  The first backup the backup memory grows from 8M to 77M and grows again and does not release the memory until the oravss service is restarted.

DIAGNOSTIC ANALYSIS
====================
oravssw.exe memory 8336k to 8500k per service running idle.  After backup this climbs to 67,192k + memory used by oravssw.exe.  The next backup it increases to 77,900k consumed and not released.  The memory returns to idle size only after bouncing the service.

WORKAROUND?
==============
No

TECHNICAL IMPACT
=================
Customer has custom script now to stop/restart the service to free the memory. This is unacceptable as a long term solution.

RELATED ISSUES (bugs, forums, RFAs)
=================================
Bug 9063341 : MEMORY LEAK OCCURS FROM ORAVSSW.EXE EVERY TIME BACKUP JOB HAS DONE”

So we restart the Oracle VSS Writer:
Vss_oravssw_restart

We instantly see the committed memory drop from 30.5Gb to 4.9Gb 🙂 :

VSS_Server_Memory_after_restart

Also the oravssw.exe drops from under 26Gb to 4Mb in Resource Monitor 🙂 :

VSS_oravssw_leak_after_restart

And no more “ORA-04030: out of process memory” 🙂

The long term solution would be to apply the Bundle Patch for the correct version mentioned above.

If you found this blog post useful, please like as well as follow me through my various Social Media avenues available on the sidebar and/or subscribe to this oracle blog via WordPress/e-mail.

Thanks

Zed DBA (Zahid Anwar)