How the finnegan does one regenerate a VMDK file’s UUID programmatically?

As promised in my last post, here it is – the real deal on the generation and re-generation of UUIDs for VMDK files on an ESX Server platform. A bit of background on this topic might be of interest and possibly of use to the reader.

It was around six months ago that I became involved in some work requiring me to dabble with VMDK files at an intimate level. One of the steps necessitated the re-generation and assignment of UUID for a VMDK file (of course, the base VMDK file and not the ‘flat’ or ‘rdmp’ file). As much as I tried searching online, I could not find any constructive approach to actually generating these UUIDs. This made sense to the extent that VMware had vested interest in preserving the cloud of secrecy to its UUID generation algorithms. At this juncture, I must clarify that the UUID generation scheme for a VMDK is different from that of a Virtual Machine and other ESX Server platform components. For instance, a Virtual Machine (whose metadata is represented in the form of a .vmx file) may have the following UUID information:

uuid.location = “56 4d 3e 6b 50 fe 21 4c-e6 9d 74 44 da ac de 23”
uuid.bios = “56 4d 3e 6b 50 fe 21 4c-e6 9d 74 44 da ac de 23”

The algorithm (or at least the basic methodology) by which these are generated is more or less documented in existing VMware documents. This will not be discussed in the blog. Digressing no more, in brief, I found absolutely no information on how to generate a VMDK UUID for the ESX Server environment. The VI-SDK (and the vSphere SDK) documentation provides information on an API that is used to change the UUID of a VMDK. The API is:

SetVirtualDiskUUID with four parameters described in order as

_this (Managed Object Reference to the VirtualDiskManager

name (String) // referring to the Datastore path or URL of the VirtualDisk

datacenter(Managed Object Reference to the Datacenter) // not required for ESX Server

uuid(String) // The new UUID to update the VMDK file with.

Now the interesting part is that the ‘uuid’ component is a necessary parameter to this API call. This absolutely makes no sense for my requirements of course, as I need the system to generate me a new UUID just like the ‘vmkfstools’ command can generate on the fly. In case you are not familiar with this method, consider the following representative example:

Suppose we want to regenerate the UUID of a VirtualDisk named New_W2K3_3.vmdk then follow the following steps:

1. Locate the ‘vmkfstools’ binary (in case it has not been already set in the PATH enviroment variable) using

[root@z0ltanik]# whereis vmkfstools

You should see a response such as

vmkfstools: /usr/sbin/vmkfstools /usr/man/man1/vmkfstools.1.gz

2. Suppose that vmfkstools resides in /usr/sbin, traverse to the directory where the VMDK file exists (or use relative/absolute paths) and invoke

[root@z0ltanik New_W2k3]# /usr/sbin/vmkfstools -J getuuid New_W2k3_3.vmdk

The output will be something like the following

UUID is 60 00 C2 95 78 96 04 68-53 cb d0 5e 0f a7 d0 32

This is the current UUID of the VMDK file.

3. To regenerate (and assign) a new UUID to the aforementioned VMDK file, simply invoke the following command. The response follows suit, displaying the
new UUID

[root@z0ltanik New_W2k3]# /usr/sbin/vmkfstools -J setuuid New_W2k3_3.vmdk

UUID is 60 00 C2 97 5e 42 fd 6f-bc ee 83 c5 51 c6 40 34

Et voila! You are all done. I had been hoping for a similar behavior from the SetVirtualDiskUUID API. Perhaps the VMware SDK engineers should have attended Joshua Bloch’s API design sessions eh? Just kidding of course – for the most part the vSphere (and its older avatar, the VI-SDK) SDK is impeccably designed. Enough digression already mates! Let’s get down to the meat of a simple Java example of how to read UUIDs from a file and translate them into new valid and perfectly acceptable UUIDs subject of course to the following caveats:

1. The user is responsible for writing the business logic to ensure the unique-ness of the new UUID in the Virtual Machine namespace (yes, NOT the ESX Server namespace). A trivial task indeed.

2. This code represents a representative example and suited to my requirements of retrieving a VMDK file’s existing VMDK and modifying it. However, this can be easily modified to generate UUIDs from scratch ad infinitum. The moot point here is that the first 7 characters of the UUID must always be “60 00 C2 9“. There you go – a revelation for you lads!

3. No major error condition checks, boundary condition checks etc. Feel free to add them yourself.

The code:

     /**
      * Program to read UUIDs from 'uuid.txt', generate valid UUIDs and display them on the console.
      *
      * @author Timmy z0ltan Jose
      */

      import java.util.Random;
      import java.io.BufferedReader;
      import java.io.FileReader;
      import java.io.IOException;

      public class SetUuid {

      public static void main(String args[]) {

            String oldUuid = null;
            BufferedReader myReader = null;

           try {
                    myReader = new BufferedReader(new FileReader("uuid.txt"));

                    while( (oldUuid = myReader.readLine()) != null) {

                            System.out.println("\nOld Uuid = " + oldUuid);

                            System.out.println("New Uuid = " + getNewUuid(oldUuid));
                     }

                } catch (IOException ex) {
                        ex.printStackTrace();
                } finally {
                      if (myReader != null) {
                          try {
                                 myReader.close();
                               } catch ( IOException ioex) {
                                      ioex.printStackTrace();
                                  }
                      }
                }
      }

      private static String getNewUuid(String oldUuid) {

            String fixed = null;
            String toggle = null;
            String randomOne = null;
            String randomTwo = null;
            String timestamp = null;
            String newUuid = " ";

            StringBuffer origUuid = new StringBuffer(oldUuid);

            for (int i = 0; i < origUuid.length(); i++) {

               if (origUuid.charAt(i) == ' ' || origUuid.charAt(i) == '-') {
                      origUuid.deleteCharAt(i);
               }
             }

            fixed = origUuid.substring(0, 7);

            toggle = Integer.toHexString(new Random().nextInt(16));

            byte[] randBytes = new byte[4];

            new Random().nextBytes(randBytes);
            String result = "";
           
            for (int i=0; i < randBytes.length; i++) {
                 result += Integer.toString( ( randBytes[i] & 0xff ) + 0x100, 16).substring( 1 );
            }

            randomOne = result;

            byte[] b = new byte[2];
            randomTwo = "";

            new Random().nextBytes(b);

            for (int j = 0; j < b.length; j++) {

                  randomTwo += Integer.toHexString( (b[j] & 0xff) + 0x100).substring(1);
            }

             timestamp = toggle + Long.toHexString(System.nanoTime());

             String firstHalf = fixed + toggle + randomOne;

             for (int j = 1; j < firstHalf.length(); j+=2) {

                    newUuid += " " + firstHalf.charAt(j-1) + firstHalf.charAt(j);
             }

              newUuid += "-";

              String secondHalf = randomTwo + timestamp.substring(0, 12);
              String newerUuid = (" " + secondHalf.charAt(0) + secondHalf.charAt(1)).trim();

              for (int k = 3; k < secondHalf.length(); k+=2) {

                     newerUuid += " " + secondHalf.charAt(k-1) + secondHalf.charAt(k);
               }

              return (newUuid + newerUuid.trim()).trim();

      }
    }

And here, of course, is the uuid.txt file:

60 00 C2 97 50 b3 99 57-47 b9 2b fd fb 45 ed 5f
60 00 C2 9f f0 04 aa 90-0f 6e 94 0a b0 33 fa f3
60 00 C2 93 54 19 90 f8-f1 8f 7b 25 7b 72 e2 3a
60 00 C2 97 c1 57 ca e4-a2 24 98 3f c2 0d 55 c2
60 00 C2 9e b1 00 fb 5e-a2 45 99 09 7f 0d fd 00

Sample output from a single run of the above code:

Old Uuid = 60 00 C2 97 50 b3 99 57-47 b9 2b fd fb 45 ed 5f
New Uuid = 60 00 C2 90 bb dd a8 31-b3 a8 0c 67 37 cc 6a f0

Old Uuid = 60 00 C2 9f f0 04 aa 90-0f 6e 94 0a b0 33 fa f3
New Uuid = 60 00 C2 93 16 22 c0 93-76 1d 3c 67 37 e5 b2 2c

Old Uuid = 60 00 C2 93 54 19 90 f8-f1 8f 7b 25 7b 72 e2 3a
New Uuid = 60 00 C2 9e 2f 80 ad 29-53 aa ec 67 37 fd 55 43

Old Uuid = 60 00 C2 97 c1 57 ca e4-a2 24 98 3f c2 0d 55 c2
New Uuid = 60 00 C2 95 24 ba 3b b4-17 18 5c 67 38 27 7b bd

Old Uuid = 60 00 C2 9e b1 00 fb 5e-a2 45 99 09 7f 0d fd 00
New Uuid = 60 00 C2 9c 69 24 b2 49-44 f2 cc 67 38 3e 81 b0

The code is simple enough as to not necessitate the presence of comments. However, if needed, I will explain the modus operandi in a separate blog. Right now I have to go get my power meal done. So long! 😉

P.S: Some screwy issues with the code formatter. Well, que sera sera….. for now!

P.P.S: Fixed the formatting issue – should have used the <pre> tag instead of <code>….. duh!

Advertisements
How the finnegan does one regenerate a VMDK file’s UUID programmatically?

A programmatic approach to VMware solutions: Part 1

This is the first of a series of planned blogs pertaining to the use of the VMware SDK APIs to seamlessly exploit VMware ESX Server services programmatically.
This is the result of the experience that I have gained by working on a live project to provide VMware support to an existing Java-based enterprise software product.
An early disclaimer is highly in order: All these examples are idiomatic and do not relate to the actual code used in the aformentioned product. Phew! Now that we have got that out of our chest, let’s proceed onto the meat of the matter directly.

The roadmap of this tutorials series is roughly sketched out as follows:

Part 1Introduction to VMware and definition of the scope of the tutorial.

Part 2The VMware ESX server service exposed as a web-service, brief discussion of the WSDL file, generation of the required JAR files for the Java client.

Part 3Introductory code samples – logging on to the VMware ESX server, connecting and verification of session.

Part 4 In depth discussion of Raw LUNs and VMFS disks and their applications in industry. Technical differences and varied approach in code.

Part 5 In depth discussion of the VMware ESX Server SDK.

Part 6 Continued discussion on the VMware ESX Server SDK.

Part 7 Concluding session on the VMware ESX Server SDK.

Part 8Operations on the Virtual Machine -code samples.

Part 9 Operations on the ESX Server (Host) – code samples.

Part 10Operations using the VMware SDK APIs specifically for the Storage domain – code samples.

Part 11Threading, concurrency and gotchas while using integrating the VMware programs with existing products.

Alright, so that is pretty much the road ahead. More sessions can be charted out based on feedback. Any specific operations can be discussed in-depth in future blogs.

Introduction to VMware:

VMware is a virtualization software which basically comes in two flavors – desktop and enterprise. In this tutorial series, I will be dealing with the enterprise server solution – VMware ESX Server and how to write programs to access the services provided by the server, which can be integrated with enterprise solutions. I will be using Java as the client language throughout this tutorial. This is not a series where the internal details/ performance statistics et cetera will be discussed, as also the relative merits of VMware with respect to other virtualization software. Those topics will be, at best, touched upon and skimmed through. Now there are enough resources to implement client programs to avail of the services exposed by the VMware ESX server but not nearly enough resources which show how to actually go beyond writing mere scripts and actually create enterprise ready VMware solutions. More details can be availed on the VMware site or better still, on Wikipedia.

As I have mentioned already, I will not be delving too deeply in the specifications of the ESX Server. Such information is widely available (though the VMware site documentation, in my most humble opinion, sucks!) on the Internet.

That’s all for part 1. The next session is shortly forthcoming. Till then, peace!

A programmatic approach to VMware solutions: Part 1