How to benchmark virtual machines

When you are comparing various hypervisors, you may want to compare the performance of virtual machines (VMs) running on different hypervisors. Also, if you are comparing various public cloud providers, you will want to benchmark VMs offered by available providers. Whatever the reason may be, it is useful to know how to benchmark VMs properly.

Difficulty in any kind of benchmark tests is the fact that benchmark numbers are highly dependent on the type of workloads or applications used to derive the numbers. To assess the relative performance of any computer system properly, people come up with different types of benchmark tests, such as real program benchmark, synthetic benchmark, microbenchmark, etc. In each benchmark type, there are also a variety of benchmark suites available differing in the resulting workloads. In Linux, there are several benchmark software available, and Phoronix Test Suite is one such tool.

Phoronix Test Suite is an open-source benchmarking platform which provides an extensible list of test profiles and test suites. These test profiles and suites can measure different kinds of performance characteristics of an underlying system (e.g., CPU, disk I/O, RAM, networking). Phoronix Test Suite also allows you to upload your benchmark result along with system specification to openbenchmarking.org, so that you can compare your result with others.

Using the ready-made test profiles and suites of Phoronix Test Suite, you can easily assess the relative performance of a virtual machine. Here is the guide on how to benchmark a virtual machine with Phoronix Test Suite.

First, download phoronix-test-suite from http://www.phoronix-test-suite.com/?k=downloads, and install it (along with its dependent packages).

$ sudo apt-get install php5-cli, curl

$ tar xvfvz phoronix-test-suite-4.0.1.tar.gz
$ cd phoronix-test-suite
$ sudo ./install-sh

Phoronix Test Suite automatically downloads over network test profiles and test suites that you choose, and optionally uploads benchmark results to openbenchmarking.org. Thus, if your virtual machine is behind a proxy, you need to do one-time proxy configuration by entering:

$ phoronix-test-suite network-setup

In order to see a list of available test profiles and test suite, type the following commands respectively.

$ phoronix-test-suite list-tests
$ phoronix-test-suite list-suites

If you choose a specific test suite (e.g., ffmpeg) from the list, you can run the suite simply as follows.

$ phoronix-test-suites run ffmpeg

Once the test suite is completed successfully, the result will be saved locally, and optionally uploaded to openbenchmarking.org. In order to see the list of all saved benchmark restuls:

$ phoronix-test-suite list-results