Virtualization and Performance Testing

  • Questions
  • Virtualization and Performance Testing
Question ID: 104137
2
0

What are the pros, cons and considerations for installing LoadRunner or Performance Center in a virtualized environment? Can anything be run from virtual machines?

Marked as spam
Posted by (Questions: 1, Answers: 3)
Asked on June 28, 2010 5:36 pm
22 views
Answers (4)
4
Private answer

The following considerations apply when planning the deployment of Performance Center and/or LoadRunner within a virtualized environment: This information comes from an HP Software kb article on the subject.

a) Load Generator

This component can be affected by timing issues that result from being operated within a virtual environment. As a consequence it is not recommended to run the Load Generator for load testing purposes in a Virtual Environment when timing accuracy is a key requirement.

For example, in the case of VMware's ESX virtualization technology, there are issues in relation to the accuracy of the system clock that the operating system uses for time management. In physical hardware, the operating system can use hardware clock cycles to determine (relative) system time. This direct link is broken when using virtual machines as ESX now hands out CPU cycles in accordance with its management of system resources.

This means that when using VMware (or other virtualization technologies where applicable) there is the potential for the accuracy or precision of time measurements (e.g. transaction response time) to be reduced. When deploying load generators on virtual machines the level of accuracy required should be considered and a physical hardware platform utilized to run this component if accurate measurement data (e.g. milliseconds) must be captured. Where the requirement is for lower levels of measurement accuracy (e.g. seconds) virtual machines can be used for this component.

Furthermore it is worth noting that in environments where multiple load generators are in use and where accurate timing is required the best practice is that each load generator system is implemented using a common hardware and software platform so that these systems are equivalent. This avoids any differences in timing occurring during load tests. If load generators are implemented within a virtualized environment it will be difficult to ensure at all times that the system resources that are made available to each virtualized load generator are equivalent.

b) Controller

This component can be implemented within a virtualized environment. However in some cases where timestamp offsets are applicable, the level of precision obtained within a virtual environment will be reduced in a similar way to that described above for load generators. Therefore again if accurate timing is required it may be necessary to utilize a physical hardware platform for this component.

c) User & Admin Site

These components can be implemented within a virtual environment given that their role is primarily to manage and support the performance testing solution.

d) File Server

This component can be implemented within a virtual environment given its supporting role to management of the performance testing solution.

e) Database Server

This component can be implemented within a virtual environment given its supporting role to management of the performance testing solution.

f) VuGen

Apart from possible timing issues which may not be a critical consideration when developing scripts, this component can be implemented within a virtual environment.

g) Analysis

Given that the analysis of load testing results is not time dependent, this component can also be implemented within a virtual environment.

The following reference provides more information with respect to the timing issue for VMware virtualisation technology:

http://www.vmware.com/pdf/vmware_timekeeping.pdf

An additional aspect to also consider is the licensing of the product and how that may be affected by it's use within a virtualized environment - in particular with respect to the range of operations that are available when managing virtual machines. This may be an issue depending

Marked as spam
Posted by (Questions: 6, Answers: 167)
Answered on June 28, 2010 6:37 pm
2
Private answer

For LoadRunner, the controller can be virtualized as long as it is not starved for resources. Keep an eye on its CPU and memory utilization and add more resources as required. The same goes for the Controller hosts in Performance Center.

The LoadGenerators are already virtualized (thus the reason they're called ''virtual users'') so virtualizing the LGs is not recommended.

In Performance Center, any of the servers can be virtual but, again, it is not recommended that the Load Generators be virtual machines.

Of course, as with any virtualization scheme, performance of the machines could be an issue. The User site would be the most likely source of performance issues since it is the primary ''user facing'' application in PC.

As far as the systems under test are concerned, the testing software (LoadRunner or Performance Center) doesn't know or care whether the servers are virtual or not. However, performance can be drastically affected by virtualization: the servers under test share resources with other virtual machines (whether part of the test or not) and, therefore, performance in a virtual environment might not accurately reflect performance in a non-virtualized environment.

It is always recommended that the load testing environment should mirror, as closely as possible, the production environment: extrapolating results in most multi-tier environments is a guess unless comparable tests have been run in both environments so that there is a ''performance factor'' that can be applied, such as, we know our production environment can handle 3 times the test environment, and that performance factor has been shown to be reliable.

Marked as spam
Posted by (Questions: 1, Answers: 5)
Answered on June 28, 2010 6:35 pm
1
Private answer

HPE Performance Center team state:

For ***Virtual Environments***

> **HP ALM Performance Center is certified
> to work with VMWare ESX/ESXi 5.0 and
> higher. Due to the rapidly evolving
> architectures provided by
> Virtualization vendors as long as the
> 3rd party vendor guarantees full
> compatibility of the virtualized
> environment to the HP ALM Performance
> Center approved system requirements
> for physical hardware, then HP ALM
> Performance Center will function as
> designed.**

*emphasized text*

Marked as spam
Posted by (Questions: 12, Answers: 361)
Answered on February 1, 2017 9:15 pm
0
Private answer

As virtualization keeps improving, the main reason to not to recommend virtualization was especially on the load generators.
This is mostly because of the clock of the machines, used to get slight discrepancies.
But as it can be guessed the discrepancies were small.
If your test is very suceptible to slight response time metric variations, it is not recommended to use virtualization for load generators.

For the rest of the solution, it is only recommended that the virtual machines have good enough reosurces, as they are stated on the installation guides. If not slightly exceeded to ensure.

Marked as spam
Posted by (Questions: 0, Answers: 1)
Answered on April 3, 2015 7:34 pm