3. Why Choose Integrity VM?
The newest addition to the partitioning continuum
provides OS isolation while allowing sharing of CPU and I/O resources
with multiple partitions.
Key Benefits
We will again compare VMs with each of the other
partitioning alternatives to provide some context for discussing the
key benefits of implementing Integrity VM.
VMs provide the same level of OS and
software-fault isolation as nPars but provides it at a much higher level
of granularity. You can share CPUs and I/O cards and you can even
provide differentiated access to those shared resources. You have
control over how much of each resource should be allocated to each
partition (e.g. 50% for one VM and 30% for another). As with vPars,
sharing and flexibility comes at the cost of hardware-fault isolation.
This is why we recommend first partitioning the system with nPars and
then using other partitioning solutions to further partition the nPars
to provide finer granularity and increased flexibility. There is one
other benefit when comparing VMs to nPars—you can run Integrity VM on
non-cell-based platforms. Any Integrity system that supports a standard
HP-UX installation can be set up to run Integrity VM.
VMs and vPars provide many of the same benefits.
You have OS-level isolation, so you can create these partitions and
tune the operating systems to meet the specific needs of the
applications that run there. This includes kernel tunables as well as
patches and shared library versions. The key differentiation for the
Integrity VM product is its ability to share CPUs and I/O cards. This
makes it much more suitable for very small workloads that still need the
OS-level isolation.
One other interesting thing you can do with VMs
that you can't do with vPars is that you can create an ISO image of a CD
and mount that image as a virtual CD onto any or all of your VMs. This
is very convenient for the installation of updates or patch bundles that
need to be applied to multiple VMs. One other significant benefit of
Integrity VM is its future support for Windows, Linux and OpenVMS. VMs
also support I/O virtualization solutions like Auto-Port Aggregation on
the VM host so that you can create a large I/O interface and then share
it with multiple VMs. This allows the VMs to get access to losts of
bandwidth as well as the redundancy you get with API without requiring
any special configuration on the VMs themselves.
The last comparison for this section is with
Secure Resource Partitions (SRPs). There are many similarities in how
Secure Resource Partitions and Integrity VM manage the resources of the
system. The implementations of these controls are different, but they
have many of the same user interface paradigms. One key thing you get
with VMs is OS-level isolation, including file systems, namespaces,
patch levels, shared library versions, and software faults. You get none
of these with Secure Resource Partitions.
Key Tradeoffs
There are a few key tradeoffs of the Integrity
VM product that you should be aware of before deciding whether it is the
right solution for you. These include no support for HP 9000 systems
and performance of some I/O intensive workloads.
The first of these is the fact that Integrity VM
requires features of the Itanium processor that are not available or
very different on Precision Architecture.
Because Integrity VM is a fully virtualized
system technology, all I/O traffic is handled by virtual switches in the
VM monitor. This makes it possible to have multiple VMs sharing a
single physical I/O card. However, this switching imposes some overhead
on I/O operations. This overhead is relatively small, but it is impacted
by the fact that the virtual CPU that is holding the I/O interrupt that
is servicing the I/O request doesn't have all the CPU cycles on the
real CPU. This makes it possible for the interrupt to come in when
another VM has control of the CPU, which further delays the receipt of
the I/O that was requested. When the system is lightly loaded, this
impact can be fairly minor, but when the CPUs are very busy and there
are I/O intensive workloads issuing many I/O requests, you can expect a
higher level of overhead. Future releases of the VM product will improve
this and provide alternative solutions specifically for I/O intensive
workloads.
Integrity VM Sweet Spots
Most companies have a number of large
applications and databases that require significant resources to satisfy
their service-level requirements. Most also have a large number of
applications that are used daily but have a small number of users and
therefore don't require significant resources. Some of these are still
mission critical and require an isolated OS instance and a
mission-critical infrastructure like that available on HP's Integrity
servers. These are the applications that are best suited to VMs. They
have short-term spikes that may require more than one CPU, but their
normal load for the majority of the day is a fraction of a CPU. These
would normally be installed on a small system, possibly two to four
CPUs, to meet the short-term peaks. However, the average utilization in
this environment is often less than 20%. Putting these types of
workloads in a VM provides the flexibility to scale the VM to meet the
resource requirements when the load peaks but scale back the resources
when the workload is idle. Putting a number of these workloads on an
nPar or small server allows you to increase the overall utilization
while still providing isolation and having sufficient resources
available to react to the short term spikes.
Sweet Spot #1: Small Mission-critical Applications
If you have applications that don't need a
whole CPU most of the time but do need OS isolation, Integrity VM
provides both granularity and isolation.
Another sweet spot is the ability of Integrity
VM to support small CPU-intensive workloads. It turns out that
CPU-intensive workloads incur very little overhead inside a VM. We have
seen cases where the difference in performance for some CPU-intensive
benchmarks in a VM compared to a stand-alone system was less than 1%.
This was using a single virtual CPU VM, so if you have small
CPU-intensive workloads, a VM is a great option.
Sweet Spot #2: Small CPU Intensive Applications
Single CPU Integrity VM carry very little performance overhead for CPU-intensive applications.
The first release of Integrity VM will be tuned
for 4 virtual CPUs. A real sweet spot for VMs is running a bunch of four
virtual-CPU VMs on a system or nPar with four or eight physical CPUs.
This way there is a very even load of virtual CPUs to physical CPUs and
each VM can scale up to nearly four physical CPU speeds. If you have a
handful of workloads that normally average less than one CPU of
consumption but occasionally spike up from two to four CPUs at peak,
running a number of these on a fouror eight-CPU system will allow the
average utilization of the physical resources to exceed 50% while still
allowing each VM to scale up to four CPUs to meet the peak demands when
those usage spikes occur.
Sweet Spot #3: VMs with the same CPU count as the system or nPar
Running a number of four-virtual-CPU VMs on a
four-physical-CPU system or partition allows sharing of CPUs and ensures
an even load across the physical CPUs. This will also work if there are
a number of four-virtual-CPU VMs running on an eight-CPU system or
nPar—but the key is you want to have an even load on the physical CPUs.
A derivative of this sweet spot is one where you
run a handful of application clusters in VMs that share the physical
servers they are running on. Figure 2 shows an example of three application clusters with three two-CPU nodes each.
These applications will occasionally peak to the
point where they need more than one CPU on each node, but the average
utilization is in the 10–15% range. Now let's consider running these
same clusters using VMs. This is shown in Figure 3.
We have configured these as four-virtual-CPU VMs
and they are running on four-physical-CPU systems or nPars. Now let's
consider what we have done. We have:
Nearly doubled the maximum CPU capacity
for each cluster—because each virtual CPU can be scaled to get close to a
physical CPU if the other clusters are idle or near their normal
average load. Each cluster can now get nearly 12 CPUs of capacity if
needed.
Reduced the number of systems to manage. Although we still have nine OS images, we now have only three physical systems.
Lowered the CPU count for software licenses from 18 to 12.
Increased the average utilization of these systems.
To summarize, we have increased capacity,
lowered the software costs, lowered the hardware costs, and lowered
system maintenance costs.
Sweet Spot #4: Multiple Application Clusters in VMs
Running a number of overlapping application
clusters on a set of VMs increases average utilization, increases the
peak capacity of each cluster, and lowers hardware, software, and
maintenance costs.
4. Why Choose Secure Resource Partitions?
When security compartments were added to HP-UX
and integrated with Resource Partitions, the name was changed to Secure
Resource Partitions (SRPs). This didn't fundamentally change the
effectiveness of Resource Partitions, but it did increase the number of
cases where Secure Resource Partitions would be a good fit. This is
because you can now ensure that the processes running in each SRP cannot
communicate with processes in the other partitions. In addition, each
compartment gets its own IP address, and network traffic to each
compartment is isolated from the other compartments by the network stack
in the kernel. This, on top of the resource controls for CPU, real
memory, and disk I/O bandwidth, provides a very nice level of isolation
for multiple applications running in a single copy of the operating
system.
Key Benefits
When comparing SRPs to nPars we could probably
repeat the discussion from the section on VMs with one additional
benefit, which is a reduction in the number of OS images you need to
manage. Secure Resource Partitions provides much higher granularity of
resource allocation. In fact, it allows you to go down below 1% CPU if
you want. This might be useful if you have applications that are not
always running in the compartment—either a failover package or maybe a
batch job. You can also share memory and I/O. In fact, SRPs are
currently the only partitioning solution in the VSE that allows online
reallocation of memory entitlements. This will change with a future
version of HP-UX that will begin to support online memory reallocation
between partitions. The most significant advantage here is the fact that
you have fewer OS images to manage. Industry analysts estimate that 40%
of the total cost of ownership of a system is the ongoing maintenance
and management of the system. This includes the mundane daily tasks such
as backups, patches, OS upgrades, user password resets, and the like.
Because you would have fewer copies of the OS and applications
installed, less of your time and money would be spent managing them.
When comparing SRPs to vPars, the key things to
consider are finer granularity of resource allocation, resource sharing,
more-complete system support, and fewer OS images to manage. In other
words, they are much the same benefits when compared to nPars. Even
though vPars provide finer granularity than nPars, SRPs are still finer,
and you can share CPUs and I/O cards, which lowers the costs of running
small partitions. Also, the tradeoffs on the types of systems supported
by vPars don't exist for SRPs. Any system that supports HP-UX will
support Secure Resource Partitions.
When comparing SRPs to Integrity VM, there are
only a few key benefits because the resource controls are very similar.
The first is the fact that SRPs are supported on all HP-UX systems,
including PA-RISC–based HP 9000 systems, whereas VMs are supported only
on Integrity systems. The other was mentioned above for vPars and
nPars—SRPs do not require separate OS images to be built and managed for
each partition. One other benefit is that SRPs allow the sharing of
memory, which none of the OS-based partitions will support until after a
future release of HP-UX. You can also get slightly finer CPU-sharing
granularity with SRPs. VMs allow you to have a minimum of 5% of a CPU
for each VM—you can go down below 1% with SRPs, although you should be
very careful when taking advantage of that. It would be pretty easy to
starve a workload this way. This might be useful if you have workloads
that normally are not running except under special circumstances—things
such as failover or a job that only runs a small portion of the day,
week, or month. In these cases it would be important to implement some
type of automation (eg. Workload Manager) that would recognize that the
workload has started and increase the entitlement to something more
appropriate.
One other feature of Secure Resource Partitions
that isn't available in the other solutions is the full flexibility to
decide which resources you want to control, whether you want the
security controls, and even what types of security controls you want.
This, of course, can be a double-edged sword because you will want to
understand what the impact of each choice will be. We provide some
guidance on this later in this section.
Key Tradeoffs
The reduction of the number of OS images that
need to be managed has both benefits and tradeoffs. Even though the
applications are running in isolated environments, they are still
sharing the same copy of the operating system. They share the same file
system, the same process namespace, and the same kernel. A few examples
of this are:
There is one set of users: A user
logging into one compartment has the same user ID as they would have if
they logged into another compartment.
All
applications are sharing the same file system: You can isolate portions
of the file system to one or more compartments, but this is not the
default. A design goal was to ensure that all applications would run
normally in default compartments.
There
is one set of kernel tunables: You need to configure the kernel to
support the maximum value for each tunable as required for all the
compartments at the same time. Many of the more commonly updated kernel
tunables are now dynamic in HP-UX 11iV2, but this is still something to
be aware of.
Secure Resource Partitions Sweet Spots
The most compelling benefit of Secure Resource
Partitions over the other partitioning solutions is the fact that you
can reduce the number of OS images you need to manage. The tradeoff, of
course, is that you have less isolation. The primary places where this
isolation is an issue are:
When the different applications need
different library versions or their patch-level requirements are
incompatible: If different applications need different patch levels,
different kernel tunables or have some other incompatibility.
The
fact that the applications are owned by different business units: It
can be difficult to get approval to reboot a partition if it will impact
applications that are owned by different lines of business.
The first of these issues can be resolved by
focusing on the sweet spot where you run multiple copies of the same
application in a single OS image. This way they will all have the same
requirements for patches and kernel tunables. It is often a
well-understood process for consolidating multiple instances of the same
application in a single OS instance. This is especially true if the
applications are the same version of the same application. The second
issue can also be resolved if you have multiple copies of the same
application that are owned by the same line of business or if the
different lines of business have a very good working relationship and
are willing to “do the right thing” for each other.
These issues can also be mitigated by limiting
the number of applications you attempt to consolidate. If you are used
to doing consolidation, you have probably already worked through these
issues. But if you haven't, you might want to start by limiting the
number of applications you attempt to run in a single OS image. One
thing to consider is that if you consolidate only two applications in
each OS image, you have cut your OS count in half.
That is a huge savings. So if each of your business units has a number
of databases, or a number of application servers, you can put two to
three of them on each OS image and get a tremendous savings because you
cut the number of patches, backups, etc. every time you put two or more
of them in the same OS image.
Sweet Spot #1: Multiple Instances of the Same Application
If you have multiple instances of the same
version of the same application, put two or more of them in a single OS
instance and use Secure Resource Partitions to isolate them from each
other.
Another opportunity is the possibility of implementing the same overlapping cluster solution described in Figures 3-2 and 3-3
using Secure Resource Partitions. This carries the additional benefit
of a reduction of the system management overhead because you would also
be reducing the number of OS and application installations that need
management and maintenance from nine to three.
Sweet Spot #2: Multiple Overlapping Clusters
Running multiple overlapping clusters of the
same application on a set of systems or nPars provides all the benefits
of this solution in VMs but also reduces the number of OS and
application instances that need to be managed and maintained.
What Features of SRPs Should I Use?
You have a tremendous amount of flexibility in
deciding which features of Secure Resource Partitions to activate. How
do you decide which controls to use and what impact might they have on
your workloads?
Choosing a CPU Allocation Mechanism
The first control to consider is for CPU. You
have two choices here. You can use the fair share scheduler or processor
sets.
The key difference between FSS and PSETs is how
they allocate CPUs to each of the partitions. The FSS groups allocate a
portion of the CPUs cycles on all the CPUs, whereas PSETs allocates all
the cycles on a subset of the CPUs. An example would help here. Let's
consider running two applications on an eight-CPU system where we want
one of them to get 75% and the other to get 25% of the CPU. With FSS,
you would configure in 75 shares for the first and 25 shares for the
second application. The first application would use all eight CPUs but
would get only three out of every four CPU ticks and the second
application would get the other CPU tick. With PSETs, you would
configure the first application to have a PSET with six CPUs and the
other with two CPUs. Each application would only see the number of CPUs
in its PSET and would get every CPU cycle on those CPUs.
Given the sweet spot configuration of a small
number of the same application running in an OS image, all you need to
consider is whether you want to use FSS or PSETs. The only reason you
would want to use PSETs is if there was some reason that the application
required exclusive access to the CPUs. Because this is Unix, there are
no applications that require exclusive access to CPUs. However, if you
are using any third-party management tools that allocate CPU resources
themselves, you might want to find out if they will work correctly when
they get partial CPUs.
Another benefit of FSS is that it allows
sharing of unused CPU ticks. When CPU capping is turned off, any time a
partition has no processes in the CPU run queue, the next partition is
allowed to use the remainder of the CPU tick. Effectively, FSS allows
you to define a minimum guaranteed entitlement for each application, but
if one application is idle, it allows other busy applications to use
those unused shares. When everyone is busy, everyone will get their
specified entitlement.
Should I Use Memory Controls?
Memory controls are implemented as separate
memory-management subsystems inside a single copy of the operating
system. The control mechanism is paging, so the impact on applications
is transparent, with the exception of performance, of course. The key
benefit you would get from turning on memory controls is when you have a
low-priority workload that is consuming more than its fair share of
memory and causing performance problems on a higher-priority
application.
If you decide to turn on memory controls, you
then need to decide if you want to isolate the memory or allow sharing.
The tradeoff here is that isolating memory might result in some idle
memory not being available to a different application that could use it.
The other side is that without capping it is likely that more paging
will occur to reallocate memory to the partition that “owns” it.
Should I Use Disk I/O Controls?
The disk I/O controls are implemented through a
callout from the queuing mechanisms in the volume managers LVM and
VxVM. Several things to note about this are:
This only takes effect when there is
competition for disk I/O. The queue only starts to build when there are
more requests than can be met by the available bandwidth. This isn't a
problem, but you should know that an application can exceed its
entitlement if there is bandwidth available. There is no capping mode
for I/O.
This is only at the volume group
level. Multiple compartments need to be sharing some volume groups to
get the benefits of these controls.
The short answer is that turning on the disk
I/O controls is useful if you occasionally have bandwidth problems with
one or more volume groups and you have multiple applications that are
sharing that volume group.
Should I Use Security Compartments?
The default security containment is intended to
be transparent to applications. This means that processes will not be
allowed to communicate to processes in other compartments, but they can
communicate freely with other processes in the compartment. The default
is that the file system is protected only by the standard HP-UX file
system security. If you want more file system security, you will need to
configure that after creating the default compartments. Other features
include the fact that each SRP will have its own network interfaces.
You should consider turning security on if:
You have multiple different
applications and you need to be sure they won't be able to interfere
with or communicate with each other.
You
have multiple network facing applications and you want to make sure that
if the security of any of them are compromised, the damage they can
cause will be contained to the compartment they are running in.
You
want each application to have its own IP address on the system and the
packets on those interfaces will only be visible to the application they
were destined for. Multiple interfaces can share the physical cards,
but each interface and its IP addresses can be assigned only to a single
SRP.
