A reservation is the amount of physical RAM or the amount of CPU cycles that are reserved for the virtual machine.

Memory Reservation
If a virtual machine has a memory reservation but has not yet accessed its full reservation, the unused memory can be reallocated to other virtual machines.

Memory Reservation Used
Used for powered-on virtual machines, the system reserves memory resources according to each virtual machine’s reservation setting and overhead. After a virtual machine has accessed its full reservation, ESX Server allows the virtual machine to retain this much memory, and will not reclaim it, even if the virtual machine becomes idle and stops accessing memory. TPS will work however within reserved memory.

To recap the info stated in the Resource Management Guide, when a VM hits its full reservation, ESX will never reclaim that amount of reserved memory even if the machine idles and drops below its guaranteed reservation. It cannot reallocate that machine memory to other virtual machines.

A virtual machine will only power on if its reservations can be guaranteed.

When considering this scenario you should set an reservation on business critical VMs. Set it to an minimum of 35% of the configured memory and only set it higher if hosts/clusters are experiencing serious memory congestion and the VM is also seriously affected.

Sets a cap on the consumption of CPU time or physical memory by the virtual machine.

Say you have a VM that has 1 GB of RAM configured, and you set the limit to 2 GB. Does this make sense? NO! A VM will never use more than the configured memory. What if you set the limit to 512 MB. What will happen? Since the VM is only allowed to use 512 MB of physical memory the remaining memory will be swapped by the VMKernel.

When modern OS's boot, one of the first things they do is check to see how much RAM they have available then tune their caching algorithms and memory management accordingly. Applications such as SQL, Oracle and JVMs do much the same thing.

I guess the best way to explain in one line is: The limit is not exposed to the OS itself and as such the OS and the applications will suffer and so will the service provided to the user.

Since OS's and application might request RAM (sometimes even everything they can) which they don't always need it is better to decrease provisioned memory than to create an artificial boundary by applying a memory limit. The limit will more than likely impose an unneeded and unwanted performance impact. Simply lowering the amount of provisioned memory might impact performance but most likely will not as the OS will tune it’s caching algorithms and memory management accordingly.

Note: Never set a limit in a template. If, after cloning, the configured memory gets raised the limit will not automatically be raised as well resulting in affected performance.

So, are there use cases for using limits? There are:

Sometimes, for political reasons you want servers to behave badly. Just to prove your point by cheating.

You temporarily have to really overcommit a server so you can do maintenancance (or something like that) on another server. So you can selectively squeeze VMs or resource pools to free up memory temporarily.

Sometimes software is hard coded to not run unless there is a certain amount of CPU and or RAM available. Even when they never use it. A limit (but still high enough to prevent swapping) might do the trick.

This is an example of what happens when you set an limit under the configured memory of a Windows XP guest. I just tried a few things and performance went down really fast. I just used OS included software and a pdf reader, no office, no Firefox etc. Afterwards I tried to manipulate the figures by removing Guest Virtual Memory and by suspending and resuming the guest but this didn't really change anything significantly. If you look at the screenshots I'd say I'm impressed by the memory techniques VMware uses. There is swapping by the VMKernel but a lot of memory is compressed and shared. However, with some larger applications as office and Firefox I think there would be a lot more swapping.

The most important thing to say about shares is this: They only come in the equation when there is congestion.

Shares specify the relative priority or importance of either a resource pool or virtual machine. If a resource pool or VM has twice as many shares than a another resource pool or VM it will be entitled to twice as many resources than the other.

Keep in mind that if you assign custom shares to a VM they will be removed when a virtual machine is vMotioned or migrated to another host.

A resource pool is a logical abstraction for managing resources. It can be used for dividing resources between different VMs and can be grouped to create a hierarchy. The topmost resource pool is called the root resource pool. Each standalone host and each DRS cluster has an (invisible) root resource pool that groups the resources of the host or the DRS cluster. According to the documentation using resource pools can have these benefits:

• Flexible hierarchical organization
• Isolation between pools and sharing within pools
• Access control and delegation
• Separation of resources from hardware
• Management of sets of virtual machines running a multitier service

Resource Pools however hold a big catch when using them. Resources are divided at the resource pool level FIRST. Consider what that means. Say you have three resource pools, the first with 10 VMs, the second one with 3 VMs and the third one with 1 VM. They get respectively high, normal and low shares. This will end up that every VM in the first resource pool gets 400 shares, every VM in the second pool gets 667 shares and the single VM in the third pool gets 1000 shares. The picture below makes it all clear:

Note: The ratio for dividing shares between pools using “High”, “Medium” and “Low” is 4:2:1.

Luckily (or not? ) you can assign custom shares to pools. There is a downside, that will cost you a lot of operational overhead. For example, for every VM you add to a pool you will have to add 1000 shares to the resource pool if normal, 4000 if high and 500 if low.

This is an extra attribute you can configure on resource pools. It allows a resource pool that cannot satisfy a reservation request to search through its hierarchy to find unreserved capacity to satisfy the reservation request.

Note: Expanded reservations are not released until the virtual machine that caused the expansion is shut down or its reservation is reduced.

http://frankdenneman.nl/2009/12/impact-of-memory-reservation/
http://www.yellow-bricks.com/2010/07/06/memory-limits/
http://www.yellow-bricks.com/2010/02/22/the-resource-pool-priority-pie-paradox/
http://www.yellow-bricks.com/2009/11/13/resource-pools-and-shares/