This commit adds vmpooler inspection of configuration issues to host
selection. Specifically, configIssue is checked, which should allow an
issue like quickstats not being reported to be identified even when the
alarm will not trigger. Without this change a host will continue to be
used if quickstats are not reported when alarms are not triggered
because of this condition, which results in a single host being the
target for all deploys and migrations, overloading the host and causing
its VMs to have degraded performance.
Previously the vsphere provider assumed that there was one and only one
datacenter (DC) in the vsphere instance. However this is simply not true for
many vSphere installations. This commit:
- Adds the ability to define a vSphere datacenter at the Pool or Provider level
whereby the Pool setting takes precedence
- If no datacenter is specified the default behaviour of picking the first DC
in the vSphere instance
- Updated all tests for the new setting
- Update the vmpooler configuration file example with relevant setting name
and expected behaviour
- Fixed a bug in the rvmomi_helper whereby if no DC was found it would return
all DCs. This is opposite behaviour of the real RBVMOMI library as it returns
nil
Previosuly in find_vmdks in vsphere_helper it uses the call
vmdk_datastore._connection to get the underlying connection, however this is
already available as function parameter. This commit removes this bad code and
the associated test fixtures.
The generic connection pooler is only responsible for managing the connection
objects, however the providers themselves are responsible for ensuring that the
connection is alive/healthy etc. Previously, the older vSphere helper would
reconnect however this was lost when the connection pooler was introduced. This
commit adds a method that checks the connection before use, and then reconnects
if the connection is in a bad state.
Previously the vSphere Provider would share a single vSphere connection for all
pools under management. This would cause issues in large environments as this
would cause errors to be thrown or operations to slow down. This commit
modifies the vSphere Provider to use a connection pool when communicating with
the vSphere API
- Uses the GenericConnectionPool object to manage the connection pool
- Uses a default connection pool size of:
Whatever is biggest from:
- How many pools this provider services
- Maximum number of cloning tasks allowed
- Need at least 2 connections so that a pool can have inventory functions
performed while cloning etc.
- A large connection_pool_timeout is used as a connection object is consumed
during a VM clone, which can take up to 2 minutes
- Removes the `get_connection` method as that is now obsolete due to the
connection pool
- Removes the `close` method as it is now obsolete
- Modified the spec tests slightly, to stop mocking get_connection as it no
longer exists, and set a super low pool timeout so that if a test fails, it
will fail quickly instead of taking the default time of 60+ seconds
Previously a connection pooler class was added. This commit modifies the Dummy
VM Provider to use a connection pooler. While the Dummy provider strictly
speaking does not use connections, this allows testing to see what happens when
connection pools are stressed or exhausted. This commit:
- Modifies functions to use a connection pool object for the public API
functions
- Modifies the VMPooler YAML with new settings for connection pool size and
timeout
Previously the vSphere based configuration was in the root of the configuration
YAML. As there is deprecation support to move the old configuration to the new
location, the vSphere provider can be updated. This commit updates the vSphere
Provider and tests to use the new configuration location under:
:providers:
:vsphere:
Previously the base VM provider class was added however it was missing various
functions from its definition. This commit:
- Modifies the VMPooler configuration to add an empty provider config. if the
provider config is missing
- Helper method to return all of the pools this provider is responsible for
Previously the only VM Provider was vSphere however this made testing and making
changes difficult as it required a functioning vSphere instance. This commit
adds a Dummy Provider which presents a VM provider to Pool Manager but manages
provisioned "VM"s in a hashtable. The Dummy Provider can also be configured to
randomly fail operations and take random amounts of time to perform operations,
such as cloning a VM, which is useful to see how the Pool Manager copes with
these events.
This commit also updates the configuration YAML documentation and adds
appropriate unit tests.
Previously the vSphere provider did not implement any of the required methods
from the base class. This commit modifies the vSphere provider so that in
can properly implement the following methods:
- name
- vms_in_pool
- get_vm_host
- find_least_used_compatible_host
- migrate_vm_to_host
- get_vm
- create_vm
- destroy_vm
- vm_ready?
- vm_exists?
- create_disk
- create_snapshot
- revert_snapshot
This commit also includes changes to syntax for rubocop violations.
Previously it was expected that the timeout setting should be passed when
determining whether a VM was ready. However this should be in the pool
configuration and is not required to be a method parameter.
Previously many of the methods did not have a pool name passed as a parameter.
While this may be ok for the vSphere provider, it may not for other providers.
This commit changes the base provider to consistently use (pool,vm,other..) as
method parameters. This commit also updates the base spec tests as well.
Additionally:
- Updated documentation around expected error states
- Updated documentation to be more consistent in format
- Added snapshot and disk manager functions and unit tests
- Update the initialization method to take in a more formal defintion with
required global objects for metrics, logging and configuration
- Added helper functions
- logger : Allows providers to log information as per Pool Manager
- metrics : Allows providers to submit metrics as per Pool Manager
- provider_options : Allows providers to access initialization options for a
provider
- pool_config : Get the configuration for a specific pool
- provider_config : Get the configuration of this specific provider
- global_config: Get the VMPooler global configuration
Previously, all calls to the vSphere API assumed an instance variable called
`@connection`. This commit prepares the provider for a connection pooler:
- Removes all references to `@connection` where needed and funnels all calls to
get a vSphere connection through the newly renamed method `get_connection`.
For the moment, this still uses `@connection` behind the scenes but will make
it easier in the future to migrate to a connection pooler
- Removes all references and tests for the ensure_connected method as it's no
longer required
- All methods that explicitly need a connection object will have this as part of
the method parameters
- The connect_to_vsphere method has been changed so that instead of setting the
instance level `@connection` object it just returns the newly created connection.
This can then be easily consumed by a connection pooler later.
Previously, the vSphere Provider had two methods called
`find_least_used_compatible_host`: one in the base class and one in the vSphere
helper methods. This commit renames the vSphere helper methods and tests to
`find_least_used_vsphere_compatible_host` to stop the conflict.
This commit fixes minor rubocopy violations in eleven source files. Minor
violations are those that include formatting, single quotes, and recently added
classes.
Previously all of the VM provisioning code was intertwined with the VM lifecycle
code e.g. The VSphere specific code is mixed with Redis code. This makes it
impossible to add aditional providers or disable VSphere integration. This
commit begins the process to refactor the VSphere code out of the lifecycle code
by introducing the concept of VM Providers. A Provider will contain the logic/
code to manage VMs i.e. create/destroy/inquire. Therefore the Pool Manager can
query a strict interface into one or more Providers. Initially only a VSphere
provider will be available.
This commit adds the base class for all providers and describes the API or
contract that the Pool Manager will use to manage VMs.
