Previously the Pool Manager would use vSphere objects directly. This commit
- Modifies the pool_manager to use the VM provider methods instead
- Splits the destroy_vm function into two. One function spawns the thread
while the other actually does the work. This makes testing much easier.
Previously the Pool Manager would use vSphere objects directly. This commit
- Modifies the pool_manager to use the VM provider methods instead
- Splits the check_ready_vm function into two. One function spawns the thread
while the other actually does the work. This makes testing much easier.
Previously the Pool Manager would use vSphere objects directly. This commit
- Modifies the pool_manager to use the VM provider methods instead
- Modified to return true or false to indicate that the VM was failed
Previously the Pool Manager would use vSphere objects directly. This commit
- Modifies the pool_manager to use the VM provider methods instead
- Removes the open_socket method and tests as it is only required in the vSphere
VM provider
Previously the base VM provider class was added however it was missing various
functions from its definition. This commit:
- Modifies the VMPooler configuration to add an empty provider config. if the
provider config is missing
- Helper method to return all of the pools this provider is responsible for
Previously the only VM Provider was vSphere however this made testing and making
changes difficult as it required a functioning vSphere instance. This commit
adds a Dummy Provider which presents a VM provider to Pool Manager but manages
provisioned "VM"s in a hashtable. The Dummy Provider can also be configured to
randomly fail operations and take random amounts of time to perform operations,
such as cloning a VM, which is useful to see how the Pool Manager copes with
these events.
This commit also updates the configuration YAML documentation and adds
appropriate unit tests.
Previously the vSphere provider did not implement any of the required methods
from the base class. This commit modifies the vSphere provider so that in
can properly implement the following methods:
- name
- vms_in_pool
- get_vm_host
- find_least_used_compatible_host
- migrate_vm_to_host
- get_vm
- create_vm
- destroy_vm
- vm_ready?
- vm_exists?
- create_disk
- create_snapshot
- revert_snapshot
This commit also includes changes to syntax for rubocop violations.
Previously it was expected that the timeout setting should be passed when
determining whether a VM was ready. However this should be in the pool
configuration and is not required to be a method parameter.
Previously many of the methods did not have a pool name passed as a parameter.
While this may be ok for the vSphere provider, it may not for other providers.
This commit changes the base provider to consistently use (pool,vm,other..) as
method parameters. This commit also updates the base spec tests as well.
Additionally:
- Updated documentation around expected error states
- Updated documentation to be more consistent in format
- Added snapshot and disk manager functions and unit tests
- Update the initialization method to take in a more formal defintion with
required global objects for metrics, logging and configuration
- Added helper functions
- logger : Allows providers to log information as per Pool Manager
- metrics : Allows providers to submit metrics as per Pool Manager
- provider_options : Allows providers to access initialization options for a
provider
- pool_config : Get the configuration for a specific pool
- provider_config : Get the configuration of this specific provider
- global_config: Get the VMPooler global configuration
Previously all log messages may be written to a text file, however during
development or debugging it is also useful if the log messages are written to
the console. This commit changes the logger class to emit messages to the
console, via `puts`, if the VMPOOLER_DEBUG environment variable is set.
Previously, all calls to the vSphere API assumed an instance variable called
`@connection`. This commit prepares the provider for a connection pooler:
- Removes all references to `@connection` where needed and funnels all calls to
get a vSphere connection through the newly renamed method `get_connection`.
For the moment, this still uses `@connection` behind the scenes but will make
it easier in the future to migrate to a connection pooler
- Removes all references and tests for the ensure_connected method as it's no
longer required
- All methods that explicitly need a connection object will have this as part of
the method parameters
- The connect_to_vsphere method has been changed so that instead of setting the
instance level `@connection` object it just returns the newly created connection.
This can then be easily consumed by a connection pooler later.
Previously, the vSphere Provider had two methods called
`find_least_used_compatible_host`: one in the base class and one in the vSphere
helper methods. This commit renames the vSphere helper methods and tests to
`find_least_used_vsphere_compatible_host` to stop the conflict.
Previously, in `find_vm` and `find_pool` if in an expected object was found the
entire pool manager, and probably API process, will terminate due to the use of
`abort`. This commit changes the use of abort to raise so that the error can be
trapped and handled instead of the entire process being shutdown. This is also
required so that the methods can be tested, otherwise rspec is shutdown
prematurely.
Previously, it was not able to mock objects to impersonate various RBVMOMI
objects. This commit changes the case statement to use `base.is_a?`
which can be mocked and allow mocked objects to mimic real objects.
This commit fixes minor rubocopy violations in eleven source files. Minor
violations are those that include formatting, single quotes, and recently added
classes.
Previously, it was not able to mock objects to impersonate a RbVmomi::VIM::Folder
object. This commit changes the case statement to an if statement using is_a?
which can be mocked and allow mocked objects to mimic real objects.
VM provisioning will be handled by VM Providers. This commit renames the use of
vsphere to provider where appropriate and changes the per-pool helper from
vsphere to providers to more accurately represent it's intended use.
Previously all of the VM provisioning code was intertwined with the VM lifecycle
code e.g. The VSphere specific code is mixed with Redis code. This makes it
impossible to add aditional providers or disable VSphere integration. This
commit begins the process to refactor the VSphere code out of the lifecycle code
by introducing the concept of VM Providers. A Provider will contain the logic/
code to manage VMs i.e. create/destroy/inquire. Therefore the Pool Manager can
query a strict interface into one or more Providers. Initially only a VSphere
provider will be available.
This commit adds the base class for all providers and describes the API or
contract that the Pool Manager will use to manage VMs.
In commit 03ad7bfa46 the global variables for credentials was change to an
instance variable however one reference was missed. This commit fixes this
omission.
This commit updates usage of global variables in vsphere_helper to be instance variables. They do not need to be global variables as brought up in issue #194. Without this change we are setting global variables when they are not needed.
This commit updates VM Pooler to fix any existing rubocop offenses and also
fixes any offenses in the lib/vmpooler.rb file. This commit also regenerates
the Todo file.
Previously, the clone_vm method took various VSphere specific parameters e.g.
template folder. However in order make VMPooler less VSphere specific this
method should just take the pool configuration and then it can determine the
appropriate settings itself. This commit also moves the threading to a clone_vm
while the actual method which does the work is now _clone_vm as per all other
multithread worker methods in pool_manager. This commit also updates the spec
tests appropriately.
Previously in check_ready_vm, if the VM is powered off, the VM is moved in
redis however the function doesn't return there, and instead then checks if the
hostname is the same, and then if TCP socket 22 is open. This is unnecessary as
we already know the VM is turned off so of course the hostname is wrong and TCP
22 is unavailable. The same applies for the VM hostname.
This commit instead returns after it is found a VM is no longer ready. This
commit also amends the spec tests for the correct behaviour.
Add spec tests for check_snapshot_queue
Previously the check_snapshot_queue method would execute the loop indefinitely
as it did not have a terminating condition. This made it impossible to test.
This commit modifies the check_snapshot_queue method so that it can take a
maxloop and delay parameter so that it can be tested.
Add spec tests for check_disk_queue
Previously the check_disk_queue method would execute the loop indefinitely as it
did not have a terminating condition. This made it impossible to test. This
commit modifies the check_disk_queue method so that it can take a maxloop and
delay parameter so that it can be tested.
Add spec tests for check_pool
Previously the check_pool method would execute the loop indefinitely as it did not
have a terminating condition. This made it impossible to test. This commit
modifies the check_pool method so that it can take a maxloop and delay parameter
so that it can be tested.
Add spec tests for execute!
Previously the execute! method would execute the loop indefinitely as it did not
have a terminating condition. This made it impossible to test. This commit
modifies the execute! method so that it can take a maxloop and delay parameter
so that it can be tested.
If a user attempts to start vmpooler using dummy authentication
without setting the environment variable VMPOOLER_DEBUG, the vmpooler
will now refuse to start.
Previously it was difficult to do local development as VMPooler requires an LDAP
service for authentication. This commit adds a dummy authentication provider.
The provider has passes authentication if the username and password are
different, and fails if the username and password are the same. This commit
also updates the documentation in the config YML file.
If `ENV['VMPOOLER_CONFIG']` is defined, it is read in as a YAML
configuration. This allows vmpooler to run in a docker daemon via
`docker run -e VMPOOLER_CONFIG -p 80:4567 -it vmpooler` rather than
embedding a YAML file within the container.
This commit updates vmpooler to ensure clone errors, and other pool manager errors are raised to the parent method. Without this change vmpooler gets stuck after a connection fails during clone operations and will not attempt to clone again.
This commit updates vmpooler to clear the migrations queue at application start time. When the application is shut down it is not considerate of any activities, like migrations, in flight. The result is that when the application is started again any stale entries in vmpooler__migration will be left until manually removed, which can prevent migrations from occurring.
This commit adds retry logic and configurable delays to vsphere helper.
Without this change vmpooler instances that have large numbers of pools
can create enough connections in a short period of time to cause vcenter
issues.
