This commit updates vmpooler to allow the API component and dashboard to
run separately from pool_manager. Without this change vmpooler does not
offer a mechanism to run only the API, or pool_manager components.
Two instances of hardcoded puma environment settings are removed. This
is still set in the init script explicitly as well as via an environment
variable in the dockerfile.
To extend the mechanism of running the API or pool_manager components to
instances running in docker an entrypoint is added in the dockerfile.
The entrypoint allows a user to specify whether to run the API or
pool_manager components when running the application. The default
behavior is preserved where both components are run.
To support these changes vmpooler.rb is updated to allow more of the
configuration to be specified via individual environment variables. It
was already possible to specify the entire config block as an
environment variable, but this is more difficult to manage and less of a
standard implementation than specifying individual parameters. Where
specified environment variable options will override a value configured
via the configuration file or environment.
The running pool configuration when starting pool_manager is loaded to
redis at pool_manager start time. This allows the API to load the
running pool configuration from redis and be able to run without
requiring the pool configuration.
Lastly, the dockerfile leveraging entrypoint will no longer start
vmpooler with the init script or write logs to a file. Instead, LOGFILE
is set to /dev/stdout and the vmpooler application is started directly.
This behavior is preferred because the log file writes to disk are an
unnecessary overhead. Without this change the docker installation will
attempt to daemonize the vmpooler application and always requires puma.
* (POOLER-81) Add time_remaining information
Before, the only time calculation displayed for a given VM was the
lifetime parameter. Added the remaining parameter which will
display time until the VM is destroyed as a float.
Additionally, start_time and end_time were added to api to return
as UTC based times (e.g. 2018-07-10 11:01:03 -0700).
* Remove abs eval from GET, rework spec tests to check each field.
This allows us to account for "flakiness" of the remaining return.
* Change datetime to RFC3339 for start_time and end_time
* Revert "(POOLER-34) Ship clone request to ready time to metrics (#277)"
This reverts commit a865e6bd2f.
* Revert "(POOLER-81) Add time_remaining information (#276)"
This reverts commit 1910cffaf7.
* (POOLER-34) Ship clone request to ready time to metrics
Before, we were already capturing this metric but we failed to ship
it anywhere. This ships the appropriate metric as `time_to_ready_state`
Dashboards can be found in grafana.
* Add spec test to ensure metric is being shipped properly on
move_pending_vm_to_ready call.
* (POOLER-81) Add time_remaining information
Before, the only time calculation displayed for a given VM was the
lifetime parameter. Added the time_remaining parameter which will
display time until the VM is destroyed in hours, minutes, seconds.
Additionally, updated the running parameter to display in a similar
fashion as time_remaining.
* Add spec testing for testing time_remaining stat
This commit updates usage of max_tries to ensure that the increment of the try value happens after evaluation of whether max_tries has been exceeded. Without this change max_tries doesn't work as advertised. Additionally, checking looks to see if try is greater than or equal to so the try count exceeding max_tries would also be detected.
This commit updates get_pool_vms to only return if the object is a VirtualMachine. Without this change a folder wtihin a VM target folder can be discovered and destroyed as though it were a VM.
This commit updates how template delta disk creation is evaluated. Without this change template deltas are created for every template on each applicatoin startup. This change updates this behavior to instead run template delta disk creation only once per template configured for a pool. Without this change it is possible to get a template to a state where the XML depth is too great to be read with default settings and the template requires a new clone to resolve.
This commit updates check_pool inventory check to prevent multiple instances from running at once. Without this change the inventory check may run in multiple threads simultaneously.
This commit renames find_folder to find_vm_folder to clarify its intent to retrieve folders from the VM hierarchy. Without this change find_folder implies it may find folders that are not within the VM hierarchy.
This commit replaces find_folder to use information already known about folders to determine their location and quickly fail when it does not exist without traversing the hierarchy. Without this change find_folder is very inneficient and can take a long time to return depending on how deep in the folder tree the folder exists.
This commit reduces object lookups used for get_vm. Without this change get_vm looks up the VM folder, which is already used and known to find the vm with find_vm.
This commit moves net/ldap require from authenticate_ldap in api/helpers to vmpooler.rb. Without this change net/ldap and rubygems are required again every time authenticate_ldap is run.
This commit updates vmpooler to support setting an array of search bases
in addition to a single base provided as a string. Without this change
it is not possible to specify multiple search bases to use with the LDAP
authentication provider. Additionally, test coverage is added to
the authentication helper method.
This commit updates pool_manager to expire a redis VM key when a clone fails. Without this change VMs that fail to clone have their metadata left forever.
This commit implements a vm_mutex hash to allow synchronizing VM operations that should only happen once across threads. Without this change pool_manager will try to evaluate or destroy a VM multiple times, which results in an error being thrown by one of the destroy attempts as only one can succeed and a duplication of resources unnecessarily when there are no errors.
This commit adds a configuration endpoint to the vmpooler API. Pool
size, and pool template, can be adjusted for pools that are configured
at vmpooler application start time. Pool template changes trigger a pool
refresh, and the new template has delta disks created automatically by
vmpooler.
Additionally, the capability to create template delta disks is added to
the vsphere provider, and this is implemented to ensure that templates
have delta disks created at application start time.
The mechanism used to find template VM objects is simplified to make the flow of logic easier to understand. As an additional benefit, performance of this lookup is improved by using FindByInventoryPath.
A table of contents is added to API.md to ease navigation. Without this change API.md has no table of contents and is difficult to navigate.
Add mutex object for managing pool configuration updates
This commit adds a mutex object for ensuring that pool configuration changes are synchronized across multiple running threads, removing the possibility of two threads attempting to update something at once, without relying on redis data. Without this change this is managed crudely by specifying in redis that a configuration update is taking place. This redis data is left so the REPOPULATE section of _check_pool can still identify when a configuration change is in progress, and prevent a pool from repopulating at that time.
Add wake up event for pool template changes
This commit adds a wake up event to detect pool template changes.
Additionally, GET /config has a template_ready section added to the
output for each pool, which makes clear when a pool is ready to populate
itself.
This commit changes the vsphere connection behavior to set insecure false. Without this change insecure is always set to true when making a connection regardless of the setting provided with the provider configuration.
This commit updates add_disk to remove propertyCollector, which was used
to back the find_vmdks method to locate the disk file on datastore and
then use its length to name the new disk. Instead, the number of disks
on the VM is used to ensure a unique disk resource title. Without this
change add_disk can take 10-50x longer due to the propertyCollector
method. Additionally, without this change propertyCollector is used in a
non threadsafe manner, which may cause stability issues for vsphere
provider backends.
This commit replaces find_vm and find_vm_heavy with a more performant and reliable mechanism of identifying VM objects. Specifically, FindByInventoryPath is able to leverage known data about a VM, its folder path and datacenter, and use that to identify whether that VM exists by its location. Without this change find_vm_heavy is called each time a VM cannot be found, which is frequent, and in doing so uses PropertyCollector in a manner that is not thread-safe. Additionally, this PropertyCollector usage does not clean up its traces, which can cause vCenter appliance instability issues on VCSA 6.x.
This commit updates the method used to load a configuration file to use
YAML.load_file, which is how it was configured previously. The
capability to specify an alternate configuration file via the
VMPOOLER_CONFIG_FILE is retained, and now works as expected.
This commit add a redis hash where there is one key per pool, and the
stored value is the last time a VM was booted e.g. the last time
a VM went from 'pending' to 'ready'. This is also displayed in the
API as lastBoot:'2018-03-23 17:43:39 +0000'. The data can then be
used by any external system, in this case our alarming system.
variable
Previously, there were two ways to configure Vmpooler, either by
changing the contents of vmpooler.yaml or by assigning the raw YAML
to the VMPOOLER_CONFIG environment variable. This commit adds a new
environment variable called VMPOOLER_CONFIG_FILE that can be assigned
the name of a config file to use. Also fixes#240 by whitelisting the
Symbol class when calling YAML.safe_load in Vmpooler.config.
This commit updates cpu_utilization_for and memory_utilization_for to detect when quickstats are not present. Without this change a nil result is transformed to 0, which is perceived as a host that has no utilization.
This commit removes an unnecessary rescue that results in duplicate clone error messages. Without this change clone failures due to unavailable host resources are logged twice. A log message is added to specify the host the VM is running on when migration_limit is not set and migration is disabled. Lastly, when a migration fails it reports the host the VM is running on in addition to the reason for the failed migration.
This change documents new vshpere specific parameters introduced related
to host selection and folder creation. Without this change these paremeters are not
documented.
This commit updates host selection to write check_time_finished whether
host retrieval was successful or not. Without this change when host
selection fails threads waiting for host selection to complete will
stuck waiting because check_time_finished doesn't update. Additionally,
because it would leave checking => true it would not attempt to inspect
and run host selection again.
Host selection for clones and migrations now make clear that no hosts
are available and fail logging a message. Without this change both
migrations and clones would fail with cryptic error messages logged
indicating clone and migrations failed.
Additionally, this change makes max_age configurable so a user can
specify that host selection should happen more or less frequently, as
required for migrations or clone operations when host selection is
enabled.
This commit updates the providers to move provider_hosts under the vsphere provider, which is the only place it's applicable. Methods where redis is passed through are updated to remove this pass through and use the globally available redis object, where applicable. Remove_vmpooler_migration_vm method is not needed and is removed.
This commit moves the migrate_vm logic to the vsphere provider. Without
this change migrate_vm has lots of vsphere specific logic in
pool_manager migrate_vm method.
