Prior to this commit the vsphere migration code updated the redis value
for where the VM is running (`vmpooler__vm__#{vm_name}/host`) before
attempting the migration. This meant that if the migration failed, there
was no record of what the original host for the VM was. Additionally,
the VM was marked as migrated before the migration happened which
didn't reflect reality if the migration failed.
This commit moves the redis update during migration to after the
migration has completed. This means that if an exception is thrown in
the migration code, the original host won't be lost and the host won't
be considered as migrated when it was not.
This commit adds a capability to vmpooler to reset a pool, deleting its ready and pending instances and replacing them with fresh ones. Without this change vmpooler does not offer a mechanism to reset a pool without also changing its template.
Previously, we restricted the adjective and noun portion of the name
each to 7 characters to ensure that the final name would not be more
than 15 after adding a hyphen. Given that the _total_ length is what
matters, we can generate a noun up to 11 characters (to ensure we leave
room for a hyphen and a 3 letter adjective) and adjust our acceptable
adjective size accordingly. This lets many more names be generated than
would otherwise, while still respecting the 15 character limit.
Due to the limited set of 11 letter nouns and corresponding 3 letter
adjectives, as well as some complex combinatorics, setting the noun
length to 11 causes a net increase in conflicts. We therefore actually
set it to 10, which causes a net decrease in conflicts.
We favor generating longer nouns rather than longer adjectives (by
selecting the noun first) because longer adjectives tend to be more
unwieldy words, and thus more awkward to say and generally less fun.
* (POOLER-123) Implement a max TTL
Before this change, we could checkout a vm and set the lifetime to a
very high number which would esssentially keep the vm running forever.
Now implementing a config setting max_lifetime_upper_limit which enforces
a maximum lifetime in hours both for initial checkout and extending a
running vm
* (POOLER-123) Improve PUT vm endpoint error messaging
Prior to this commit the PUT vm endpoint didn't give any useful
information about why a user's request failed.
This commit updates PUT to output a more helpful set of error messages
in the `failure` key that gets returned in the JSON response.
* (POOLER-123) Update max_lifetime_upper_limit key
This commit switches the max_lifetime_upper_limit key from being a
symbol to being a string, which is what the config hash seems to contain.
* (maint) Add option to disable Redis persistence in docker-compose
This commit is just a handy little command override to the redis
container to prevent persistence.
Prior to this commit the pooler had no awareness of the complete set of
hostnames that are currently in use. This meant that it was possible to
allocate the same hostname twice, which would result in the original
host with that hostname becoming unreachable.
This commit adds a check for the existence of the
`vmpooler__vm__<hostname>` key before attempting to clone the vm.
This should prevent duplicate hostnames.
If the hostname is already taken, `_clone_vm` will retry with a new
random hostname multiple times before raising an exception.
Prior to this commit the hostname_shorten regex wouldn't match the
updated human readable hostnames because they contain dashes.
This commit updates the regex to capture dashes in the hostname, and
adds a few specs to verify that behavior.
Prior to this commit hostnames for VMs provisioned by vmpooler were 15
random characters. This is difficult for humans to tell apart.
This commit updates the naming to use the `spicy-proton` gem to generate
adjective noun pair names for the VMs, which I think would be easier for
humans to tell apart, as well as fun and memorable to say.
The random name should not exceed 15 characters in order to prevent
issues with NETBIOS, etc as discussed in the attached ticket.
This commit updates the way that checkoutlock is defined so it is not passed through bin/vmpooler. Without this change there's an unnecessary layer the mutex passes through.
This commit adds a shared mutex to vmpooler API so that checkout requests can be synchronized across threads. Without this change it is possible in some scenarios for vmpooler to allocate the same SUT to different API requests for a VM.
The host['boottime'] variable in the function _check_ready_vm no longer
has its parent object in reference due to the refactoring in pull
request #269. So in order to get the same information without the
performance impact from duplicate object lookups, we get similar
information from the time that the VM is ready.
This commit updates the create_linked_clone pool option to correctly detect when linked clones have been set at a pool level. Without this change a pool setting create_linked_clone to false is not interpreted correctly, and a linked clone is created if possible.
This change adds the running host for a VM to the API data available via /vm/hostname. Without this change the running host would be logged to vmpooler log, but not available any other way. Additionally, the data will specify if a machine has been migrated. Without this change parent host data for a vmpooler machine is not available via the vmpooler API.
This commit adds a new configuration parameter to allow setting whether to create linked clones on a global, or per pool basis. Without this change vmpooler would always attempt to create linked clones. The default behavior of creating linked clones is preserved.
This allows the user to change the cluster in which the targeted pool
will clone to. Upon configuration change, the thread will wake up and
execute the change within 1 second.
This commit updates the reference to domain from vmpooler config. Without this change the domain value is read as an empty string and breaks checkouts.
This commit duplicates the vm_ready? check to the API layer to allow for API to validate that a VM is alive at checkout. Without this change API relies upon the checks in pool_manager validating pools. This change should allow for additional insight into whether a machine is in a ready state and resopnding at checkout time.
Before this change looping over many pools would query the redis backend
for each pool, leading in slow response from the backend for configurations
with many pools (60+)
Changed the requests to use redis pipelines https://redis.io/topics/pipelining
This is supported since the beginning, so will not force any redis update for
users. The pipeline method runs the queries in batches and we need to loop
over the result and reduces the number of requests to redis by N=number of
pools in the configuration.
Before this change we used the API /status endpoint to get specific information
on pools such as the number of ready VMs and the max.
This commit creates two new endpoints to get to that information much quicker
1) poolstat?pool= takes a comma separated list of pools to return, and will provide
the max, ready and alias values.
2) /totalrunning will calculate the total number of running VMs across all pools
This commit updates how migrating and pending queues are processed. Sets to be processed are created with sadd in redis, and iterated over as a list in ruby. The latest member is added to the beginning of this set in redis, and becomes the first member of the set in ruby. To ensure that items are processed in the order they are added it is necessary to reverse the list before iterating through its members. Without this change the newest members of the set are processed first, which creates inconsistent times to evaluation.
This commit updates how a VM is checked out to ensure that there is no window where the VM could be considered discovered, and therefore destroyed. Without this change the VM is retrieved by calling 'spop' on the ready queue, and then adding it to the running queue. This change moves to selecting the VM by retrieving the last member of the set, and moving it with 'smove' from ready to running. As a result of this change vmpooler moves from retrieving the VMs from the ready state randomly, to instead retrieve the oldest VM in the queue. This change should reduce churn where it would otherwise not be required to satisfy demand.
