(POOLER-158) Add capability to provision VMs on demand

This change adds a capability to vmpooler to provision instances on
demand. Without this change vmpooler only supports retrieving machines
from pre-provisioned pools.

Additionally, this change refactors redis interactions to reduce round
trips to redis. Specifically, multi and pipelined redis commands are
added where possible to reduce the number of times we are calling redis.

To support the redis refactor the redis interaction has changed to
leveraging a connection pool. In addition to offering multiple
connections for pool manager to use, the redis interactions in pool
manager are now thread safe.

Ready TTL is now a global parameter that can be set as a default for all
pools. A default of 0 has been removed, because this is an unreasonable
default behavior, which would leave a provisioned instance in the pool
indefinitely.

Pool empty messages have been removed when the pool size is set to 0.
Without this change, when a pool was set to a size of 0 the API and pool
manager would both show that a pool is empty.
This commit is contained in:
kirby@puppetlabs.com 2020-04-06 10:52:05 -07:00
parent 1f6f08d172
commit 811fd8b60f
34 changed files with 3326 additions and 1098 deletions

View file

@ -7,8 +7,7 @@ vmpooler provides configurable 'pools' of instantly-available (running) virtual
## Usage
At [Puppet, Inc.](http://puppet.com) we run acceptance tests on thousands of disposable VMs every day. Dynamic cloning of VM templates initially worked fine for this, but added several seconds to each test run and was unable to account for failed clone tasks. By pushing these operations to a backend service, we were able to both speed up tests and eliminate test failures due to underlying infrastructure failures.
At [Puppet, Inc.](http://puppet.com) we run acceptance tests on thousands of disposable VMs every day. Vmpooler manages the lifecycle of these VMs from request through deletion, with options available to pool ready instances, and provision on demand.
## Installation
@ -85,7 +84,7 @@ docker run -it vmpooler manager
### docker-compose
A docker-compose file is provided to support running vmpooler easily via docker-compose.
A docker-compose file is provided to support running vmpooler easily via docker-compose. This is useful for development because your local code is used to build the gem used in the docker-compose environment.
```
docker-compose -f docker/docker-compose.yml up
@ -113,7 +112,6 @@ A dashboard is provided to offer real-time statistics and historical graphs. It
## Command-line Utility
- The [vmpooler_client.py](https://github.com/puppetlabs/vmpooler-client) CLI utility provides easy access to the vmpooler service. The tool is cross-platform and written in Python.
- [vmfloaty](https://github.com/briancain/vmfloaty) is a ruby based CLI tool and scripting library written in ruby.
## Vagrant plugin