(POOLER-158) Add capability to provision VMs on demand

This change adds a capability to vmpooler to provision instances on
demand. Without this change vmpooler only supports retrieving machines
from pre-provisioned pools.

Additionally, this change refactors redis interactions to reduce round
trips to redis. Specifically, multi and pipelined redis commands are
added where possible to reduce the number of times we are calling redis.

To support the redis refactor the redis interaction has changed to
leveraging a connection pool. In addition to offering multiple
connections for pool manager to use, the redis interactions in pool
manager are now thread safe.

Ready TTL is now a global parameter that can be set as a default for all
pools. A default of 0 has been removed, because this is an unreasonable
default behavior, which would leave a provisioned instance in the pool
indefinitely.

Pool empty messages have been removed when the pool size is set to 0.
Without this change, when a pool was set to a size of 0 the API and pool
manager would both show that a pool is empty.
This commit is contained in:
kirby@puppetlabs.com 2020-04-06 10:52:05 -07:00
parent 1f6f08d172
commit 811fd8b60f
34 changed files with 3326 additions and 1098 deletions

View file

@ -56,11 +56,11 @@ describe Vmpooler::API do
context 'without history param' do
it 'returns basic JSON' do
create_ready_vm('pool1', 'vm1')
create_ready_vm('pool1', 'vm2')
create_ready_vm('pool1', 'vm3')
create_ready_vm('pool2', 'vm4')
create_ready_vm('pool2', 'vm5')
create_ready_vm('pool1', 'vm1', redis)
create_ready_vm('pool1', 'vm2', redis)
create_ready_vm('pool1', 'vm3', redis)
create_ready_vm('pool2', 'vm4', redis)
create_ready_vm('pool2', 'vm5', redis)
get '/dashboard/stats/vmpooler/pool'
@ -90,11 +90,11 @@ describe Vmpooler::API do
end
it 'returns JSON with history when redis has values' do
create_ready_vm('pool1', 'vm1')
create_ready_vm('pool1', 'vm2')
create_ready_vm('pool1', 'vm3')
create_ready_vm('pool2', 'vm4')
create_ready_vm('pool2', 'vm5')
create_ready_vm('pool1', 'vm1', redis)
create_ready_vm('pool1', 'vm2', redis)
create_ready_vm('pool1', 'vm3', redis)
create_ready_vm('pool2', 'vm4', redis)
create_ready_vm('pool2', 'vm5', redis)
get '/dashboard/stats/vmpooler/pool', :history => true
@ -140,18 +140,18 @@ describe Vmpooler::API do
end
it 'adds major correctly' do
create_running_vm('pool-1', 'vm1')
create_running_vm('pool-1', 'vm2')
create_running_vm('pool-1', 'vm3')
create_running_vm('pool-1', 'vm1', redis)
create_running_vm('pool-1', 'vm2', redis)
create_running_vm('pool-1', 'vm3', redis)
create_running_vm('pool-2', 'vm4')
create_running_vm('pool-2', 'vm5')
create_running_vm('pool-2', 'vm6')
create_running_vm('pool-2', 'vm7')
create_running_vm('pool-2', 'vm8')
create_running_vm('pool-2', 'vm4', redis)
create_running_vm('pool-2', 'vm5', redis)
create_running_vm('pool-2', 'vm6', redis)
create_running_vm('pool-2', 'vm7', redis)
create_running_vm('pool-2', 'vm8', redis)
create_running_vm('diffpool-1', 'vm9')
create_running_vm('diffpool-1', 'vm10')
create_running_vm('diffpool-1', 'vm9', redis)
create_running_vm('diffpool-1', 'vm10', redis)
get '/dashboard/stats/vmpooler/running'