(maint) Remove phatom VMs and ensure inventory is successful

Previously, if inventory failed for some reason, it would return an incomplete
set of VMs which could then cause the pool to perform off behaviours such as
fill the pool high than it should, or remove VMs which exist.  Also, if the
redis cache of VMs in a pool had a VM but it did not actually exist in the
inventory it would never be removed.

This commit:
- Immediately exits the check_pool if an error occurs during inventory
  collection
- Will mark a VM as completed if it exists in Redis, but does not exist in
  inventory
- Adds tests for these behaviours
This commit is contained in:
Glenn Sarti 2017-07-10 17:21:27 -07:00
parent 9b0e55f959
commit e55a8825af
2 changed files with 79 additions and 4 deletions

View file

@ -603,6 +603,7 @@ module Vmpooler
end
rescue => err
$logger.log('s', "[!] [#{pool['name']}] _check_pool failed with an error while inspecting inventory: #{err}")
return pool_check_response
end
# RUNNING
@ -615,6 +616,8 @@ module Vmpooler
rescue => err
$logger.log('d', "[!] [#{pool['name']}] _check_pool with an error while evaluating running VMs: #{err}")
end
else
move_vm_queue(pool['name'], vm, 'running', 'completed', 'is a running VM but is missing from inventory. Marking as completed.')
end
end
@ -627,6 +630,8 @@ module Vmpooler
rescue => err
$logger.log('d', "[!] [#{pool['name']}] _check_pool failed with an error while evaluating ready VMs: #{err}")
end
else
move_vm_queue(pool['name'], vm, 'ready', 'completed', 'is a ready VM but is missing from inventory. Marking as completed.')
end
end