Prior to this, `pool_manager.rb` allowed the `metrics` argument to be optional,
but at this point it will be an instance of `Vmpooler::Statsd`,
'Vmpooler::Graphite', or `Vmpooler::DummyStatsd`, so making this non-optional.
Cleaned up that file's tests, cosmetically, as well as recognizing that the
behavioral difference between graphite and statsd does not depend on the pool
manager.
Prior to this, the `pool_manager.rb` library could take handles for both
graphite and statsd endpoints (which were considered mutually exclusive),
and then would use one. There was a bevy of conditional logic around sending
metrics to the graphite/statsd handles (and actually at least one bug of
omission).
Here we refactor more, building on earlier work:
- Our graphite class comes into line with the API of our Statsd and DummyStatsd classes
- In `pool_manager.rb` we now accept a single "metrics" handle, and we drop all the conditional logic around statsd vs. graphite
- We move the inconsistent error handling out of the calling classes and into our metrics classes, actually logging to `$stderr` when we can't publish metrics
- We unify the setup code to use `config` to determine whether statsd, graphite, or a dummy metrics handle should be used, and make that happen.
- Cleaned up some tests. We could probably stand to do a bit more work in this area.
Prior to this we could easily run into situations where `statds_prefix` would
be `nil` (and possibly the `statsd` handle itself). There was some significant
complexity and brittleness in how statsd was set up.
Refactored so that:
- `statsd_prefix` is no longer exposed to any callers of statsd methods
- there is now a `Vmpooler::DummyStatsd` class which can be returned when we are not actually going to publish stats, but would like to keep the calling interface consistent
- setup of the statsd handle is via just passing in `config[:statsd]`, if `nil`, this will result in a dummy handle being return
- defaulting of `server` values was fixed -- this did not actually work in the previous implementation. `config[:statsd][:server]` is now required.
- tests use a `DummyStatsd` instance instead of an rspec double.
- calls to `statsd.increment` were taking incorrect arguments (some our fault, some part of the prior implementation), and were not collecting data on which pools were "invalid" or "empty". Fixed this and are now explicitly tracking the invalid/empty pool names.
These were caused in part by dropping changes from the original PR when we
dropped the v1_spec.rb master test file (in favor of the updated and separated
versions).
There were several problems with how the pooler checked out vms with
respect to empty pools, invalid pools, and aliases:
- If the vmpooler config did not contain any aliases and the caller
requested a vm from an empty pool or a non-existent one, the vmpooler
would error with:
NoMethodError - undefined method `[]' for nil:NilClass
If the config contained a non-nil alias section, then:
- If the caller requested a vm from an empty pool and either the vm
didn't have an alias or the aliased pool was empty or non-existent, then
the request for that vm would be silently ignored. The vmpooler would
return 200 if the caller asked for multiple vms and the vmpooler was
able to checkout at least one vm. Otherwise it would return 404.
- Similarly, if the caller requested a vm from a non-existent pool, then
the request was silently ignored.
This commit adds a `pool_names` Set to the config containing all valid
pool names including aliases. This is used to determine whether a
requested template name is valid or not. This is necessary because redis
does not distinguish between empty and non-existent sets, e.g. the
following returns false in both cases:
backend.exists('vmpooler__ready__' + key)
If the caller requests a vm (single or multiple), and any vm references
an invalid pool name, we immediately return 404. Otherwise, we know the
request is for valid pool names, since the vmpooler requires a restart
to change pool names and counts.
We then attempt to acquire each vm, trying to match on pool name or
failing back to aliased pool name, as was the previous behavior.
The resulting behavior is:
- If the caller asks for at least one vm from an unknown pool, then
don't try to checkout any vms and respond with 404.
- If the caller asks for a vm, and at least one pool is empty, then
respond with 503, returning checked out vms back to the pool.
- Otherwise return 200 with the list of checked out vms.
This commit also makes `alias` optional again.
This commit also re-enables tests that were merged in from master, but
originally commented out due to the bugs described above..
* Add redis to our travis runs
* De-mockistify v1/token specs; use real redis.
Open questions:
- Do we need to do better cleanup here?
- Should we be using a separate database to prevent clobbering other local db's?
* Remove mockist tests from main suite.
* (MAINT) gitignore some common files
* (maint) Clean up some of the /vm/ tests
* (maint) Convert specs for /vm/template
* (maint) Clean up, reorganize specs
* (maint) Move extracted spec helper methods to spec_helper
* (maint) rename create_vm -> create_ready_vm
* (WIP) add partially-converted /vm/hostname specs
* (maint) clean up vm_spec
* (WIP) notes for next steps
* (maint) Define :config in token tests
Miscellaneous whitespace cleanup.
* (maint) Lift #redis definition into spec helper library
* (maint) drop unneeded clear_pool helper
Given the way we're flushing redis (which seems super performant), we don't
need to clear pools any more at the beginning of tests.
* (maint) Drop clear_pool from vm/template specs
* (maint) Update vm/hostname tag and lifetime specs
* (maint) Convert vm deletion specs
* (maint) Convert specs for vm snapshot operations
* (maint) Drop now-obsolete v1_spec.rb
* (maint) cosmetic cleanup in spec helper
* (maint) begin de-mockistifying api_spec.rb
* (maint) repair incorrect test
The mockist version of the test allows redis' scard to return nil, which
it does not actually do in real life. Verified the behavior in the code
via a debugger. Fixed the test.
* (maint) finish converting dashboard specs
* (maint) rename api_spec to dashboard_spec
* (maint) Don't clobber default redis database when running specs
This PR adds spec testing for pool 'alias' functionality introduced in
17b24d6, as well as the following previously non-existant tests:
- new tests for handling requests for a VM from a nonexistant pool
- new tests for the `POST /vm/:template` endpoint
Prior to this commit, a running VM could fail a ping check and be
destroyed. This causes issues when network hiccups occur or the machine
is performing a reboot.
A VM that is in a ready state will now be destroyed when handed back or
it hits the lifetime TTL.
As we approach an "official" v1.0.0 of the API I'd like to remove some old
nested "ok" responses. These were left in as the Beaker vmpooler
hypervisor used them, but I long-ago patched that code and I think it's
time to deprecate these.
* rename the Redis token 'timestamp' var to 'created'
* update the Redis token 'last' var when token is successfully validataed
* expose the Redis token 'last' var in GET /token route
The return values from most redis calls inform the caller of whether a
key or hash value exists. Several exists() calls can be removed in
favor of this approach.
Updated spec tests to account for a removal of exists() and ismember()
calls in API tests.
Make a note in the logfile when a pool is detected to be empty.
Also:
- vmpooler__empty__<pool> Redis key to determine when to log
- lifetime/TTL checks moved to `_check_running_vm` method
- no longer pay attention to VMware-based 'host.runtime.bootTime'
This PR implements a bunch of other stuff to account for rspec testing:
- Thread creation and looping in `check_pool`
- Everything else in `_check_pool`
Add spec tests for pool_manager#check_running_vm. In the process of
writing these tests, the method was broken in to smaller methods for
testability reasons.
This moves the inline Helpers contained in V1.rb to their own file:
helpers.rb. In making this change, any API.settings call was removed
from the helper method itself and passed through from V1.
This also adds tests for hostname shortener and validate date string.
This commit breaks apart the method check_pending_vm in to three,
smaller methods that are easier to test. There are no functionality
changes or bug fixes.
This adds spec tests for the overall functionality that was
check_pending_vm.
