Prior to this we could easily run into situations where `statds_prefix` would
be `nil` (and possibly the `statsd` handle itself). There was some significant
complexity and brittleness in how statsd was set up.
Refactored so that:
- `statsd_prefix` is no longer exposed to any callers of statsd methods
- there is now a `Vmpooler::DummyStatsd` class which can be returned when we are not actually going to publish stats, but would like to keep the calling interface consistent
- setup of the statsd handle is via just passing in `config[:statsd]`, if `nil`, this will result in a dummy handle being return
- defaulting of `server` values was fixed -- this did not actually work in the previous implementation. `config[:statsd][:server]` is now required.
- tests use a `DummyStatsd` instance instead of an rspec double.
- calls to `statsd.increment` were taking incorrect arguments (some our fault, some part of the prior implementation), and were not collecting data on which pools were "invalid" or "empty". Fixed this and are now explicitly tracking the invalid/empty pool names.
We're returning [nil,nil] in this case, meaning that name will not be set. This
means we'll get an error trying to concatenate the stats string. Use the
requested template name here instead.
These were caused in part by dropping changes from the original PR when we
dropped the v1_spec.rb master test file (in favor of the updated and separated
versions).
json 2.0.x was released on July 1 and is not compatible with ruby < 2.0.
Since we still support that version, add a pessimistic pin, which is
what we were using prior to July 1.
There were several problems with how the pooler checked out vms with
respect to empty pools, invalid pools, and aliases:
- If the vmpooler config did not contain any aliases and the caller
requested a vm from an empty pool or a non-existent one, the vmpooler
would error with:
NoMethodError - undefined method `[]' for nil:NilClass
If the config contained a non-nil alias section, then:
- If the caller requested a vm from an empty pool and either the vm
didn't have an alias or the aliased pool was empty or non-existent, then
the request for that vm would be silently ignored. The vmpooler would
return 200 if the caller asked for multiple vms and the vmpooler was
able to checkout at least one vm. Otherwise it would return 404.
- Similarly, if the caller requested a vm from a non-existent pool, then
the request was silently ignored.
This commit adds a `pool_names` Set to the config containing all valid
pool names including aliases. This is used to determine whether a
requested template name is valid or not. This is necessary because redis
does not distinguish between empty and non-existent sets, e.g. the
following returns false in both cases:
backend.exists('vmpooler__ready__' + key)
If the caller requests a vm (single or multiple), and any vm references
an invalid pool name, we immediately return 404. Otherwise, we know the
request is for valid pool names, since the vmpooler requires a restart
to change pool names and counts.
We then attempt to acquire each vm, trying to match on pool name or
failing back to aliased pool name, as was the previous behavior.
The resulting behavior is:
- If the caller asks for at least one vm from an unknown pool, then
don't try to checkout any vms and respond with 404.
- If the caller asks for a vm, and at least one pool is empty, then
respond with 503, returning checked out vms back to the pool.
- Otherwise return 200 with the list of checked out vms.
This commit also makes `alias` optional again.
This commit also re-enables tests that were merged in from master, but
originally commented out due to the bugs described above..
Add the tracking of successful, failed, invalid, and empty pool vm gets. It is possible we may want to tweak this, but have validated with spec tests and pcaps.
```
vmpooler-tmp-dev.ready.debian-7-x86_64:1|c
vmpooler-tmp-dev.running.debian-7-x86_64:1|c
vmpooler-tmp-dev.checkout.invalid:1|c
vmpooler-tmp-dev.checkout.success.debian-7-x86_64:1|c
vmpooler-tmp-dev.checkout.empty:1|c
vmpooler-tmp-dev.running.debian-7-x86_64:1|c
vmpooler-tmp-dev.clone.debian-7-x86_64:12.10|ms
vmpooler-tmp-dev.ready.debian-7-x86_64:1|c
```
* Add redis to our travis runs
* De-mockistify v1/token specs; use real redis.
Open questions:
- Do we need to do better cleanup here?
- Should we be using a separate database to prevent clobbering other local db's?
* Remove mockist tests from main suite.
* (MAINT) gitignore some common files
* (maint) Clean up some of the /vm/ tests
* (maint) Convert specs for /vm/template
* (maint) Clean up, reorganize specs
* (maint) Move extracted spec helper methods to spec_helper
* (maint) rename create_vm -> create_ready_vm
* (WIP) add partially-converted /vm/hostname specs
* (maint) clean up vm_spec
* (WIP) notes for next steps
* (maint) Define :config in token tests
Miscellaneous whitespace cleanup.
* (maint) Lift #redis definition into spec helper library
* (maint) drop unneeded clear_pool helper
Given the way we're flushing redis (which seems super performant), we don't
need to clear pools any more at the beginning of tests.
* (maint) Drop clear_pool from vm/template specs
* (maint) Update vm/hostname tag and lifetime specs
* (maint) Convert vm deletion specs
* (maint) Convert specs for vm snapshot operations
* (maint) Drop now-obsolete v1_spec.rb
* (maint) cosmetic cleanup in spec helper
* (maint) begin de-mockistifying api_spec.rb
* (maint) repair incorrect test
The mockist version of the test allows redis' scard to return nil, which
it does not actually do in real life. Verified the behavior in the code
via a debugger. Fixed the test.
* (maint) finish converting dashboard specs
* (maint) rename api_spec to dashboard_spec
* (maint) Don't clobber default redis database when running specs
They way we were using graphite was incorrect for the type of data we were sending it. statsd is the appropriate mechanism for our needs.
statsd and graphite are mutually exclusive and configuring statsd will take precendence over Graphite. Example of configuration in vmpooler.yaml.example
* (QENG-3919) spike for implementation of all-or-nothing checkout
* Fix two botched variable references
* Aggregate API helper methods
* Add specs for failed multi-vm allocation API endpoints
* (QENG-3919) Add tests for multiple vm requests
* (QENG-3919) Add (failing) specs for POST /vm/pool1+pool2 usages
This exposes the old (bad) behavior on this other code path. Will fix this up next.
* (QENG-3919) Bring query params version in line with JSON post version
Not clear to me why these had to be implemented so differently.
* (QENG-3919) extract common method from both methods of VM allocation
* (QENG-3919) Naming fix, cosmetic cleanups
I mean, I presume all these commits are going to get squashed away on merge anyway.
* (QENG-3919) Update API docs
We consider it a bug that the actual behavior was not this behavior, but the
documentation was also silent on this point.
* (QENG-3919) minor readability tweak in refactored method
* (QENG-3919) Clean up interim comments re: status codes
* (QENG-3919) Drop now-orphaned `checkout_vm` method
We kept this up-to-date while we were upgrading and refactoring, but, turns out,
this method is no longer called anywhere. 💀🔥
* (QENG-3919) Return 503 status on failed allocation
Making sure we go back to the original functionality, which was:
- status 200 when vms successfully allocated
- status 404 when a pool name is unknown
- status 404 when no pool name is specified
- status 503 when vm allocation failed
* (QENG-3919) add net-ldap to Gemfile
Maybe we shouldn't foil-ball gems onto servers.
* (QENG-3919) Turns out, spush isn't a redis command
And hence we see once again the weakness of mockist tests.
* (QENG-3919) Pin the net-ldap gem to 0.11 for the jrubies, etc.
* (QENG-3919) Correct an old spelling error in spec descriptions
* (QENG-3919) Further tweak net-ldap version
* (QENG-3919) return_single_vm -> return_vm_to_ready_state
cc @shermdog
Add an additional disk to a running VM via the vmpooler API.
````
$ curl -X POST -H X-AUTH-TOKEN:a9znth9dn01t416hrguu56ze37t790bl --url vmpooler.company.com/api/v1/vm/fq6qlpjlsskycq6/disk/8
````
````json
{
"ok": true,
"fq6qlpjlsskycq6": {
"disk": "+8mb"
}
}
````
Provisioning and attaching disks can take a moment, but once the task completes it will be reflected in a `GET /vm/<hostname>` query:
````
$ curl --url vmpooler.company.com/api/v1/vm/fq6qlpjlsskycq6
````
````json
{
"ok": true,
"fq6qlpjlsskycq6": {
"template": "debian-7-x86_64",
"lifetime": 2,
"running": 0.08,
"state": "running",
"disk": [
"+8mb"
],
"domain": "delivery.puppetlabs.net"
}
}
This commit adds the following functions:
- `add_disk`: the wrapper function to add a new disk to a VM
Usage is:
````
add_disk(vmname, disksize, datastore)
````
`vmname` is the name of the VM to add the disk to, `disksize` is the
disk size in MB, and `datastore` is the datastore on which to provision
the new disk.
`add_disk` required the addition of the following helper functions:
- `find_device`: locate a device object in vSphere
- `find_disk_controller`: find the disk controller used by a VM
- `find_disk_devices`: find the disk devices used by a VM
- `find_disk_unit_number`: find a free SCSI ID to assign to a new disk
- `find_vmdks`: find names of VMDK disks attached to a VM
The symbol notation for the YAML variables "server," "username," and, "password," were not working for me. When changing them to strings the script seems to work.
This PR adds spec testing for pool 'alias' functionality introduced in
17b24d6, as well as the following previously non-existant tests:
- new tests for handling requests for a VM from a nonexistant pool
- new tests for the `POST /vm/:template` endpoint
The following pool configuration would allow a pool to be aliased in POST
requests as 'centos-6-x86_64', 'centos-6-amd64', or 'centos-6-64':
````yaml
- name: 'centos-6-x86_64'
alias: [ 'centos-6-amd64', 'centos-6-64' ]
template: 'templates/centos-6-x86_64'
folder: 'vmpooler/centos-6-x86_64'
datastore: 'instance1'
size: 5
````
The 'alias' configuration can be either a string or an array.
Note that even when requesting an alias, the pool's 'name' is returned in
the JSON response:
````
$ curl -d '{"centos-6-64":"1"}' --url vmpooler/api/v1/vm
````
````json
{
"ok": true,
"centos-6-x86_64": {
"hostname": "cuna2qeahwlzji7"
},
"domain": "company.com"
}
````
Prior to this commit, a running VM could fail a ping check and be
destroyed. This causes issues when network hiccups occur or the machine
is performing a reboot.
A VM that is in a ready state will now be destroyed when handed back or
it hits the lifetime TTL.
An SSH check was added before moving a VM from pending to ready.
However, the result of that check did not matter and move_pending would
still be called. This moves the move_pending call to within the begin
block that holds the SSH check. If the check fails, then only
fail_pending will be called.
