This adds an "operation" label to the user metrics and moves incrementing from the manager to api, so that the user metrics show when resources are allocated, as well as destroyed. Previously, user metrics were only updated upon destroying a resource.
I think its better suited to increment the metric as part of the api instead of the pool_manger, because it's expected to do so when a user successfully checks out or deletes a VM, but can be problematic when doing so in the provider since it can clone VMs before actually being checked out by a user.
It appears we renamed `/ondemand/` to `/ondemandvm/` at some point and,
as a result, have not been stripping hostnames from that endpoint's
metrics. This has caused issues with metrics collection due a very high
cardinality.
This change removes usage of redis multi from API. Without this change the redis usage can cause issues because no connection pool is used, so another worker may try to use the same backend object causing a failure.
Checkout metric counters were against the template name and not the
actual pool used which prevents us from counting the checkouts in the
pixa4 pools for example.
Note - this is a re-rerun - the last commit on this file over-wrote
the change.
This change fixes an error in validate_token where hset does not have the correct number of parameters. Without this change validate_token causes a error.
This change sets a VM as running in redis as soon as it is checked out. Without this change when allocated several instances it is possible for a machine that has been allocated for a checkout, but not yet marked as active, to be identified as running when it should not be, which was added in POOLER-191. Without this change a machine may be destroyed during checkout by pool_manager if there are several instances being allocated.
Additionally, redis multi is added for vm checkout operations to minimize the round trips to redis during a checkout operation. Without this addition each VM checkout causes several redis interactions.
Checkout metric counters were against the template name and not the
actual pool used which prevents us from counting the checkouts in the
pixa4 pools for example.
* (maint) Speedup the tagging method
While looking at the instrumentation data for the ABS queue processor,
I noticed a lot of time spent in the HTTP PUT method, which in the code
was easy to isolate, as it is only used via the vmpooler tagging functions
ie the API /vm/foobar/ with 'tag' key-value pairs.
While I'm not sure the original hset() make sense to me, there was an easy
way to speed them up by using pipelined. I would expect a very good speed
increase with this turned on.
* tag rubocop to <1.0 because the 1.0 version returns 130 new offenses
This change adds detection of running instances that are in a running
queue, but have no data in a active queue for the same pool. When this
happens a machine will live forever, impacting the running count, and
preventing the machine from being killed. Without this change running
instances that are not marked as active will live forever.
