Introducing the Prometheus Stats code into ABS showed that the Clarity
could be improved a bit with better variable naming, some refactoring
to reduce repitition and documenting the Metrics table itself.
Filtering these changes back to the vmpooler code base.
This commit updates folder purging references to ensure that provider
name references are referring to the named provider, rather than the
provider type. Without this change folder purging fails because it
cannot identify target folders.
This commit updates the method used for chceking the status of an
ondemand request to ensure that if multiple aliases are used to fulfill
a request that they are correctly presented as a single pool again when
everything is ready. Without this change it is possible for only one
group of an aliased pool to show up in pending or completed requests.
Ensure that the correct stats are registered for the Manager and the api
respectively. E.g. all checkout counters are for the api only, whereas
clone times belong to the manager.
Also new ondemand functionality stats weren't registered, so add these
along with missing delete stats.
Review changes suggested to revise the Metrics related files into a more
logical class structure.
Also fixup grammar typos in docs strings and any trailing metrics that
have been recently added to vmpooler.
Use the example provided in the Ruby Client to provide a customised
collector appropriate to log all calls to the API. The customised
filtering is used to replace individual node names and templates
for the /vm and request ID's for the /ondemand endpoints.
This module was failing our rubocop checks so have updated it since
it now forms part of vmpooler.
Separate trapping for litmus jobs is also included so that they don't
interfere with stats from the jenkins pipelines.
Break down the usage stats into smaller groups so as to manage the
number of stat lines collected for Prometheus.
This may need some further revision to filter out Litmus stats, or
otherwise collect litmus usage information.
The redis pooler connection metric used "metric_prefix" which is
misleading, so split this into connpool_type and connpool_provider.
Also remove some earlier jruby compatibility code to reduce
rebase conflicts when this is rebased on top of Matt's changes.
Need the logger code in promstats.rb so move logger initialisation to
the top of the vmpooler script, remove the class method from vmpooler.rb
and make appropriate downstream changes to metrics and pooler manger
handling.
This also means we can start decent logging in the API if we wish do.
Fixed up to rebase on top of Matts POOLER-158 changes
This is a re-architect of the vmpooler initialisation code to:
1. Allow an API service for both manager and the api
2. Add the Prometheus endpoints to the web service.
Needed to change the way the Rack Service is started as instantiating
using ".New" leads to a failure to initialise the http Stats
collection.
3. Selectively load the pooler api and/or Prometheus endpoints.
4. Rework API Spec tests for revised API loading. Needed to tidy up the
initialisation and perform a reset! after each test to avoid "leaks"
and dependencies between the tests.
Add a new Prometheus class as an additional stats feed along with the
existing feeds.
Move the metrics initialisation code into its own class and sub-class
the individual metrics implementations under this.
Screen out redundant begin block rubocop compliance checks as these
are causing vsphere.rb and pool_manager.rb to fail with differing
ruby version checks.
* (POOLER-174) Reduce duplicate of on demand code introduced in POOLER-158
refactored every parsing of request of type 'pool_alias:pool:count' into a
utility class, that is used by pool_manager and the api v1 class
* add some metrics to the od request generation
* fix rubocop offenses, we are now friends
This commit updates vmpooler graphite stats behavior when a connection cannot be opened to the graphite server to warn instead of allowing the full backtrace to be output to stderr. Without this change a backtrace is output to stderr when a graphite connection fails.
It seems like generate_and_check_hostname does not need a method argument
this was fixed in one PR, and another change made in another PR and was used
inconsistently
This change adds a capability to vmpooler to provision instances on
demand. Without this change vmpooler only supports retrieving machines
from pre-provisioned pools.
Additionally, this change refactors redis interactions to reduce round
trips to redis. Specifically, multi and pipelined redis commands are
added where possible to reduce the number of times we are calling redis.
To support the redis refactor the redis interaction has changed to
leveraging a connection pool. In addition to offering multiple
connections for pool manager to use, the redis interactions in pool
manager are now thread safe.
Ready TTL is now a global parameter that can be set as a default for all
pools. A default of 0 has been removed, because this is an unreasonable
default behavior, which would leave a provisioned instance in the pool
indefinitely.
Pool empty messages have been removed when the pool size is set to 0.
Without this change, when a pool was set to a size of 0 the API and pool
manager would both show that a pool is empty.
