(POOLER-52) Add a generic connection pool

Previously VMPooler had no concept of a connection pooler.  While there is an
up to date connection pooler Gem (connection_pool), that supports MRI and jRuby,
it lacked metrics which are useful to diagnose errors and judge pool size.
This commit:

- Brings in the connection_pool gem
- Creates a new class called generic_connection_pool which inherits from the
  ConnectionPool class in the connection_pool gem.
- Extends the connection pool object with a new function called `with_metrics`
  This copies the code from the original `with` method but emits metrics for
  how long it took to get an object from the pool, and then how many objects
  are left in the pool.  This is sent using VMPooler's metrics object.
  Extending the object was used instead of overriding as it was not possible to
  inject into the existing function and monkey patching did not seem the correct
  way.

  In order use the metics, the GenericConnectionPool object modifies the
  initialize method to use :metrics and :metrics_prefix options
- Also added tests for the GenericConnectionPool class to ensure the new
  functions are tested.  Note that the functionality that was not extended is
  not tested in VMPooler.
This commit is contained in:
Glenn Sarti 2017-04-05 10:54:58 -07:00
parent 0aa550f852
commit 888ffc4afc
4 changed files with 146 additions and 1 deletions

View file

@ -12,7 +12,7 @@ module Vmpooler
require 'yaml'
require 'set'
%w[api graphite logger pool_manager statsd dummy_statsd providers].each do |lib|
%w[api graphite logger pool_manager statsd dummy_statsd generic_connection_pool providers].each do |lib|
begin
require "vmpooler/#{lib}"
rescue LoadError

View file

@ -0,0 +1,53 @@
require 'connection_pool'
module Vmpooler
class PoolManager
class GenericConnectionPool < ConnectionPool
# Extend the ConnectionPool class with instrumentation
# https://github.com/mperham/connection_pool/blob/master/lib/connection_pool.rb
def initialize(options = {}, &block)
super(options, &block)
@metrics = options[:metrics]
@metric_prefix = options[:metric_prefix]
@metric_prefix = 'connectionpool' if @metric_prefix.nil? || @metric_prefix == ''
end
if Thread.respond_to?(:handle_interrupt)
# MRI
def with_metrics(options = {})
Thread.handle_interrupt(Exception => :never) do
start = Time.now
conn = checkout(options)
timespan_ms = ((Time.now - start) * 1000).to_i
@metrics.gauge(@metric_prefix + '.available', @available.length) unless @metrics.nil?
@metrics.timing(@metric_prefix + '.waited', timespan_ms) unless @metrics.nil?
begin
Thread.handle_interrupt(Exception => :immediate) do
yield conn
end
ensure
checkin
@metrics.gauge(@metric_prefix + '.available', @available.length) unless @metrics.nil?
end
end
end
else
# jruby 1.7.x
def with_metrics(options = {})
start = Time.now
conn = checkout(options)
timespan_ms = ((Time.now - start) * 1000).to_i
@metrics.gauge(@metric_prefix + '.available', @available.length) unless @metrics.nil?
@metrics.timing(@metric_prefix + '.waited', timespan_ms) unless @metrics.nil?
begin
yield conn
ensure
checkin
@metrics.gauge(@metric_prefix + '.available', @available.length) unless @metrics.nil?
end
end
end
end
end
end