888ffc4afc
Previously VMPooler had no concept of a connection pooler. While there is an up to date connection pooler Gem (connection_pool), that supports MRI and jRuby, it lacked metrics which are useful to diagnose errors and judge pool size. This commit: - Brings in the connection_pool gem - Creates a new class called generic_connection_pool which inherits from the ConnectionPool class in the connection_pool gem. - Extends the connection pool object with a new function called `with_metrics` This copies the code from the original `with` method but emits metrics for how long it took to get an object from the pool, and then how many objects are left in the pool. This is sent using VMPooler's metrics object. Extending the object was used instead of overriding as it was not possible to inject into the existing function and monkey patching did not seem the correct way. In order use the metics, the GenericConnectionPool object modifies the initialize method to use :metrics and :metrics_prefix options - Also added tests for the GenericConnectionPool class to ensure the new functions are tested. Note that the functionality that was not extended is not tested in VMPooler. |
||
|---|---|---|
| lib | ||
| scripts | ||
| spec | ||
| .gitignore | ||
| .rubocop.yml | ||
| .rubocop_todo.yml | ||
| .travis.yml | ||
| API.md | ||
| Dockerfile | ||
| Gemfile | ||
| LICENSE | ||
| Rakefile | ||
| README.md | ||
| vmpooler | ||
| vmpooler.yaml.example | ||
vmpooler
vmpooler provides configurable 'pools' of instantly-available (running) virtual machines.
Usage
At Puppet, Inc. we run acceptance tests on thousands of disposable VMs every day. Dynamic cloning of VM templates initially worked fine for this, but added several seconds to each test run and was unable to account for failed clone tasks. By pushing these operations to a backend service, we were able to both speed up tests and eliminate test failures due to underlying infrastructure failures.
Installation
Prerequisites
vmpooler requires the following Ruby gems be installed:
It also requires that a Redis server exists somewhere, as this is the datastore used for vmpooler's inventory and queueing services.
Configuration
The following YAML configuration sets up two pools, debian-7-i386 and debian-7-x86_64, which contain 5 running VMs each:
---
:providers:
:vsphere:
server: 'vsphere.company.com'
username: 'vmpooler'
password: 'swimsw1msw!m'
:redis:
server: 'redis.company.com'
:config:
logfile: '/var/log/vmpooler.log'
:pools:
- name: 'debian-7-i386'
template: 'Templates/debian-7-i386'
folder: 'Pooled VMs/debian-7-i386'
pool: 'Pooled VMs/debian-7-i386'
datastore: 'vmstorage'
size: 5
provider: vsphere
- name: 'debian-7-x86_64'
template: 'Templates/debian-7-x86_64'
folder: 'Pooled VMs/debian-7-x86_64'
pool: 'Pooled VMs/debian-7-x86_64'
datastore: 'vmstorage'
size: 5
provider: vsphere
See the provided YAML configuration example, vmpooler.yaml.example, for additional configuration options and parameters.
Running via Docker
A Dockerfile is included in this repository to allow running vmpooler inside a Docker container. A vmpooler.yaml configuration file can be embedded in the current working directory, or specified inline in a VMPOOLER_CONFIG environment variable. To build and run:
docker build -t vmpooler . && docker run -e VMPOOLER_CONFIG -p 80:4567 -it vmpooler
API and Dashboard
vmpooler provides an API and web front-end (dashboard) on port :4567. See the provided YAML configuration example, vmpooler.yaml.example, to specify an alternative port to listen on.
API
vmpooler provides a REST API for VM management. See the API documentation for more information.
Dashboard
A dashboard is provided to offer real-time statistics and historical graphs. It looks like this:
Graphite is required for historical data retrieval. See the provided YAML configuration example, vmpooler.yaml.example, for details.
Command-line Utility
- The vmpooler_client.py CLI utility provides easy access to the vmpooler service. The tool is cross-platform and written in Python.
- vmfloaty is a ruby based CLI tool and scripting library written in ruby.
Vagrant plugin
- vagrant-vmpooler Use Vagrant to create and manage your vmpooler instances.
Build status
License
vmpooler is distributed under the Apache License, Version 2.0. See the LICENSE file for more details.