Provide configurable 'pools' of instantly-available (running) virtual machines
Find a file
Nick Lewis c4f3a49782 Generate a wider set of legal names
Previously, we restricted the adjective and noun portion of the name
each to 7 characters to ensure that the final name would not be more
than 15 after adding a hyphen. Given that the _total_ length is what
matters, we can generate a noun up to 11 characters (to ensure we leave
room for a hyphen and a 3 letter adjective) and adjust our acceptable
adjective size accordingly. This lets many more names be generated than
would otherwise, while still respecting the 15 character limit.

Due to the limited set of 11 letter nouns and corresponding 3 letter
adjectives, as well as some complex combinatorics, setting the noun
length to 11 causes a net increase in conflicts. We therefore actually
set it to 10, which causes a net decrease in conflicts.

We favor generating longer nouns rather than longer adjectives (by
selecting the noun first) because longer adjectives tend to be more
unwieldy words, and thus more awkward to say and generally less fun.
2020-01-27 16:11:13 -08:00
bin Simplify declaration of checkoutlock mutex 2019-10-23 03:52:08 -07:00
docker (POOLER-123) Implement a max TTL (#349) 2019-12-05 09:35:30 -07:00
docs (POOLER-142) Add running host to vm API data 2019-08-21 12:43:06 -07:00
examples (POOLER-133) Identify when a ready VM has failed 2018-12-03 12:21:08 -08:00
lib Generate a wider set of legal names 2020-01-27 16:11:13 -08:00
scripts Update to generic launcher 2018-06-08 12:10:44 -07:00
spec Merge pull request #348 from Secure-24/issue_205 2019-12-12 11:15:26 -08:00
.gitignore Adds a gemspec file 2018-06-12 12:01:26 -07:00
.rubocop.yml Update rubocop configs 2019-11-04 14:57:53 -08:00
.rubocop_todo.yml Update rubocop configs 2019-11-04 14:57:53 -08:00
.travis.yml (POOLER-138) Support multiple pools per alias 2019-02-01 09:05:23 -08:00
CHANGELOG.md Update CHANGELOG for 0.8.2 2019-11-11 15:28:13 -08:00
CODEOWNERS Add CODEOWNERS file to vmpooler 2019-08-26 14:55:42 -07:00
CONTRIBUTING.md (doc) Add changelog and contributing guidlines 2018-07-16 16:46:03 -07:00
Gemfile Add spicy-proton to vmpooler.gemspec 2019-10-25 09:00:01 -07:00
LICENSE (maint) update README.md and LICENSE to reflect rebranding 2016-07-07 23:08:38 +00:00
PROVIDER_API.md Adds a new mechanism to load providers from any gem or file path. (#263) 2018-07-24 16:35:18 -07:00
Rakefile (maint) Only load rubocop rake tasks if gem is available 2017-02-22 15:23:45 -08:00
README.md Fix Dockerfile link in readme and add note about http requests for dev (#316) 2019-03-04 10:05:42 -08:00
Vagrantfile Updated Vagrant box and associated docs 2017-10-25 14:39:15 -07:00
vmpooler.gemspec Add spicy-proton to vmpooler.gemspec 2019-10-25 09:00:01 -07:00
vmpooler.yaml.dummy-example Fix Dockerfile link in readme and add note about http requests for dev (#316) 2019-03-04 10:05:42 -08:00
vmpooler.yaml.example (POOLER-123) Implement a max TTL (#349) 2019-12-05 09:35:30 -07:00

vmpooler

vmpooler

vmpooler provides configurable 'pools' of instantly-available (running) virtual machines.

Usage

At Puppet, Inc. we run acceptance tests on thousands of disposable VMs every day. Dynamic cloning of VM templates initially worked fine for this, but added several seconds to each test run and was unable to account for failed clone tasks. By pushing these operations to a backend service, we were able to both speed up tests and eliminate test failures due to underlying infrastructure failures.

Installation

Prerequisites

vmpooler is available as a gem

To use the gem gem install vmpooler

Dependencies

Vmpooler requires a Redis server. This is the datastore used for vmpooler's inventory and queueing services.

Configuration

Configuration for vmpooler may be provided via environment variables, or a configuration file.

Please see this configuration document for more details about configuring vmpooler via environment variables.

The following YAML configuration sets up two pools, debian-7-i386 and debian-7-x86_64, which contain 5 running VMs each:

---
:providers:
  :vsphere:
    server: 'vsphere.example.com'
    username: 'vmpooler'
    password: 'swimsw1msw!m'

:redis:
  server: 'redis.example.com'

:config:
  logfile: '/var/log/vmpooler.log'

:pools:
  - name: 'debian-7-i386'
    template: 'Templates/debian-7-i386'
    folder: 'Pooled VMs/debian-7-i386'
    pool: 'Pooled VMs/debian-7-i386'
    datastore: 'vmstorage'
    size: 5
    provider: vsphere
  - name: 'debian-7-x86_64'
    template: 'Templates/debian-7-x86_64'
    folder: 'Pooled VMs/debian-7-x86_64'
    pool: 'Pooled VMs/debian-7-x86_64'
    datastore: 'vmstorage'
    size: 5
    provider: vsphere

See the provided YAML configuration example, vmpooler.yaml.example, for additional configuration options and parameters or for supporting multiple providers.

Running via Docker

A Dockerfile is included in this repository to allow running vmpooler inside a Docker container. A configuration file can be used via volume mapping, and specifying the destination as the configuration file via environment variables, or the application can be configured with environment variables alone. The Dockerfile provides an entrypoint so you may choose whether to run API, or manager services. The default behavior will run both. To build and run:

docker build -t vmpooler . && docker run -e VMPOOLER_CONFIG -p 80:4567 -it vmpooler

To run only the API and dashboard

docker run -p 80:4567 -it vmpooler api

To run only the manager component

docker run -it vmpooler manager

docker-compose

A docker-compose file is provided to support running vmpooler easily via docker-compose.

docker-compose -f docker/docker-compose.yml up

Running Docker inside Vagrant

A vagrantfile is included in this repository. Please see vagrant instructions for details.

API and Dashboard

vmpooler provides an API and web front-end (dashboard) on port :4567. See the provided YAML configuration example, vmpooler.yaml.example, to specify an alternative port to listen on.

API

vmpooler provides a REST API for VM management. See the API documentation for more information.

Dashboard

A dashboard is provided to offer real-time statistics and historical graphs. It looks like this:

dashboard

Graphite is required for historical data retrieval. See the provided YAML configuration example, vmpooler.yaml.example, for details.

Command-line Utility

  • The vmpooler_client.py CLI utility provides easy access to the vmpooler service. The tool is cross-platform and written in Python.
  • vmfloaty is a ruby based CLI tool and scripting library written in ruby.

Vagrant plugin

Development and further documentation

For more information about setting up a development instance of vmpooler or other subjects, see the docs/ directory.

Build status

Build Status

License

vmpooler is distributed under the Apache License, Version 2.0. See the LICENSE file for more details.