mirror of
https://github.com/puppetlabs/vmpooler.git
synced 2026-01-26 10:08:40 -05:00
9bb4df7d8e
This commit adds a configuration endpoint to the vmpooler API. Pool size, and pool template, can be adjusted for pools that are configured at vmpooler application start time. Pool template changes trigger a pool refresh, and the new template has delta disks created automatically by vmpooler. Additionally, the capability to create template delta disks is added to the vsphere provider, and this is implemented to ensure that templates have delta disks created at application start time. The mechanism used to find template VM objects is simplified to make the flow of logic easier to understand. As an additional benefit, performance of this lookup is improved by using FindByInventoryPath. A table of contents is added to API.md to ease navigation. Without this change API.md has no table of contents and is difficult to navigate. Add mutex object for managing pool configuration updates This commit adds a mutex object for ensuring that pool configuration changes are synchronized across multiple running threads, removing the possibility of two threads attempting to update something at once, without relying on redis data. Without this change this is managed crudely by specifying in redis that a configuration update is taking place. This redis data is left so the REPOPULATE section of _check_pool can still identify when a configuration change is in progress, and prevent a pool from repopulating at that time. Add wake up event for pool template changes This commit adds a wake up event to detect pool template changes. Additionally, GET /config has a template_ready section added to the output for each pool, which makes clear when a pool is ready to populate itself.
645 lines
15 KiB
Markdown
645 lines
15 KiB
Markdown
# Table of contents
|
|
1. [API](#API)
|
|
2. [Token operations](#token)
|
|
3. [VM operations](#vmops)
|
|
4. [Add disks](#adddisks)
|
|
5. [VM snapshots](#vmsnapshots)
|
|
6. [Status and metrics](#statusmetrics)
|
|
7. [Pool configuration](#poolconfig)
|
|
|
|
### API <a name="API"></a>
|
|
|
|
vmpooler provides a REST API for VM management. The following examples use `curl` for communication.
|
|
|
|
#### Token operations <a name="token"></a>
|
|
|
|
Token-based authentication can be used when requesting or modifying VMs. The `/token` route can be used to create, query, or delete tokens. See the provided YAML configuration example, [vmpooler.yaml.example](vmpooler.yaml.example), for information on configuring an authentication store to use when performing token operations.
|
|
|
|
##### GET /token
|
|
|
|
Get a list of issued tokens.
|
|
|
|
```
|
|
$ curl -u jdoe --url vmpooler.company.com/api/v1/token
|
|
Enter host password for user 'jdoe':
|
|
```
|
|
```json
|
|
{
|
|
"ok": true,
|
|
"utpg2i2xswor6h8ttjhu3d47z53yy47y": {
|
|
"created": "2015-04-28 19:17:47 -0700"
|
|
}
|
|
}
|
|
```
|
|
|
|
##### POST /token
|
|
|
|
Generate a new authentication token.
|
|
|
|
```
|
|
$ curl -X POST -u jdoe --url vmpooler.company.com/api/v1/token
|
|
Enter host password for user 'jdoe':
|
|
```
|
|
```json
|
|
{
|
|
"ok": true,
|
|
"token": "utpg2i2xswor6h8ttjhu3d47z53yy47y"
|
|
}
|
|
```
|
|
|
|
##### GET /token/<token>
|
|
|
|
Get information about an existing token (including associated VMs).
|
|
|
|
```
|
|
$ curl --url vmpooler.company.com/api/v1/token/utpg2i2xswor6h8ttjhu3d47z53yy47y
|
|
```
|
|
```json
|
|
{
|
|
"ok": true,
|
|
"utpg2i2xswor6h8ttjhu3d47z53yy47y": {
|
|
"user": "jdoe",
|
|
"created": "2015-04-28 19:17:47 -0700",
|
|
"last": "2015-11-04 12:28:37 -0700",
|
|
"vms": {
|
|
"running": [
|
|
"dqs4914g2wjyy5w",
|
|
"hul7ib0ssr0f4o0"
|
|
]
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
##### DELETE /token/<token>
|
|
|
|
Delete an authentication token.
|
|
|
|
```
|
|
$ curl -X DELETE -u jdoe --url vmpooler.company.com/api/v1/token/utpg2i2xswor6h8ttjhu3d47z53yy47y
|
|
Enter host password for user 'jdoe':
|
|
```
|
|
```json
|
|
{
|
|
"ok": true
|
|
}
|
|
```
|
|
|
|
#### VM operations <a name="vmops"></a>
|
|
|
|
##### GET /vm
|
|
|
|
Retrieve a list of available VM pools.
|
|
|
|
```
|
|
$ curl --url vmpooler.company.com/api/v1/vm
|
|
```
|
|
```json
|
|
[
|
|
"debian-7-i386",
|
|
"debian-7-x86_64"
|
|
]
|
|
```
|
|
|
|
##### POST /vm
|
|
|
|
Useful for batch operations; post JSON (see format below), get back VMs.
|
|
|
|
If an authentication store is configured, an authentication token supplied via the `X-AUTH-TOKEN` HTTP header will modify a VM's default lifetime. See the provided YAML configuration example, [vmpooler.yaml.example](vmpooler.yaml.example), and the 'token operations' section above for more information.
|
|
|
|
```
|
|
$ curl -d '{"debian-7-i386":"2","debian-7-x86_64":"1"}' --url vmpooler.company.com/api/v1/vm
|
|
```
|
|
```json
|
|
{
|
|
"ok": true,
|
|
"debian-7-i386": {
|
|
"hostname": [
|
|
"o41xtodlvnvu5cw",
|
|
"khirruvwfjlmx3y"
|
|
]
|
|
},
|
|
"debian-7-x86_64": {
|
|
"hostname": "y91qbrpbfj6d13q"
|
|
},
|
|
"domain": "company.com"
|
|
}
|
|
```
|
|
|
|
**NOTE: Returns either all requested VMs or no VMs.**
|
|
|
|
##### POST /vm/<pool>
|
|
|
|
Check-out a VM or VMs.
|
|
|
|
```
|
|
$ curl -d --url vmpooler.company.com/api/v1/vm/debian-7-i386
|
|
```
|
|
```json
|
|
{
|
|
"ok": true,
|
|
"debian-7-i386": {
|
|
"hostname": "fq6qlpjlsskycq6"
|
|
},
|
|
"domain": "company.com"
|
|
}
|
|
```
|
|
|
|
Multiple VMs can be requested by using multiple query parameters in the URL:
|
|
|
|
```
|
|
$ curl -d --url vmpooler.company.com/api/v1/vm/debian-7-i386+debian-7-i386+debian-7-x86_64
|
|
```
|
|
|
|
```json
|
|
{
|
|
"ok": true,
|
|
"debian-7-i386": {
|
|
"hostname": [
|
|
"sc0o4xqtodlul5w",
|
|
"4m4dkhqiufnjmxy"
|
|
]
|
|
},
|
|
"debian-7-x86_64": {
|
|
"hostname": "zb91y9qbrbf6d3q"
|
|
},
|
|
"domain": "company.com"
|
|
}
|
|
```
|
|
|
|
**NOTE: Returns either all requested VMs or no VMs.**
|
|
|
|
##### GET /vm/<hostname>
|
|
|
|
Query a checked-out VM.
|
|
|
|
```
|
|
$ curl --url vmpooler.company.com/api/v1/vm/pxpmtoonx7fiqg6
|
|
```
|
|
```json
|
|
{
|
|
"ok": true,
|
|
"pxpmtoonx7fiqg6": {
|
|
"template": "centos-6-x86_64",
|
|
"lifetime": 12,
|
|
"running": 3.1,
|
|
"state": "running",
|
|
"tags": {
|
|
"department": "engineering",
|
|
"user": "jdoe"
|
|
},
|
|
"domain": "company.com"
|
|
}
|
|
}
|
|
```
|
|
|
|
##### PUT /vm/<hostname>
|
|
|
|
Modify a checked-out VM.
|
|
|
|
The following are valid PUT parameters and their required data structures:
|
|
|
|
parameter | description | required structure
|
|
--------- | ----------- | ------------------
|
|
*lifetime* | VM TTL (in hours) | integer
|
|
*tags* | free-form VM tagging | hash
|
|
|
|
Any modifications can be verified using the [GET /vm/<hostname>](#get-vmhostname) endpoint.
|
|
|
|
If an authentication store is configured, an authentication token is required (via the `X-AUTH-TOKEN` HTTP header) to access this route. See the provided YAML configuration example, [vmpooler.yaml.example](vmpooler.yaml.example), and the 'token operations' section above for more information.
|
|
|
|
```
|
|
$ curl -X PUT -d '{"lifetime":"2"}' --url vmpooler.company.com/api/v1/vm/fq6qlpjlsskycq6
|
|
```
|
|
```json
|
|
{
|
|
"ok": true
|
|
}
|
|
```
|
|
|
|
```
|
|
$ curl -X PUT -d '{"tags":{"department":"engineering","user":"jdoe"}}' --url vmpooler.company.com/api/v1/vm/fq6qlpjlsskycq6
|
|
```
|
|
```json
|
|
{
|
|
"ok": true
|
|
}
|
|
```
|
|
|
|
##### DELETE /vm/<hostname>
|
|
|
|
Schedule a checked-out VM for deletion.
|
|
|
|
```
|
|
$ curl -X DELETE --url vmpooler.company.com/api/v1/vm/fq6qlpjlsskycq6
|
|
```
|
|
```json
|
|
{
|
|
"ok": true
|
|
}
|
|
```
|
|
|
|
#### Adding additional disk(s) <a name="adddisks"></a>
|
|
|
|
##### POST /vm/<hostname>/disk/<size>
|
|
|
|
Add an additional disk to a running VM.
|
|
|
|
````
|
|
$ curl -X POST -H X-AUTH-TOKEN:a9znth9dn01t416hrguu56ze37t790bl --url vmpooler.company.com/api/v1/vm/fq6qlpjlsskycq6/disk/8
|
|
````
|
|
````json
|
|
{
|
|
"ok": true,
|
|
"fq6qlpjlsskycq6": {
|
|
"disk": "+8gb"
|
|
}
|
|
}
|
|
````
|
|
|
|
Provisioning and attaching disks can take a moment, but once the task completes it will be reflected in a `GET /vm/<hostname>` query:
|
|
|
|
````
|
|
$ curl --url vmpooler.company.com/api/v1/vm/fq6qlpjlsskycq6
|
|
````
|
|
````json
|
|
{
|
|
"ok": true,
|
|
"fq6qlpjlsskycq6": {
|
|
"template": "debian-7-x86_64",
|
|
"lifetime": 2,
|
|
"running": 0.08,
|
|
"state": "running",
|
|
"disk": [
|
|
"+8gb"
|
|
],
|
|
"domain": "delivery.puppetlabs.net"
|
|
}
|
|
}
|
|
|
|
````
|
|
|
|
#### VM snapshots <a name="vmsnapshots"></a>
|
|
|
|
##### POST /vm/<hostname>/snapshot
|
|
|
|
Create a snapshot of a running VM.
|
|
|
|
````
|
|
$ curl -X POST -H X-AUTH-TOKEN:a9znth9dn01t416hrguu56ze37t790bl --url vmpooler.company.com/api/v1/vm/fq6qlpjlsskycq6/snapshot
|
|
````
|
|
````json
|
|
{
|
|
"ok": true,
|
|
"fq6qlpjlsskycq6": {
|
|
"snapshot": "n4eb4kdtp7rwv4x158366vd9jhac8btq"
|
|
}
|
|
}
|
|
````
|
|
|
|
Snapshotting a live VM can take a moment, but once the snapshot task completes it will be reflected in a `GET /vm/<hostname>` query:
|
|
|
|
````
|
|
$ curl --url vmpooler.company.com/api/v1/vm/fq6qlpjlsskycq6
|
|
````
|
|
````json
|
|
{
|
|
"ok": true,
|
|
"fq6qlpjlsskycq6": {
|
|
"template": "debian-7-x86_64",
|
|
"lifetime": 2,
|
|
"running": 0.08,
|
|
"state": "running",
|
|
"snapshots": [
|
|
"n4eb4kdtp7rwv4x158366vd9jhac8btq"
|
|
],
|
|
"domain": "delivery.puppetlabs.net"
|
|
}
|
|
}
|
|
````
|
|
|
|
##### POST /vm/<hostname>/snapshot/<snapshot>
|
|
|
|
Revert a VM back to a snapshot.
|
|
|
|
````
|
|
$ curl X POST -H X-AUTH-TOKEN:a9znth9dn01t416hrguu56ze37t790bl --url vmpooler.company.com/api/v1/vm/fq6qlpjlsskycq6/snapshot/n4eb4kdtp7rwv4x158366vd9jhac8btq
|
|
````
|
|
````json
|
|
{
|
|
"ok": true
|
|
}
|
|
````
|
|
|
|
#### Status and metrics <a name="statusmetrics"></a>
|
|
|
|
##### GET /status
|
|
|
|
A "live" status endpoint, representing the current state of the service.
|
|
|
|
```
|
|
$ curl --url vmpooler.company.com/api/v1/status
|
|
```
|
|
```json
|
|
{
|
|
"capacity": {
|
|
"current": 716,
|
|
"total": 717,
|
|
"percent": 99.9
|
|
},
|
|
"clone": {
|
|
"duration": {
|
|
"average": 8.8,
|
|
"min": 2.79,
|
|
"max": 69.76
|
|
},
|
|
"count": {
|
|
"total": 1779
|
|
}
|
|
},
|
|
"queue": {
|
|
"pending": 1,
|
|
"cloning": 0,
|
|
"booting": 1,
|
|
"ready": 716,
|
|
"running": 142,
|
|
"completed": 0,
|
|
"total": 859
|
|
},
|
|
"status": {
|
|
"ok": true,
|
|
"message": "Battle station fully armed and operational."
|
|
}
|
|
}
|
|
```
|
|
|
|
If there are empty pools, the "status" section will convey this:
|
|
|
|
```json
|
|
"status": {
|
|
"ok": false,
|
|
"message": "Found 2 empty pools.",
|
|
"empty": [
|
|
"centos-6-x86_64",
|
|
"debian-7-x86_64"
|
|
]
|
|
}
|
|
```
|
|
|
|
##### GET /summary[?from=YYYY-MM-DD[&to=YYYY-MM-DD]]
|
|
|
|
Returns a summary, or report, for the timespan between `from` and `to` (inclusive)
|
|
parameters. The response includes both an overall and daily view of tracked
|
|
metrics, such as boot and cloning durations.
|
|
|
|
Any omitted query parameter will default to now/today. A request without any
|
|
parameters will result in the current day's summary.
|
|
|
|
```
|
|
$ curl --url vmpooler.company.com/api/v1/summary
|
|
```
|
|
```json
|
|
{
|
|
"boot": {
|
|
"duration": {
|
|
"average": 106.6,
|
|
"min": 83.09,
|
|
"max": 121.06,
|
|
"total": 639.36,
|
|
},
|
|
"count": {
|
|
"average": 6,
|
|
"min": 6,
|
|
"max": 6,
|
|
"total": 6,
|
|
}
|
|
},
|
|
"clone": {
|
|
"duration": {
|
|
"average": 4.6,
|
|
"min": 2.78,
|
|
"max": 8.1,
|
|
"total": 63.94,
|
|
},
|
|
"count": {
|
|
"average": 14,
|
|
"min": 14,
|
|
"max": 14,
|
|
"total": 14,
|
|
}
|
|
},
|
|
"tag": {
|
|
"department": {
|
|
"engineering": 14,
|
|
"help desk": 10,
|
|
"IT": 44,
|
|
"total": 68
|
|
},
|
|
"user": {
|
|
"arodgers": 54,
|
|
"cmatthews": 10,
|
|
"jnelson": 4,
|
|
"total": 68
|
|
}
|
|
},
|
|
"daily": [
|
|
{
|
|
"date": "2015-03-11",
|
|
"boot": {
|
|
"duration": {
|
|
"average": 106.6,
|
|
"min": 83.09,
|
|
"max": 121.06,
|
|
"total": 639.36
|
|
},
|
|
"count": {
|
|
"total": 6
|
|
}
|
|
},
|
|
"clone": {
|
|
"duration": {
|
|
"average": 4.6,
|
|
"min": 2.78,
|
|
"max": 8.1,
|
|
"total": 63.94
|
|
},
|
|
"count": {
|
|
"total": 14
|
|
}
|
|
},
|
|
"tag": {
|
|
"department": {
|
|
"engineering": 14,
|
|
"help desk": 10,
|
|
"IT": 44,
|
|
"total": 68
|
|
},
|
|
"user": {
|
|
"arodgers": 54,
|
|
"cmatthews": 10,
|
|
"jnelson": 4,
|
|
"total": 68
|
|
}
|
|
}
|
|
}
|
|
]
|
|
}
|
|
```
|
|
|
|
```
|
|
$ curl -G -d 'from=2015-03-10' -d 'to=2015-03-11' --url vmpooler.company.com/api/v1/summary
|
|
```
|
|
```json
|
|
{
|
|
"boot": {...},
|
|
"clone": {...},
|
|
"daily": [
|
|
{
|
|
"date": "2015-03-10",
|
|
"boot": {
|
|
"duration": {
|
|
"average": 0,
|
|
"min": 0,
|
|
"max": 0,
|
|
"total": 0
|
|
},
|
|
"count": {
|
|
"total": 0
|
|
}
|
|
},
|
|
"clone": {
|
|
"duration": {
|
|
"average": 0,
|
|
"min": 0,
|
|
"max": 0,
|
|
"total": 0
|
|
},
|
|
"count": {
|
|
"total": 0
|
|
}
|
|
},
|
|
"tag": { }
|
|
},
|
|
{
|
|
"date": "2015-03-11",
|
|
"boot": {
|
|
"duration": {
|
|
"average": 106.6,
|
|
"min": 83.09,
|
|
"max": 121.06,
|
|
"total": 639.36
|
|
},
|
|
"count": {
|
|
"total": 6
|
|
}
|
|
},
|
|
"clone": {
|
|
"duration": {
|
|
"average": 4.6,
|
|
"min": 2.78,
|
|
"max": 8.1,
|
|
"total": 63.94
|
|
},
|
|
"count": {
|
|
"total": 14
|
|
}
|
|
},
|
|
"tag": { }
|
|
}
|
|
]
|
|
}
|
|
```
|
|
|
|
#### Managing pool configuration via API <a name="poolconfig"></a>
|
|
|
|
##### GET /config
|
|
|
|
Returns the running pool configuration
|
|
|
|
Responses:
|
|
* 200 - OK
|
|
* 404 - No configuration found
|
|
```
|
|
$ curl https://vmpooler.company.com/api/v1/config
|
|
```
|
|
```json
|
|
{
|
|
"pool_configuration": [
|
|
{
|
|
"name": "redhat-7-x86_64",
|
|
"template": "templates/redhat-7.2-x86_64-0.0.3",
|
|
"folder": "vmpooler/redhat-7-x86_64",
|
|
"datastore": "stor1",
|
|
"size": 1,
|
|
"datacenter": "dc1",
|
|
"provider": "vsphere",
|
|
"capacity": 1,
|
|
"major": "redhat",
|
|
"template_ready": true
|
|
}
|
|
],
|
|
"status": {
|
|
"ok": true
|
|
}
|
|
}
|
|
```
|
|
|
|
Note: to enable poolsize and pooltemplate config endpoints it is necessary to set 'experimental_features: true' in your vmpooler configuration. A 405 is returned when you attempt to interact with these endpoints when this configuration option is not set.
|
|
|
|
##### POST /config/poolsize
|
|
|
|
Change pool size without having to restart the service.
|
|
|
|
All pool template changes requested must be for pools that exist in the vmpooler configuration running, or a 404 code will be returned
|
|
|
|
When a pool size is changed due to the configuration posted a 201 status will be returned. When the pool configuration is valid, but will not result in any changes, 200 is returned.
|
|
|
|
Pool size configuration changes persist through application restarts, and take precedence over a pool size value configured in the pool configuration provided when the application starts. This persistence is dependent on redis. So, if the redis data is lost then the configuration updates revert to those provided at startup at the next application start.
|
|
|
|
An authentication token is required in order to change pool configuration when authentication is configured.
|
|
Responses:
|
|
* 200 - No changes required
|
|
* 201 - Changes made on at least one pool with changes requested
|
|
* 400 - An invalid configuration was provided causing requested changes to fail
|
|
* 404 - An unknown error occurred
|
|
* 405 - The endpoint is disabled because experimental features are disabled
|
|
```
|
|
$ curl -X POST -H "Content-Type: application/json" -d '{"debian-7-i386":"2","debian-7-x86_64":"1"}' --url https://vmpooler.company.com/api/v1/config/poolsize
|
|
```
|
|
```json
|
|
{
|
|
"ok": true
|
|
}
|
|
```
|
|
|
|
##### POST /config/pooltemplate
|
|
|
|
Change the template configured for a pool, and replenish the pool with instances built from the new template.
|
|
|
|
All pool template changes requested must be for pools that exist in the vmpooler configuration running, or a 404 code will be returned
|
|
|
|
When a pool template is changed due to the configuration posted a 201 status will be returned. When the pool configuration is valid, but will not result in any changes, 200 is returned.
|
|
|
|
A pool template being updated will cause the following actions, which are logged in vmpooler.log:
|
|
* Destroy all instances for the pool template being updated that are in the ready and pending state
|
|
* Halt repopulating the pool while creating template deltas for the newly configured template
|
|
* Unblock pool population and let the pool replenish with instances based on the newly configured template
|
|
|
|
Pool template changes persist through application restarts, and take precedence over a pool template configured in the pool configuration provided when the application starts. This persistence is dependent on redis. As a result, if the redis data is lost then the configuration values revert to those provided at startup at the next application start.
|
|
|
|
An authentication token is required in order to change pool configuration when authentication is configured.
|
|
|
|
Responses:
|
|
* 200 - No changes required
|
|
* 201 - Changes made on at least one pool with changes requested
|
|
* 400 - An invalid configuration was provided causing requested changes to fail
|
|
* 404 - An unknown error occurred
|
|
* 405 - The endpoint is disabled because experimental features are disabled
|
|
```
|
|
$ curl -X POST -H "Content-Type: application/json" -d '{"debian-7-i386":"templates/debian-7-i386"}' --url https://vmpooler.company.com/api/v1/config/pooltemplate
|
|
```
|
|
```json
|
|
{
|
|
"ok": true
|
|
}
|
|
```
|