1 ADEPT, the Deterministic Executor for Parallel Testing¶
ADEPT includes a small python program and a collection of Ansible playbooks, and related configurations. Together, they help orchestrate parallel creation, configuration, use and cleanup of ephemeral virtual machines.
1.1 Introduction¶
ADEPT provides the ground-work for managing and executing tests against systems through multiple Ansible playbooks. It supports the industry-standard practice of utilizing a separate triggering/scheduling versus execution host. Jobs may be defined in-repo or externally, and use the standard Ansible directory structure. Jobs may define their own playbooks, roles, and scripts or re-use any of the content provided.
Systems management, be it local or cloud, is extremely flexible. Though an OpenStack setup is the default, any custom host-management tooling may be used. Changing and maintaining management tooling is very smooth since the interface is simple and well defined. No persistent systems or data-stores are required, though both may be utilized.
Finally, since initiator-host capabilities are often unknown and fixed,
ADEPT has very low dependency and resource requirements. The included
adept.py
program, along with a simple YAML input file directives,
bootstraps Ansible operations for every job. While Ansible and it’s
dependencies are gathered at runtime, and confined within a python
virtual environment.
1.2 Latest Documentation¶
For the latest, most up to date documentation please visit http://A-D-E-P-T.readthedocs.io/en/latest
The latest Docker Autotest documentation is located at: http://docker-autotest.readthedocs.io
1.3 Prerequisites¶
- Red Hat based host (RHEL, CentOS, Fedora, etc), subscribed and fully updated.
- Python 2.7
- PyYAML 3.10 or later
- libselinux-python 2.0 or later
- rsync 2.5 or later
- Ansible 2.1 or later (EPEL)
- Root access not required
1.3.1 Testing/Development¶
- Ansible 2.3 or later
- python-unittest2 1.0 or later
- python2-mock 1.8 or later
- pylint 1.4 or later
- python-virtualenv or python2-virtualenv (EPEL)
- Optional (for building documentation),
make
,python-sphinx
, anddocutils
or equivalent for your platform.
1.3.2 OpenStack support:¶
- redhat-rpm-config
- gcc
- python-virtualenv or python2-virtualenv (EPEL)
- openssl-devel
- OpenStack client configuration credentials
1.4 Quickstart¶
Ansible 2.3 or later is required, along with the items listed under prerequisites.
This demonstration doesn’t do anything extraordinarily useful. However, it does
demonstrate ADEPT’s essential functions. The tasks to be performed (the job)
exist as a sparse standard Ansible directory layout,
under jobs/quickstart
. All files in the job directory, overwrite identically
named files under kommandir/
(after a working copy is made).
# Optional: set $ANSIBLE_PRIVATE_KEY_FILE
# if unset, a new temporary key will be generated in workspace
$ export ANSIBLE_PRIVATE_KEY_FILE="$HOME/.ssh/id_rsa"
# Create a place for runtime details and results to be stored
$ export WORKSPACE="$(mktemp -d --suffix=.workspace)"
# Run the ADEPT-three-step (keyboard finger-dance)
$ ./adept.py setup $WORKSPACE exekutir.xn
$ ./adept.py run $WORKSPACE exekutir.xn
$ ./adept.py cleanup $WORKSPACE exekutir.xn
# Cleanup the workspace, when you're done looking at it.
$ rm -rf $WORKSPACE
Notes:
- To see select debugging output (select variable values and infos),
append
-e adept_debug=true
onto any of theadept.py
lines above. - Setting
-e adept_debug=true
will prevent roles in the cleanup context from removing any leftover files in the workspaces. - To see massive amounts of ugly details, append one or more
--verbose
, options onto any of theadept.py
lines above.
1.5 Basic Definitions¶
context
- The label given to the end-state / completion of a transition.
Analogous to a “phase”, or collection of related steps. The context
label (string) is assumed to pass unmodified, through all
facilities (adept.py files, playbooks, scripts, etc.) and through
all layers (original calling host, through slave, and into testing
hosts). No facility or layer will alter the context label from
the one originally passed into the first, lowest-level call of
adept.py
. transition
- The collection of steps necessary to realize a context. Analogous to the act of performing all described tasks within a “phase” to reach some end-state.
setup
,run
, andcleanup
- The three context labels currently used in ADEPT. Operationally, the run context is dependent on a successful setup transition. However, the cleanup context transition does not depend on success or failure of either setup or run.
job
- A single, top-level invocation through all context transitions, concluding in a resource-clean, final end-state. Independent of the existence of any results or useful data. Logically represented by a set of files within the path pointed to by job_path.
exekutir
- The initial, starting host that executes one or more jobs.
kommandir
- The name of the “slave” VM as referenced from within playbooks,
adept files, and configurations. When a member of the
nocloud
group (see kommandir_groups), this will be the same host as the exekutir. peon
- The lowest-level VM used for the grunt-work of testing or performing some temporary but useful task. Assumed to be controlled by a one-way connection from the kommandir. Cannot and must not be able to access the kommandir or exekutir hosts.
1.6 Topology¶
1.6.1 Systems¶
exekutir --> kommandir --> peon \ -> peon \ -> peon
-or-
exekutir/kommandir --> peon \ -> peon \ -> peon
1.6.2 Directory Layout¶
docs/
- Source for all documentation input and output.
exekutir/
- Standard Ansible directory layout, dedicated specifically for use by the exekutir host. Some roles are shared by the kommandir, but all contents are limited by a reduced set of prerequisites. This directory is transferred verbatim to the exekutir’s workspace during The setup context transition.
kommandir/
Standard Ansible directory layout, dedicated specifically for use by the kommandir and peons. This directory is transferred to kommandir_workspace on the exekutir and becomes workspace on the kommandir.
Any/all files in the copy may be overridden by files from job_path. The most important of which is
job.xn
. This is the primary entry point on the kommandir for the job.
- job_path
- Sparse standard Ansible directory layout, dedicated
to one or more jobs. Its contents will overwrite any identically named files
already copied to the kommandir_workspace, on the exekutir. However,
copies of all the default playbooks are made with a
default_
prefix. This allows customizedsetup.yml
,run.yml
, andcleanup.yml
to re-use the defaults if/where needed. jobs/
- Default directory containing public job definition subdirectories, e.g., subdirectories that job_path could reference.
1.7 Operational Overview¶
This is a general, high-level outline of key steps performed during the three standard transitions (setup, run and cleanup). It omits many small details, but the overall sequence should more/less match reality.
The names in parenthesis following each bullet’s text denote the adept.py
transition file (*.xn
), or the source Ansible role or script.
1.7.1 The setup context transition¶
Fundamental setup of exekutir’s ssh keys, and workspace directory. (
exekutir.xn
)- Copy repository’s
exekutir/*
to the workspace. - Copy
$ANSIBLE_PRIVATE_KEY_FILE
(and.pub
) or generate new{{workspace}}/ssh/exekutir_key
.
- Copy repository’s
Intermediate exekutir setup, prepare the initial kommandir_workspace directory for future remote synchronization. (
exekutir/roles/exekutir_workspace_setup
role)- Copy
kommandir/*
(from repo.) to local{{kommandir_workspace}}
- Backup default playbooks to allow selective re-use.
i.e.
{{kommandir_workspace}}/*.yml --> {{kommandir_workspace}}/default_*.yml
- Copy contents of job_path to kommandir_workspace, overwriting any existing files to allow customization.
- Generate unique ssh key for kommandir to use for this job,
in
{{workspace}}/kommandir_workspace/ssh/kommandir_key
- Copy
Create or discover the remote kommandir VM if configured. (
exekutir/roles/kommandir_discovered
role)- The exekutir has set the kommandir’s Ansible group membership from contents
of the kommandir_groups list. (
exekutir/roles/common
role) - Membership in the ``nocloud` Ansible group <local_kommandir>`_ will
always cause the kommandir to be the same host as the exekutir
(i.e.,
ansible_host: localhost
). - Otherwise, the cloud_provisioning_command is executed,
with its
stdout
parsed as a YAML dictionary, updating{{workspace}}/inventory/host_vars/kommandir.yml
inventory variables.
- The exekutir has set the kommandir’s Ansible group membership from contents
of the kommandir_groups list. (
Complete kommandir VM setup (if needed), install packages, setup storage, etc. (
exekutir/roles/installed
andkommandir_setup
roles)Finalize the workspace for a remote kommandir VM (if used). (
exekutir/roles/kommandir_workspace_update
role)- A remote kommandir has a user named ``{{uuid}}` created <uuid>`_.
- Remote kommandir’s
/home/{{uuid}}
(its workspace) synchronized from the local{{kommandir_workspace}}
copy.
- Run
job.xn
on kommandir (local or remote). This file may be overridden by a copy from job_path to customize testing operations. The default simply executes thesetup.yml
playbook to create and configure peons for testing purposes. (exekutir.xn
) - For a remote kommandir,
/home/{{uuid}}
is synchronized back down to the exekutir’s local{{kommandir_workspace}}
directory. This prevents state from being bound to a remote system. (exekutir/roles/kommandir_to_exekutir_sync
role)
1.7.2 The run context transition¶
- Create or discover the kommandir VM. If needed, set it up, install packages, etc. exactly as in setup. This prevents the need to maintain a persistent slave-host.
- Synchronize the local
{{kommandir_workspace}}
to a remote kommandir (if used). (exekutir/roles/kommandir_workspace_update
) - Run
job.xn
on kommandir (local or remote). Same as in setup, this may have been overridden by a copy from job_path. The default simply executes therun.yml
playbook. (exekutir.xn
) - For a remote kommandir,
/home/{{uuid}}
is synchronized back down to the exekutir’s local{{kommandir_workspace}}
directory. This prevents state from being bound to a remote system. (exekutir/roles/kommandir_to_exekutir_sync
role)
1.7.3 The cleanup context transition.¶
N/B: This should always run, whether or not any other contexts were successful. It may not exit successfully, but it must never orphan a remote kommandir or any peons.
- Create or discover the kommandir VM. If needed, set it up, install packages, etc. This may fail again if it also failed during setup or run - this is normal.
- If possible, synchronize the local
{{kommandir_workspace}}
to a remote kommandir (if used). (exekutir/roles/kommandir_workspace_update
) - If accessible, run
job.xn
on kommandir (local or remote). The default copy simply executes thecleanup.yml
playbook to handle deallocation of peons and other resources. (exekutir.xn
) - For a remote kommandir,
/home/{{uuid}}
is synchronized back down to the exekutir’s local{{kommandir_workspace}}
directory. This prevents state from being bound to a remote system. (exekutir/roles/kommandir_to_exekutir_sync
role) - Examine the state of the remote kommandir. If configured
for automatic destruction (after some number of days), it
will be destroyed. It will also be removed if the stonith
flag is
True
, to support testing of the kommandir setup itself. (exekutir/roles/kommandir_destroyed/
role)
1.8 Examples¶
This section provides examples covering various usages and scenarios. They should all be taken as general guidelines, since they easily become out of date. If you notice a discrepancy, raise an issue regarding it.
1.8.1 Adding a public job¶
This example shows how to add a new public job into the ADEPT repository. It’s possible to store job details elsewhere, as documented here.
The first step for any new job is to decide on its name, and what its speciality is. Begin by creating a subdirectory, and a
README
(jinja2 template) to describe its purpose. TheREADME
will be rendered into the results directory of the workspace for reference. In this example, a job calledexample
will be created.$ mkdir -p jobs/example $ cat << EOF > jobs/example/README This job ({{ job_name }}) is intended to demonstrate adding a simple job. If it's found in production, somebody has made a terrible, terrible mistake. (src: {{ job_path }}/{{ job_docs_template_src }}) EOF
Now you need to determine which aspects of the kommandir directory you want overwritten. Start with identifying the peons you need created and setup. For this example, we’ll limit the selection to only a single peon. Note, it’s also possible to use custom peon definitions by also adding/overriding files in the job’s
inventory/host_vars
subdirectory.$ ls kommandir/inventory/host_vars ...many... fedora-25-docker-latest.yml # The one we'll use here ...others... $ mkdir -p jobs/example/inventory $ cat << EOF > jobs/example/inventory/peons [peons] fedora-25-docker-latest EOF
In this example, there’s an additional Ansible Role we’d like to apply after all the default plays. Since the default playbooks are all renamed with a
default_
prefix (exekutir/roles/exekutir_workspace_setup
role), we can simply overwritesetup.yml
to includedefault_setup.yml
and then apply our special role.$ mkdir -p jobs/example/roles/frobnicated/tasks $ cat << EOF > jobs/example/roles/frobnicated/tasks/main.yml --- - name: Make docker daemon run in debug mode lineinfile: path: /etc/sysconfig/docker regexp: "^OPTIONS=[\\'\\"]?(.*)[\\'\\"]?" line: "OPTIONS='\\1 --debug=true --log-level=debug'" - name: Docker daemon is restarted to reload changed options service: name: docker state: restarted EOF $ cat << EOF > jobs/example/setup.yml --- # First do all the original, default plays. - include: default_setup.yml # Now apply the new frobnicated role - hosts: peons vars_files: - kommandir_vars.yml roles: - frobnicated EOF
There’s no reason to go crazy with debug-mode tests, so we’ll just re-use whatever the basic job has going for it.
$ ln -s ../basic/kommandir_vars.yml ../basic/templates jobs/example/
Finally, the last step is to make sure the new job works. This should be performed on a system which meets the Testing/Development prerequisites.
# (from the adept repository root) $ export WORKSPACE=/tmp/workspace $ rm -rf $WORKSPACE $ mkdir -p $WORKSPACE
$ cat << EOF > $WORKSPACE/clouds.yml --- clouds: default: auth_type: thepassword auth: auth_url: http://example.com/v2.0 password: foobar tenant_name: baz username: snafu regions: - Oz verify: False
A simple
$WORKSPACE/exekutir_vars.yml
causes the exekutir to run the example job. See the variables_reference for more details.$ cat << EOF > $WORKSPACE/exekutir_vars.yml kommandir_groups: ["nocloud"] public_peons: True job_path: $PWD/jobs/example kommandir_name_prefix: "$USER" extra_kommandir_setup: command: > cp "{{ hostvars.exekutir.workspace }}/clouds.yml" "{{ hostvars.exekutir.kommandir_workspace }}/" EOF
Then we kick it off.
$ ./adept.py setup $WORKSPACE exekutir.xn localhost ###################################### Parameters: optional = '' xn = 'exekutir.xn' workspace = '/tmp/workspace' context = 'setup' ...cut...many...lines...
$ ./adept.py run $WORKSPACE exekutir.xn localhost ###################################### Parameters: optional = '' xn = 'exekutir.xn' workspace = '/tmp/workspace' context = 'run' ...cut...many...lines...
$ ./adept.py cleanup $WORKSPACE exekutir.xn localhost ###################################### Parameters: optional = '' xn = 'exekutir.xn' workspace = '/tmp/workspace' context = 'cleanup' ...cut...many...lines... $ ls $WORKSPACE ansible.cfg exekutir_ansible.log roles cache exekutir_setup_after_job.exit run_after_job.yml callback_plugins exekutir_vars.yml run_before_job.yml cleanup_after_job.yml inventory setup_after_job.yml cleanup_before_job.yml kommandir_setup.exit setup_before_job.yml clouds.yml kommandir_workspace ssh dockertest results
1.8.2 Adding a private job¶
In certain cases, it’s desireable for the details of a particular job to live outside of the
ADEPT repository. In this case, the steps are exactly the same as Adding a public job
except for one / possibly-two variables in exekutir_vars.yml
:
job_path: /path/to/job/something/else job_name: something
Here, it was necessary to set both job_name and job_path. If only the later was set,
the job_name
would have default to else
instead of something
. See the
variables_reference for more information
1.9 Hacking¶
1.9.1 Run the unittests¶
This requires the additional test/development prerequisites These tests run relatively quickly, and only do a self-sanity check to verify major operational areas.
$ unit2
...............................s......................
----------------------------------------------------------------------
Ran 54 tests in 9.998s
OK (skipped=1)
1.9.2 Run the CI test job¶
This is a special ADEPT-job which runs entirely on the local machine,
and verifies the operations of most major exekutir and kommandir plays.
It does not have perfect coverage, for example, no cloud-based resources
used. It can be run with adept_debug
and/or --verbose
modes
to retain the temporary workspace for examination.
$ ./test_exekutir_xn.sh
localhost ######################################
Parameters:
optional = '-e some_magic_variable_for_testing='value_for_magic_variable''
xn = 'exekutir.xn'
workspace = '/tmp/tmp.wfyfHGypgq.adept.workspace'
context = 'setup'
...cut...
Examining exit files
Checking kommandir discovery (before job.xn) cleanup exit file contains 0
Checking setup exit files
Verifying exekutir_setup_after_job.exit file contains 0
Verifying kommandir_setup.exit file contains 0
Checking contents of test_file_from_setup.txt
Checking run exit files
Verifying exekutir_run_after_job.exit file contains 0
Verifying kommandir_run.exit file contains 0
Checking contents of test_file_from_run.txt
Checking cleanup exit files
Verifying exekutir_cleanup_after_job.exit file contains 0
Verifying kommandir_cleanup.exit file contains 0
Checking contents of test_file_from_cleanup.txt
All checks pass
1.9.3 Local Kommandir¶
Having a kommandir (“slave”) node is useful in production because it offloads much of the grunt-work onto a dedicated system, with dedicated resources. It also decouples the job environment/setup from the execution environment. Using one only makes sense in production-environments where the 5-minute setup cost can be spread over tens or hundreds of jobs.
However, for local testing/development purposes, the extra kommandir setup time
can be excessive. If the local system (the exekutir) meets all the chosen cloud,
kommandir, and job prerequisites, it’s possible to use the exekutir also as the
kommandir. Note, however, the kommandir’s job.xn
transition file and playbooks will
still run from a dedicated workspace (created by the Exekutir).
Set the Exekutir’s ``kommandir_groups` variable <kommandir_groups>`_
to include nocloud
. If required, also enable
the flag to create network-accessable peons.
kommandir_groups: ["nocloud"]
public_peons: True
For reglar/automated use, avoid repeating any context transition more
than once, against the same workspace or manually running Ansible.
However, for development/debugging purposes, depending on the job-specifics,
most contexts may be re-applied (within reason). Doing this may require
manual manipulation of the uuid unless existing VMs are to be re-used.
Otherwise, it’s safest to apply the cleanup
context, then start over again
with setup
against a fresh workspace, with a fresh uuid.
1.9.4 OpenStack Cloud¶
This is the default for all bundled peons as per the peon_cloud_group variable
value. The openstack group variables demand that you either set the $OS_*
environment variables correctly, or dropped a clouds.yml
file in
the relevant workspace.
Important, verify that all the default peon images are accessable to your tenant by examining the group variable file:
kommandir/inventory/group_vars/openstack/peon_images.yml
. If any are incorrect, fix them before proceeding. Otherwise, those peons will most certainly fail to be created.Setup your OpenStack credentials via the standard
os-client-config
fileclouds.yml
, in the workspace, as show below. The options are specific to your particular OpenStack setup. See the format and options, documented here.# (from the adept repository root) $ export WORKSPACE=/tmp/workspace $ rm -rf $WORKSPACE $ mkdir -p $WORKSPACE
$ cat << EOF > $WORKSPACE/clouds.yml --- clouds: default: auth_type: thepassword auth: auth_url: http://example.com/v2.0 password: foobar tenant_name: baz username: snafu regions: - Oz verify: False
Populate the *exekutir’s* variables. In this example, the default (bundled) peon definitions are used (from
kommandir/inventory/host_vars/
). The other values select the job, name the kommandir VM, enable debugging and setup subscriptions. The final value makes sure the kommandir VM has access to the same cloud for creating the peons$ cat << EOF > $WORKSPACE/exekutir_vars.yml --- job_path: $PWD/jobs/basic kommandir_name_prefix: "$USER" adept_debug: True rhsm: username: nobody@example.com password: thepassword extra_kommandir_setup: command: > cp "{{ hostvars.exekutir.workspace }}/clouds.yml" "{{ hostvars.exekutir.kommandir_workspace }}/" EOF
Note: If you want/need access to the peons as well, be sure to enable the public_peons flag.
Apply the ADEPT
setup
context. Once this completes, a copy of all runtime source material will have been transferred to the workspace. This includes updating initialexekutir_vars.yml
and inventory files. As noted, manual changes made to the source, will not be reflected at runtime unless the workspace is manually updated.$ ./adept.py setup $WORKSPACE exekutir.xn localhost ###################################### Parameters: optional = '' xn = 'exekutir.xn' workspace = '/tmp/workspace' context = 'setup' ...cut...many...lines...
Apply the ADEPT
run
context and/or inspect the workspace state.$ ./adept.py run $WORKSPACE exekutir.xn localhost ###################################### Parameters: optional = '' xn = 'exekutir.xn' workspace = '/tmp/workspace' context = 'run' ...cut...many...lines...
Whether or not
setup
orrun
were successful, always applycleanup
to release cloud resources.$ ./adept.py cleanup $WORKSPACE exekutir.xn localhost ###################################### Parameters: optional = '' xn = 'exekutir.xn' workspace = '/tmp/workspace' context = 'cleanup' ...cut...many...lines... $ ls $WORKSPACE ansible.cfg exekutir_ansible.log roles cache exekutir_setup_after_job.exit run_after_job.yml callback_plugins exekutir_vars.yml run_before_job.yml cleanup_after_job.yml inventory setup_after_job.yml cleanup_before_job.yml kommandir_setup.exit setup_before_job.yml clouds.yml kommandir_workspace ssh dockertest results
1.9.5 Other Clouds¶
A multitude of topologies are possible by changing the values of a few host and group variables.
From the exekutir’s perspective, the kommandir will be created according to whichever
group is set via kommandir_groups. For example, “openstack” will cause the group variables
from exekutir/inventory/group_vars/openstack.yml
to be brought in.
From the kommandir’s perspective, all default peons are created by membership dictated
by the peon_cloud_group. This value is used to help populate the peon_groups variable.
The default value of “openstack” will cause all default peons to created according to variables
defined in the group variables files kommandir/inventory/group_vars/openstack/*.yml
.
1.10 ADEPT Variables Reference¶
From the perspective of a task in a playbook, the lookup order is defined by Ansible’s documentation. This reference is just an overview of some of the common variables used throughout ADEPT’s plays and tasks.
1.10.1 High-level Variables¶
Use of variable-overrides on the Ansible command-line is highly discouraged. It has a very hard, non-obvious side-effect, which can make debugging very difficult: It forces those variables to be read-only, and silently fails to change their contents. Instead, there are two high-level YAML-dictionary variable files guaranteed to exist in workspace.
exekutir_vars.yml
- This file’s variables are included in every play that runs on the exekutir.
Whatever its original source, the exekutir’s
exekutir_workspace_setup
role will manipulate its contents during setup, to ensure consistency of critical variables like uuid. kommandir_vars.yml
- This only resides within the kommandir_workspace,
but may have originated from inside job_path, copied by extra_exekutir_setup,
or extra_kommandir_setup. It is included by every default play that runs on
the kommandir. In all cases, the exekutir’s
exekutir_workspace_setup
role will always create and manipulate its contents during *setup* along with hard-coding critical values, like uuid.
1.10.2 Low-level Variables¶
Descriptions of specific, widely used or very important variables are defined below.
This includes variables defined by tasks, playbooks, host_vars
and group_vars
files. General guidance is to define a variable in one place, which is as close
to its usage-context as possible.
For example, if a variable is specific to a…
- …playbook, it should wind up in
kommandir_vars.yml
orexekutir_vars.yml
. - …role, place it in that role’s
defaults/main.yml
orvars/main.yml
. - …group of hosts, define it under a named group (subdirectory) of
inventory/group_vars
. - …specific host, define it under that hosts
inventory/host_vars
. - …one or more tasks, use
set_facts
in the specific role, or the “common” role.
adept_debug
- Boolean, defaults to
False
. Exekutir’s value copied to and overrides kommandir’s. Enables many Ansibledebug
statements across many roles that display variable values. Also disables removing files during the *cleanup* transition. cleanup
- Boolean, defaults to
True
. Exekutir’s value copied to kommandir’s. Enables/Disables removal of all peons by the kommandir, during the *cleanup* transition. See also, stonith.
cloud_environment
- Dictionary, defaults to
{}
. Not shared between exekutir and kommandir. Defines the environment variable names and values that should be set when executing cloud_provisioning_command and cloud_destruction_command. cloud_asserts
- List, defaults to
[]
. Not shared between exekutir and kommandir. List of Ansible conditionals (with jinja2 template resolution) which must all evaluate true, prior to executing cloud_provisioning_command and cloud_destruction_command.
cloud_provisioning_command
- Dictionary, defaults to
Null
. Not shared between exekutir and kommandir. Same keys/values used for the Ansibleshell
module, excludingenvironment
which is brought in by cloud_environment. Upon success,stdout
is expected to be a valid YAML dictionary document. All values in that document will replace the corresponding keys in the hostvarsfile variable.
cloud_destruction_command
- Dictionary, defaults to
Null
. Not shared between exekutir and kommandir. Same as cloud_provisioning_command, but defines the command for removing the host. docker_autotest_timeout
- Integer, huge number default.
Only used by
autotested
role (if enabled) on kommandir. Specifies number of minutes to set as the overall timeout for Autotest on each peon. empty
- Defines the set of values which are to be considered “not set” or “blank”. This is used as a convenience value for quickly testing whether or not a variable is not a boolean, and contains something useful.
extra_exekutir_setup
- Dictionary, defaults to
Null
. Not shared between exekutir and kommandir. Same keys/values used for the Ansibleshell
module. Represents a command to execute on the exekutir during theexekutir_workspace_setup
role. Its purpose is to allow additional files to be copied into the workspace, such as cloud or access credentials.
extra_kommandir_setup
- Dictionary, defaults to
Null
. Not shared between exekutir and kommandir. Same as extra_exekutir_setup. It is executed on the kommandir, after every time thekommandir_workspace_update
role is applied.
hostvarsfile
- Not shared between any host.
String, set to the current host’s YAML variables file.
Nominally
{{invendory_dir}}/host_vars/{{inventory_hostname}}.yml
.
job_name
- String, defaults to the
basename
of job_path. Exekutir’s value overrides kommandir’s. The value of this variable is primarily used to identify job-specific resources. For example, it is appended to the end of all peon names when they are created in an OpenStack cloud.
job_path
- String, defaults to
jobs/quickstart
. Exekutir’s value copied to kommandir. This is the absolute path containing all files, which should overwrite or support contents from the directory referenced by kommandir_workspace. This is the primary method to specialize a jobs activities. This is where you will find the job’skommandir_vars.yml
file. job_subthings
- List of strings, defaults to empty.
Only used by
autotested
role (if enabled) on kommandir. List of Docker Autotest sub/sub-subtest names to include in the run. When empty, all sub/sub-subtests are considered for running.
kommandir_groups
- List, defaults to
["nocloud"]
. Not shared between exekutir and kommandir. Any Ansible groups the kommandir should be made a member of on the exekutir. The listed groups indicate whichinventory/group_vars
files should be used on that host.
kommandir_workspace
- String, defaults to
{{ workspace }}/kommandir_workspace
on the exekutir. Only used in theexekutir/
playbooks. From the exekutir’s perspective, it represents the local path which contains the authoritative copy of the kommandir’s workspace. When the kommandir is a member of thenocloud
group no synchronization is done, so this will also be the kommandir’s actual{{workspace}}
. kommandir_name_prefix
- String, defaults to
null
. Not shared between exekutir and kommandir. When non-null, this is used as a prefix when discovering or creating a kommandir. It’s mainly used to control which kommandir is used for the job. For example, CI jobs testing ADEPT changes, should never use a production kommandir. needs_reboot
- Boolean, defaults to
False
. Only used by peons. If any role sets this toTrue
, subsequent application of therebooted
role will result in that host being rebooted, and then confirmed accessible. Afterwards, the value is always reset back toFalse
. no_log_synchronize
- Boolean, defaults to
True
. Exekutir’s value overrides kommandi’r. WhenFalse
andadept_debug
(above) isTrue
or--verbose
was used, the Ansiblesynchronize
module will output the full contents of its operation. This can be a *HUGE* number (many hundreds) of output lines. Even when debugging, it’s recommended to keep thisTrue
unless the details are really needed.
peon_groups
- List, no-default, mandatory.
Only defined for peons but only used by kommandir.
Defines the names of all groups a peon should be made a member of.
This determines many other variable values for peons, all of which
are defined by YAML files under
kommandir/inventory/group_vars/
.
peon_cloud_group
- String, defaults to
{{default_peon_cloud_group}}
- “openstack”. Only defined for peons but only used by kommandir. All of the bundled default peon definitions (underkommandir/inventory/host_vars
) list a single group indirectly by this value. Therefor changing the value of this one variable will affect (by group membership) the values brought in for cloud_environment, cloud_provisioning_command, cloud_destruction_command, etc.
public_peons
- Boolean, defaults to
False
. Only used by peons. WhenTrue
, the cloud_provisioning_command should make every effort to allow unrestricted network access to created peons. Otherwise, whenFalse
, unrestricted access is optional, except by the kommandir. pull_request_description
- String, defaults to undefined.
Exekutir’s value overrides kommandi’r.
When set to a string, this is assumed to be the description text contained
in the originating pull-request. Jobs may make use of this however they like.
Specifically, the
autotested
role will attempt to convert this into a parameter to autotest’s--args
option.
stonith
- Boolean, defaults to
False
. Only used by kommandir during theexekutir/roles/kommandir_destroyed
role. WhenTrue
during the cleanup context, it forces removal of the kommandir. This is used primarily during CI jobs for ADEPT itself, to ensure that a temporary kommandir is destroyed.
uuid
- DNS & Username compatible string, defaults to a random number. Exekutir’s value overrides kommandir’s. This is a critical value. It must never change throughout the duration of all context transitions, and for the lifetime of any kommandir. Its primary purpose is to prevent resource contention (hostnames, usernames, and directory names). However, for cloud-based kommandir’s, it is also utilized to prevent workspace location clashes.
workspace
- String, the path set by the
$WORKSPACE
environment variable byadept.py
. This is the place where all runtime state and results are stored. See also kommandir_workspace.