State formulas

First a simple terminology used within this chapter.

State

A reusable declaration that configures a specific part of a system. Each state is defined using a state declaration.

Formula

A collection of state and pillar files that configure an application or system component. Most Formulas are made up of several states spread across multiple state files.

Formulas use directory structure proposed by official Salt formula conventions and best practices described in the SaltStack documentation. Official SaltStack Formulas are located in Salt formula conventions.

Formula layout

Every formula should have the following layout:

service-formula
|-- _grains/
|   `-- service.yml
|-- _modules/
|   `-- service.yml
|-- _states/
|   `-- service.yml
|-- service_name/
|   `-- files/
|       |-- config1.yml
|       `-- config2.yml
|   |-- map.jinja
|   |-- init.sls
|   |-- _common.sls
|   |-- role1.sls
|   `-- role2/
|       |-- init.sls
|       |-- service.sls
|       `-- more.sls
|-- debian/
|   ├── changelog
|   ├── compat
|   ├── control
|   ├── copyright
|   ├── docs
|   ├── install
|   ├── rules
|   └── source
|       └── format
|-- metadata/
|   `-- service/
|       |-- role1/
|       |   |-- deployment1.yml
|       |   `-- deployment2.yml
|       `-- role2/
|           `-- deployment3.yml
|-- CHANGELOG.rst
|-- LICENSE
|-- pillar.example
|-- README.rst
`-- VERSION

Content of the formula directories in more detail.

_grains/
Optional grain modules
_modules/
Optional execution modules
_states/
Optional state modules
service/
Salt state files
debian/
apt package metadata
metadata/
Pillar/reclass metadata

Salt states files

Salt state files are located in service_name directory.

service/map.jinja

Map helps to clean the differences among operating systems.

Following snippet uses YAML to serialize the data and is the recommended way to write map.jinja file as YAML can be easily extended in place.

{%- load_yaml as role1_defaults %}
Debian:
  pkgs:
  - python-psycopg2
  dir:
    base: /srv/service/venv
    home: /var/lib/service
RedHat:
  pkgs:
  - python-psycopg2
  dir:
    base: /srv/service/venv
    home: /var/lib/service
    workspace: /srv/service/workspace
{%- endload %}

{%- set role1 = salt['grains.filter_by'](role1_defaults, merge=salt['pillar.get']('service:role1')) %}

Following snippet uses JSON to serialize the data and was favored in past.

{% set api = salt['grains.filter_by']({
    'Debian': {
        'pkgs': ['salt-api'],
        'service': 'salt-api',
    },
    'RedHat': {
        'pkgs': ['salt-api'],
        'service': 'salt-api',
    },
}, merge=salt['pillar.get']('salt:api')) %}

Following snippet sets different common role parameters according to service:role:source:engine pillar variable of given service role.

{%- set source_engine = salt['pillar.get']('service:role:source:engine') %}

{%- load_yaml as base_defaults %}
{%- if source_engine == 'git' %}
Debian:
  pkgs:
  - python-psycopg2
  dir:
    base: /srv/service/venv
    home: /var/lib/service
    workspace: /srv/service/workspace
{%- else %}
Debian:
  pkgs:
  - helpdesk
  dir:
    base: /usr/lib/service
{%- endif %}
{%- endload %}

service/init.sls

Conditional include of individual service roles.

include:
{% if pillar.service.role1 is defined %}
- service.role1
{% endif %}
{% if pillar.service.role2 is defined %}
- service.role2
{% endif %}

For simple roles use one file as role1.sls, for more complex roles use individual directories as role2.

service/init.sls file allows the node catalog to be role agnostic.

[email protected]:-# salt-call state.show_top
[INFO    ] Loading fresh modules for state activity
local:
    ----------
    base:
        - linux
        - openssh
        - ntp
        - salt
        - nodejs
        - postgresql
        - rabbitmq
        - redis
        - ruby

Service metadata will are stored as services grain.

[email protected]:-# salt-call grains.item services
local:
    ----------
    services:
        - linux
        - openssh
        - ntp
        - salt
        - nodejs
        - postgresql
        - rabbitmq
        - redis
        - ruby

And individual service roles metadata are stored as detailed roles grain.

[email protected]:-# salt-call grains.item roles
local:
    ----------
    roles:
        - git.client
        - postgresql.server
        - nodejs.environment
        - ntp.client
        - linux.storage
        - linux.system
        - linux.network
        - redis.server
        - rabbitmq.server
        - python.environment
        - backupninja.client
        - nginx.server
        - openssh.client
        - openssh.server
        - salt.minion
        - sphinx.server

Note

It is recommended to run state.sls salt prior the state.highstate command as grains may not be generated properly and some configuration parameters not set at all.

service/role1.sls

Actual salt state resources that enforce service existence. Common production and recommended pattern is to install packages, setup configuration files and ensure the service is up and running.

{%- from "redis/map.jinja" import server with context %}
{%- if server.enabled %}

redis_packages:
  pkg.installed:
  - names: {{ server.pkgs }}

{{ server.dir.conf }}/redis.conf:
  file.managed:
  - source: salt://redis/files/redis.conf
  - template: jinja
  - user: root
  - group: root
  - mode: 644
  - require:
    - pkg: redis_packages

redis_service:
  service.running:
  - enable: true
  - name: {{ server.service }}
  - watch:
    - file: {{ server.dir.conf }}/redis.conf

{%- endif %}

For development purposes other installation than s

Note

The role for role.enabled condition is to restrict the give service role from execution with default parametes, the single error is thrown instead. You can optionaly add else statement to disable or completely remove given service role.

service/role2/init.sls

This approach is used with more complex roles, it is similar to service/init.sls, but uses conditions to further limit the inclusion of unnecessary files.

For example Linux network role includes conditionally hosts and interfaces.

{%- from "linux/map.jinja" import network with context %}
include:
- linux.network.hostname
{%- if network.host|length > 0 %}
- linux.network.host
{%- endif %}
{%- if network.interface|length > 0 %}
- linux.network.interface
{%- endif %}
- linux.network.proxy

Coding styles for state files

Good styling practices for writing salt state declarations.

Line length above 80 characters

As a ‘standard code width limit’ and for historical reasons - IBM punch card had exactly 80 columns.

Single line declaration

Avoid extending your code by adding single-line declarations. It makes your code much cleaner and easier to parse / grep while searching for those declarations.

The bad example:

python:
  pkg:
    - installed

The correct example:

python:
  pkg.installed

No newline at the end of the file

Each line should be terminated in a newline character, including the last one. Some programs have problems processing the last line of a file if it isn’t newline terminated.

Trailing whitespace characters

Trailing whitespaces take more spaces than necessary, any regexp based searches won’t return lines as a result due to trailing whitespace(s).

Versioning

Formula are versioned according to Semantic Versioning, http://semver.org/.

Note

Given a version number MAJOR.MINOR.PATCH, increment the:

  1. MAJOR version when you make incompatible API changes,
  2. MINOR version when you add functionality in a backwards-compatible manner, and
  3. PATCH version when you make backwards-compatible bug fixes.

Additional labels for pre-release and build metadata are available as extensions to the MAJOR.MINOR.PATCH format.

Formula versions are tracked using Git tags as well as the VERSION file in the formula repository. The VERSION file should contain the currently released version of the particular formula.

Lab: Using pillars in States

{% set variable = salt['pillar.get']('apache', {}) %}
{{ variable }}

Lab: Create states with Salt formulas

Download https://github.com/saltstack-formulas/hostsfile-formula

Add following to master configuration:

file_roots:
  base:
    - /srv/salt/files/hostsfile-formula

Add following to minion configuration:

mine_functions:
  hostsfile_interface:
    mine_function: network.ip_addrs
    iface: eth0