Events and reactors

The Salt Event System is used to fire off events enabling third party applications or external processes to react to behavior within Salt.

The event system is comprised of a two primary components:

  • The event sockets which publishes events.
  • The event library which can listen to events and send events into the salt system.

Reactor system

Salt’s Reactor system gives Salt the ability to trigger actions in response to an event. It is a simple interface to watching Salt’s event bus for event tags that match a given pattern and then running one or more commands in response.

Salt event system with Reactor

This system binds sls files to event tags on the master. These sls files then define reactions. This means that the reactor system has two parts. First, the reactor option needs to be set in the master configuration file. The reactor option allows for event tags to be associated with sls reaction files. Second, these reaction files use highdata (like the state system) to define reactions to be executed.

Beacons

The beacon system allows the minion to hook into a variety of system processes and continually monitor these processes. When monitored activity occurs in a system process, an event is sent on the Salt event bus that can be used to trigger a reactor.

Salt beacons can currently monitor and send Salt events for many system activities, including:

  • File system changes
  • System load
  • Service status
  • Shell activity, such as user login
  • Network and disk usage

Avoid event loops

It is important to carefully consider the possibility of creating a loop between a reactor and a beacon. For example, one might set up a beacon which monitors whether a file is read which in turn fires a reactor to run a state which in turn reads the file and re-fires the beacon.

To avoid these types of scenarios, the disable_during_state_run argument may be set. If a state run is in progress, the beacon will not be run on its regular interval until the minion detects that the state run has completed, at which point the normal beacon interval will resume.

Lab: Listening to events

Start listening to an events on cfg01 node.

cfg01# salt-run state.event pretty=True

Fire an event from svc01 node.

svc01# salt-call event.send 'FIRED_EVENT'
    local:
        True

And now look at the console on cfg01 node, received event outputs:

...
    FIRED_EVENT     {
        "_stamp": "2016-02-17T22:20:30.663184",
        "cmd": "_minion_event",
        "data": {
            "__pub_fun": "event.send",
            "__pub_jid": "20160217222031653414",
            "__pub_pid": 27317,
            "__pub_tgt": "salt-call"
        },
        "id": "svc01.saltstack.local",
        "tag": "FIRED_EVENT"
    }
    salt/job/20160217232030678311/ret/svc01.saltstack.local      {
        "_stamp": "2016-02-17T22:20:30.679731",
        "arg": [
            "FIRED_EVENT"
        ],
        "cmd": "_return",
        "fun": "event.send",
        "fun_args": [
            "FIRED_EVENT"
        ],
        "id": "svc01.saltstack.local",
        "jid": "20160217232030678311",
        "retcode": 0,
        "return": true,
        "tgt": "svc01.saltstack.local",
        "tgt_type": "glob"
    }

Lab: Implementing beacons

This example demonstrates configuring the inotify beacon to monitor a file for changes, and then create a backup each time a change is detected.

The inotify beacon requires Pyinotify on the minion, install it using salt myminion pkg.install python-inotify.

[email protected]:~# salt-call pkg.install python-pip
[email protected]:~# salt-call pip.install inotify

Beacons are typically enabled by placing a beacons: top level block in the minion configuration file on node svc01 at /etc/salt/minion.d/beacons.conf.

beacons:
  inotify:
    /root/importantfile: {}

Then restart the Salt minion on node svc01.

[email protected]:~# service salt-minion restart

On your Salt master, start the event runner using the following command.

[email protected]:~# salt-run state.event pretty=true

Now you can trigger the beacon by touching the configured file

[email protected]:~# touch /root/importantfile
salt/beacon/svc01/inotify/root/importantfile  {
 "_stamp": "2015-09-09T15:59:37.972753",
 "data": {
     "change": "IN_IGNORED",
     "id": "svc01",
     "path": "/root/importantfile"
 },
 "tag": "salt/beacon/svc01/inotify/root/importantfile"
}

This indicates that the event is being captured and sent correctly. Now you can create a reactor to take action when this event occurs.

Lab: Using reactors

On your Salt master, create a file named /srv/salt/env/prd/backup.sls. If the reactor directory doesn’t exist, create it.

Add the following to backup.sls:

backup file:
 cmd.file.copy:
   - tgt: {{ data['data']['id'] }}
   - arg:
     - {{ data['data']['path'] }}
     - {{ data['data']['path'] }}.bak

Next, add the following code to trigger the reactor at /ect/salt/master.d/beacons.conf configuration file.

reactor:
  - salt/beacon/*/inotify/*/important-file:
    - /tmp/important

This reactor creates a backup each time a file named importantfile is modified on a minion that has the inotify beacon configured as previously shown.

Note

You can have only one top level reactor section, so if one already exists, add this code to the existing section. See Understanding the Structure of Reactor Formulas to learn more about reactor SLS syntax.

On your minion, make and save another change to importantfile. On the Salt master, you’ll see debug messages that indicate the event was received and the file.copy job was sent. When you list the directory on the minion, you’ll now see importantfile.bak.

All beacons are configured using a similar process of enabling the beacon, writing a reactor SLS, and mapping a beacon event to the reactor SLS.