Storage Backends

The storage backend’s job is to manage the state for a session and to load and save data from a (usually) permanent storage medium, such as a database or filesystem.

The Triggers framework ships with a single storage backend that uses the Django cache to persist data. However, you can write and use your own storage backend if desired.

Anatomy of a Storage Backend

A session state is comprised of 3 primary components:

tasks: Dict[Text, TaskConfig]
This is effectively the same dict that was provided to the trigger manager’s update_configuration() method. Keys are the task names (e.g., t_importSubject), and values are triggers.types.TaskConfig objects.
instances: Dict[Text, TaskInstance]

Contains all of the task instances that have been created for this session. Keys are the instance names (e.g., t_importSubject#0), and values are triggers.types.TaskInstance objects.

Note

Task instances may appear in this dict even if they haven’t run yet.

metadata: Dict
Contains any additional metadata that the trigger manager and/or storage backend needs in order to function properly. For example, metadata keep track of all of the triggers that have fired during this session, so that the trigger manager can initialize instances for tasks that run multiple times.

Working with Session State

The storage backend provides a number of methods for interacting with trigger tasks and instances:

__getitem__(task_instance)

Returns the task instance with the specified name. For example:

task_instance = trigger_manager.storage['t_importSubject#0']
__iter__(task_instance)

Returns an iterator for the task instances. Order is undefined. For example:

for task_instance in iter(trigger_manager.storage):
  ...
create_instance(task_config, **kwargs)

Creates a new instance of the specified trigger task.

Additional keyword arguments are passed directly to the TaskInstance initializer.

clone_instance(task_instance)

Given a task instance name or TaskInstance object, creates and installs a copy into the trigger session.

Tip

This method is used internally when replaying a failed task.

get_instances_with_unresolved_logs()
Unsurprisingly, returns all task instances with unresolved logs.
get_unresolved_instances()
Returns all task instances with unresolved status.
get_unresolved_tasks()

Returns all trigger tasks that either:

  • Do not have any instances yet, or
  • Have at least one task instance with unresolved status.
instances_of_task(task_config)
Returns all task instances that have been created for the specified trigger task.

Note

The storage backend contains several more methods, but they are intended to be used internally.

Writing Your Own Storage Backend

To create your own storage backend, you only need to define methods to load and save the session data; the base class will take care of everything else for you.

Your backend must extend the triggers.storages.base.BaseTriggerStorage class and implement the following attributes/methods:

name: Text

A unique identifier for your storage backend.

Generally this matches the name of the storage’s entry point in your project’s setup.py file (see below).

_load_from_backend(self)

Given self.uid, loads the corresponding session data from the persistence medium.

This method should return a tuple with three values:

  • Item 0 contains the trigger task configurations.
  • Item 1 contains the task instances.
  • Item 2 contains the session metadata.

Tip

These values do not have to be stored together, as long as the _load_from_backend() method knows how to consolidate them.

_save(self)

Given self.uid, saves the corresponding session data to the persistence medium.

This method should be sure to save the following values:

  • self._configs: Trigger task configurations.
  • self._instances: Trigger task instance.
  • self._metas: Session metadata.

Note the leading underscore on each of these attributes.

Tip

To serialize values for storage, use the self._serialize() method.

For more information and examples, look at the implementation of triggers.storages.cache.CacheStorageBackend.

Registering Your Storage Backend

As with trigger managers, you must register your custom storage backend before it can be used.

To do this, define a triggers.storages entry point in your project’s setup.py file:

from setuptools import setup

setup(
  ...

  entry_points = {
    'triggers.storages': [
      'custom_storage = app.triggers:CustomStorageBackend',
    ],
  },
)

Tip

Any time you make changes to setup.py, you must reinstall your project (e.g., by running pip install -e . again) before the changes will take effect.

Once you’ve registered your trigger storage backend, you can then use it in your application:

from app.triggers import CustomStorageBackend
from triggers import TriggerManager

trigger_manager =\
  TriggerManager(CustomStorageBackend(session_uid))

Important

Make sure that your application always uses the same storage backend (unless you are 110% sure you know what you are doing).