shepherd.common package

Submodules

shepherd.common.exceptions module

exception shepherd.common.exceptions.ConfigError(message, error=None, logger=None)

Bases: shepherd.common.exceptions.LoggingException

exception shepherd.common.exceptions.LoggingException(message, logger)

Bases: exceptions.Exception

exception shepherd.common.exceptions.ManifestError(message, errors=None, logger=None)

Bases: shepherd.common.exceptions.LoggingException

exception shepherd.common.exceptions.PluginError(message, errors=None, logger=None)

Bases: shepherd.common.exceptions.LoggingException

exception shepherd.common.exceptions.StackError(message, errors=None, logger=None)

Bases: shepherd.common.exceptions.LoggingException

shepherd.common.plugins module

Uses abstract base classes and IPlugin to create base plugin classes. Each defines a particular interface For example all resources should have a create, destroy and exists methods.

This combined with the PluginManager in shepherd.manager helps to creates a common method of building the modular architecture. Since, the built in resources, parsers, etc all use the same abstract classes, adding new features should be seemless.

class shepherd.common.plugins.Action

Bases: yapsy.IPlugin.IPlugin

Defines an action to run.

WARNING: Actions may be depricated as objects in future versions

if yapsy is removed as the plugin manager.

run(config, **kwargs)

Takes an undefined set of name arguments.

Args:

config (Config): the config object for locating plugins, settings, etc. kwargs (dict): named parameters that should passed to the action

class shepherd.common.plugins.Parser

Bases: yapsy.IPlugin.IPlugin

During parsing of manifests Parsers can be used to handles any modifications to the params subdict before merging the params into the resources.

This could be used for dynamic variables like getting the latest volume snapshots, external resources, etc.

NOTE: multiple parsers can be run on the paramsdict, so ensure that they are independent of each other. For example you may want a parser for modifying each dynamic variable.

run(paramsdict, config=None)

Takes a dict of the parameters and returns the modified version. It optionally takes a config which could contain default input values to use.

Args:

paramsdict (dict): the current params dict. config (Config, optional): the config object if the Parser would like to

class shepherd.common.plugins.Resource(provider)

Bases: yapsy.IPlugin.IPlugin

Defines a simple interface for interacting with resource objects.

available

create()

All resources must have a create method.

deserialize(data)

Deserializes the keys and values in a dictionary into attributes for self.

Notes:

The mapping of keys to attributes is done using self._attributes_map.

Args:

data (dict): a dictionary of the attributes to deserialize.

destroy()

All resources must have a destroy method.

get_dependencies()

Generates a list of dependencies for the resource.

Returns:

list: of other resource names this resource depends on.

global_name

local_name

provider

serialize()

Serializes the attributes to a dict using self._attributes_map.

Returns:

dict: the serialized dictionary of the attributes.

stack

tags

type

classmethod validate_create()

A default Resource decorator to remove boilerplate validation code for create requests.

classmethod validate_destroy()

A default Resource decorator to remove boilerplate validation code for destroy requests.

class shepherd.common.plugins.Storage

Bases: yapsy.IPlugin.IPlugin

Defines an API for storage plugins.

WARNING: Storage plugins should be thread safe. For example, if your storage plugin writes to a file make sure you’re locking and unlocking the file accordingly.

FEATURE: should probably support some kind of archiving approach for old and unused stacks.

dump(stack)

Takes a stack dict and stores it in the datastore of your choice.

Args:

stack (Stack): dumps the stack into the storage media

load(name)

Given a unique name.

Search the store for the serialized stack with that name. Returns a single stack dict.

Args:

name (str): the global_name of the stack to load.

search(tags)

Given a dict of tags.

Search the store for serialized stacks that match to those tags. Returning a list of the stack names that match.

Args:

tags (dict): tags to use when search for stacks.

shepherd.common.plugins.is_plugin(cls)

Accepts a class and returns a boolean as to whether the class is a valid plugin.

Kind of a hack, but it dynamically scans the plugins file for all valid shepherd plugin abstract base classes.

Args:

cls (class): the class that may be a plugin.

shepherd.common.utils module

Provides various utility functions for shepherd ranging from validating configs to mapping object attributes to and from dictionaries.

shepherd.common.utils.configure_logging(verbosity, logformat='[%(levelname)s %(asctime)s %(name)s] - "%(message)s"')

Sets up logging for the framework.

All default logging uses logging.basicConfig with the format string ‘[%(levelname)s %(asctime)s %(name)s] - “%(message)s”’

Args:

verbosity (int): a verbosity level between 0 and 5.

0. No Logging Configured
1. Root & shepherd logger = WARNING
2. Root logger = WARNING; shepherd logger = INFO
3. Root logger = WARNING; shepherd logger = DEBUG
4. Root logger = INFO; shepherd logger = DEBUG
5. Root logger = DEBUG; shepherd logger = DEBUG

logformat (str, optional): an alternative format string.

shepherd.common.utils.dict_contains(superdict, subdict)

Returns a boolean as to whether the superdict contains the same key value pairs of the subdict.

Args:

superdict (dict): dict with superset of keys in subdict subdict (dict): dict with subset of keys from superdict

Returns:

result (bool):

shepherd.common.utils.get_logger(obj)

Provides an alternative method of getting a useful logger name for an object because yapsy tends to mess up how __name__ works.

Args:

obj (object): the object to build a custom logger name for.

Returns:

logger (Logger) - where the name is the __module__.__name__ of type(obj).

shepherd.common.utils.getattrs(obj, attrmap)

Handles extracting object attributes into a dict.

Parameters:attrmap – Returns:the result dict.

Args:

obj (object): the object to extract the attributes from. attrmap (dict): a dict mapping dict keys to attribute names, where the

keys match the keys of the resulting dict and the values match their corresponding attributes.

Returns:

result (dict): dictionary of the mapped attributes from obj

shepherd.common.utils.pascal_to_underscore(pascal_str)

Converts pascal format strings to underscore format.

http://stackoverflow.com/questions/1175208/elegant-python-function-to-convert-camelcase-to-camel-case

Args:

pascal_str (TYPE): the pascal case string we want to convert to underscore format.

shepherd.common.utils.run(action_name, config, **kwargs)

Searches for the action plugin to run. Searches both the default paths as well as

Args:

action_name (TYPE): the name of the action you want to run. config (TYPE): the config object used for creating or location a stack to work on. **kwargs: a dictionary of parameters to be passed to the task.

Returns:

the action output

shepherd.common.utils.setattrs(obj, attrmap, values)

Handles setting object attributes from dict.

Args:

obj (object): the object with the attributes being set. attrmap (dict): a dict mapping dict keys to attribute names, where the

keys match the expected keys in values (underscore case) and the values are the attribute names.

values (dict): the dict whose values are mapping to the object.

Returns:

obj (object): the updated object

shepherd.common.utils.tasks_passed(results, logger, msg=None, exception=None)

Logs a warning msg and returns a bool if results contains any failures.

Args:

results (namedtuple): namedtuple containing the sets of failed

and completed tasks returned by Arbiter

logger (Logger): The logger to log to if any tasks failed msg (str, optional): A msg to log if any tasks failed. exception (Exception, optional): An Exception object to throw if

supplied.

Returns:

resp (bool): Whether all tasks in results completed.

shepherd.common.utils.validate_config(config)

Validates the config format with the schema file.

Args:

config (dict): a dictionary of the config settings.

Module contents