Analyzer¶

class dias.Analyzer(name, write_dir, state_dir)¶

Bases: caput.config.Reader

Base class for all dias analyzers.

All dias analyzers should inherit from this class, with functionality added by over-riding setup, run and/or shutdown. In addition, input parameters may be specified by adding class attributes which are instances of config.Property. These will then be read from the task config file when it is initialized. The class attributes will be overridden with instance attributes with the same name but with the values specified in the config file.

period¶

A time period (e.g. ‘1h’), indicating the schedule for this task

Type:	String

start_time¶

A time (e.g. ‘2018-01-03 17:13:13’) indicating when you want the task to first run. This is mostly used to determine the phase of your task. If this value is in the future, the scheduler won’t run your task until that time arrives. This is optional. If not given, the scheduler will start the task at an arbitrary time within one period of the start of the scheduler.

Type:	String

data_size_max¶

The amount of data this task can write in it’s data directory. dias deletes old files if this is exceeded. This should be a string containing of a number followed by a whitespace and the SI-unit (e.g. ‘1 kB’)

Type:	String

state_size_max¶

the amount of data this task can write in it’s state directory. dias deletes old files if this is exceeded. This should be a string containing of a number followed by a whitespace and the SI-unit (e.g. ‘10 MB’)

Type:	String

__init__()¶

setup()¶

run()¶

finish()¶

Attributes:	data_size_max log_level period start_time state_size_max

Methods

`add_data_metric`(self, name[, description, …])	Add a gauge metric.
`add_task_metric`(self, metric_name[, …])	Add a gauge metric.
`delete_callback`(self, deleted_files)	Tell the analyzer that files have been deleted after the task has run.
`finish`(self)	Shut down the analyzer.
`from_config`(config, \args, \\*kwargs)	Create a new instance with values loaded from config.
`init_logger`(self[, log_level_override])	Set up the logger.
`read_config`(self, config[, compare_keys, …])	Set all properties in this class from the supplied config.
`run`(self)	Run the analyzer.
`setup`(self)	Set up the analyzer.

Construct the analyzer base class.

Parameters:	name (String) – Task name. write_dir (String) – Path to write output data to. state_dir (String) – Path to write state data to.
Attributes:	data_size_max log_level period start_time state_size_max

Methods

`add_data_metric`(self, name[, description, …])	Add a gauge metric.
`add_task_metric`(self, metric_name[, …])	Add a gauge metric.
`delete_callback`(self, deleted_files)	Tell the analyzer that files have been deleted after the task has run.
`finish`(self)	Shut down the analyzer.
`from_config`(config, \args, \\*kwargs)	Create a new instance with values loaded from config.
`init_logger`(self[, log_level_override])	Set up the logger.
`read_config`(self, config[, compare_keys, …])	Set all properties in this class from the supplied config.
`run`(self)	Run the analyzer.
`setup`(self)	Set up the analyzer.

Attributes Summary

`data_size_max`
`log_level`
`period`
`start_time`
`state_size_max`

Methods Summary

`add_data_metric`(self, name[, description, …])	Add a gauge metric.
`add_task_metric`(self, metric_name[, …])	Add a gauge metric.
`delete_callback`(self, deleted_files)	Tell the analyzer that files have been deleted after the task has run.
`finish`(self)	Shut down the analyzer.
`init_logger`(self[, log_level_override])	Set up the logger.
`run`(self)	Run the analyzer.
`setup`(self)	Set up the analyzer.

Attributes Documentation

data_size_max = None

log_level = None¶

period = None

start_time = None

state_size_max = None

Methods Documentation

add_data_metric(self, name, description='', labelnames=[], unit='')¶

Add a gauge metric.

The metric will be exported with the full name dias_data_<task name>_<metric_name>_<unit>. Pass the metric name without the prefix and unit according to prometheus naming conventions (https://prometheus.io/docs/practices/naming/#metric-names). Use a base unit as described here (https://prometheus.io/docs/practices/naming/#base-units).

Parameters:	name (String) – Name of the metric. description (String) – Description of the metric (optional). labelnames (list of Strings) – Names of the labels for the metric (optional). unit (String) – The base unit of the metric (optional)
Returns:	`prometheus_client.Gauge` – The metric object to be kept and updated by the analyzer.

add_task_metric(self, metric_name, description='', labelnames=[], unit='')¶

Add a gauge metric.

The metric will be exported with the full name dias_task_<task name>_<metric_name>_<unit>. Pass the metric name without the prefix and unit according to prometheus naming conventions (https://prometheus.io/docs/practices/naming/#metric-names). Use a base unit as described here (https://prometheus.io/docs/practices/naming/#base-units).

Parameters:	name (String) – Name of the metric. description (String) – Description of the metric (optional). labelnames (list of Strings) – Names of the labels for the metric (optional). unit (String) – The base unit of the metric (optional)
Returns:	`prometheus_client.Gauge` – The metric object to be kept and updated by the analyzer.

delete_callback(self, deleted_files)¶

Tell the analyzer that files have been deleted after the task has run.

Called after run() to inform the analyzer about files that have been deleted from its write_dir due to data size overage. This is called by the dias framework.

Parameters:	deleted_files (list of pathlib.Path objects) – Files that have been deleted.

finish(self)

Shut down the analyzer.

Final clean-up stage of analyzer. Called by the dias framework.

init_logger(self, log_level_override=None)¶

Set up the logger.

Called by Task after reading the config.

Parameters:	log_level_override (String) – If this is passed, it will override the global log level of dias.

run(self)

Run the analyzer.

Main task stage of analyzer. Will be called by the dias framework according to the period set in the task config.

setup(self)

Set up the analyzer.

Initial setup stage of analyzer. Called by the dias framework when dias is set up.