17.1.1.1. AcquisitionStep

class acquisition.AcquisitionStep

Bases: object

Abstract class to describe an acquisition step.

You have to override this class.

Variables:

• debug_mode_allowed (boolean) – if True, the debug mode is allowed.
• stop_flag (boolean) – if True, stop the daemon as soon as possible.
• unit_tests (boolean) – if True, we are in unit tests mode.
• failure_policy (string) – failure policy.
• failure_policy_move_dest_dir (string) – destination directory when failure policy is move
• failure_policy_move_keep_tags (boolean) – keep tags into another file when failure policy is move ?
• failure_policy_move_keep_tags_suffix (string) – suffix to add to the filename to keep tags when failure policy is move
• args (Namespace) – argparser Namespace object with parsed cli args.
• __logger (Logger) – Logger object.
• __last_ping (Datetime) – Datetime object of the last ping() call.

Attributes Summary

args
debug_mode_allowed
failure_policy
failure_policy_move_dest_dir
failure_policy_move_keep_tags
failure_policy_move_keep_tags_suffix
plugin_name	Return the name of the plugin.
step_limit
step_name	Get the name of the step.
stop_flag
unit_tests
unit_tests_args

Methods Summary

add_extra_arguments(parser)	Add some extra argument to commande line parsing.
after(status)	Method called after the process execution.
copy_to_plugin_step(xaf, plugin_name, step_name)	Copy a XattrFile (with tags) to another plugin/step.
critical(msg, *args, **kwargs)	Log a critical message.
debug(msg, *args, **kwargs)	Log a debug message.
destroy()	Destroy what you want just before exiting.
error(msg, *args, **kwargs)	Log an error message.
error_and_die(msg, *args, **kwargs)	Log an error message and exit immediatly.
exception(msg, *args, **kwargs)	Log an error message with current exception stacktrace.
get_original_basename(xaf)	Return the original basename of the file.
get_original_dirname(xaf)	Return the original dirname of the file.
get_original_uid(xaf)	Return the original uid of the file.
get_plugin_directory_path()	Return the plugin directory (fullpath).
get_stats_client([extra_tags])
get_tag(xaf, name[, not_found_value, …])	Read a tag on a file with good prefixes.
get_tmp_filepath()	Get a full temporary filepath (including unique filename).
info(msg, *args, **kwargs)	Log an info message.
init()	Method called after CLI parsing but before processing any files.
move_to_plugin_step(xaf, plugin_name, step_name)	Move a XattrFile to another plugin/step.
ping()	Do something every second if possible.
process(xaf)	Process one file.
run()
set_tag(xaf, name, value[, add_latest])	Set a tag on a file with good prefixes.
warning(msg, *args, **kwargs)	Log a warning message.

Attributes Documentation

args = None

debug_mode_allowed = True

failure_policy = None

failure_policy_move_dest_dir = None

failure_policy_move_keep_tags = True

failure_policy_move_keep_tags_suffix = None

plugin_name

Return the name of the plugin.

This method is called if there is no “step_name” property defined. This said property MUST be defined. The name must match with ^[A-Za-z0-9_]+$ regexp.

Returns:(string) the name of the plugin.

step_limit = 1000

step_name

Get the name of the step.

This method is called if there is no “step_name” property defined. This said property SHOULD be defined. The name must match with ^[A-Za-z0-9_]+$ regexp.

Returns:(string) the name of the step.

stop_flag = False

unit_tests = False

unit_tests_args = None

Methods Documentation

add_extra_arguments(parser)

Add some extra argument to commande line parsing.

If you have to add some, you have to override this method.

Parameters:parser – an ArgumentParser object (with default options added).

after(status)

Method called after the process execution.

It’s called event in case of exceptions during process.

copy_to_plugin_step(xaf, plugin_name, step_name)

Copy a XattrFile (with tags) to another plugin/step.

Parameters:

• xaf (XattrFile) – XattrFile to move
• plugin_name (string) – plugin name
• step_name (string) – step name

Returns:

True if ok

Return type:

boolean

critical(msg, *args, **kwargs)

Log a critical message.

debug(msg, *args, **kwargs)

Log a debug message.

destroy()

Destroy what you want just before exiting.

No file will be processed after calling this method.

error(msg, *args, **kwargs)

Log an error message.

error_and_die(msg, *args, **kwargs)

Log an error message and exit immediatly.

exception(msg, *args, **kwargs)

Log an error message with current exception stacktrace.

This method should only be called from an exception handler.

get_original_basename(xaf)

Return the original basename of the file.

The original basename is the not modified basename before step0 execution. “unknown” is returned if we can’t find the basename.

Parameters:xaf (XattrFile) – file to consider. Returns:original basename. Return type:string

get_original_dirname(xaf)

Return the original dirname of the file.

The original dirname is the not modified basename before step0 execution. “unknown” is returned if we can’t find the dirname.

Parameters:xaf (XattrFile) – file to consider. Returns:original dirname. Return type:string

get_original_uid(xaf)

Return the original uid of the file.

The original uid is the first unique id given before step0 execution. “unknown” is returned if we can’t find the original uid.

Parameters:xaf (XattrFile) – file to consider. Returns:original uid. Return type:string

get_plugin_directory_path()

Return the plugin directory (fullpath).

Returns:(string) the fullpath of the plugin directory.

get_stats_client(extra_tags={})

get_tag(xaf, name, not_found_value=None, counter_str_value='latest', force_step_name=None, force_plugin_name=None)

Read a tag on a file with good prefixes.

Parameters:

• xaf (XattrFile) – file to read.
• name (string) – name of the tag (without prefixes).
• not_found_value – returned value if the tag is not found.
• counter_str_value (string) – counter string value.
• force_step_name – tagger step name (if None, current step name is taken)
• force_plugin_name – tagger plugin name (if None, current plugin name is taken)

get_tmp_filepath()

Get a full temporary filepath (including unique filename).

Returns:(string) full temporary filepath (including unique filename).

info(msg, *args, **kwargs)

Log an info message.

init()

Method called after CLI parsing but before processing any files.

move_to_plugin_step(xaf, plugin_name, step_name)

Move a XattrFile to another plugin/step.

Parameters:

• xaf (XattrFile) – XattrFile to move
• plugin_name (string) – plugin name
• step_name (string) – step name

Returns:

True if ok

Return type:

boolean

ping()

Do something every second if possible.

The call can be blocked by a long running process() call.

process(xaf)

Process one file.

Process one XattrFile. You have to override this method (unless your class inherits from BatchStep, see batch_process() in that case).

The file is moved into a temporary directory before the call with an unique filename. Extended attributes are copied to it. So you can do what you want with it.

If the method returns True:

• we considerer that the processing is ok
• the file is delete if necessary

If the method doesn’t return True:

• we considerer that the processing is not ok (a warning message is

  logged).

• the file is given to the choosen failure policy.

Parameters:xaf – XattrFile object. Returns:processing status (True: ok, False: not ok) Return type:boolean

run()

set_tag(xaf, name, value, add_latest=True)

Set a tag on a file with good prefixes.

Parameters:

• xaf (XattrFile) – file to add/set tag on.
• name (string) – name of the tag (without prefixes)
• value (string) – value of the tag
• add_latest (boolean) – add latest prefix

warning(msg, *args, **kwargs)

Log a warning message.