17. XATTRFILE API

17.1. XattrFile class

class xattrfile.XattrFile(filepath, get_redis_callable=<function metwork_get_redis_callable>, redis_timeout=86400)

File with attributes.

At the beginning, this class was a wrapper around files with POSIX extended attributes (xattr). But because of xattr limitations, we store attributes into a redis instance. The name of this class should be changed one day.

Attributes are stored inside the object. They are lazy loaded from redis.

You can access them through tags attributes as a BytesDictWithDirtyFlag dict. If you made some modifications on theses tags, you can force a redis write with commit() method. But main public methods on the object do it for you. And there is an automatic destructor to do that if necessary.

You should not manipulate corresponding filepath directly (readonly access is ok) to avoid incoherences. Please use public methods on the file to copy/delete/rename/… it.

filepath

name of file.

Type

string

tags

attributes of file (lazy loading).

get_redis_callable

a function to get a connected redis instance.

Type

callable

redis_timeout

redis keys timeout (in seconds)

Type

int

Constructor.

Parameters

• filepath (string) – full file path.

• get_redis_callable (callable) – a function called with no arg which has to return a connected redis.Redis object to a redis instance to read/store attributes.

• redis_timeout (int) – lifetime (in seconds) of redis keys (-1: means no timeout) => it means that you will loose attributes on a given file after this timeout (if no modification).

Raises

IOError – if the given path does not exist or is not a file.

__init__(filepath, get_redis_callable=<function metwork_get_redis_callable>, redis_timeout=86400)

Constructor.

Parameters

• filepath (string) – full file path.

• get_redis_callable (callable) – a function called with no arg which has to return a connected redis.Redis object to a redis instance to read/store attributes.

• redis_timeout (int) – lifetime (in seconds) of redis keys (-1: means no timeout) => it means that you will loose attributes on a given file after this timeout (if no modification).

Raises

IOError – if the given path does not exist or is not a file.

basename()

Get and return the basename of the file.

chmod(mode_int)

Change the mode of the file to the provided numeric mode.

Parameters

mode_int (integer) – mode as integer (not octal !) (int(‘0755’, 8) for example to get the integer value of well known ‘0755’ octal value).

Raises

IOError – can’t do the chmod at a filesystem level.

commit()

Write tags into redis (if they are dirty).

Raises

redis.RedisError – if there is a problem with redis.

copy(new_filepath, tmp_suffix='.t', chmod_mode_int=None)

Copy of the file (and its tags) with temporary suffix.

The temporary suffix is used during the copy to get a kind of atomic operation.

Note: tags are commited to redis during the copy.

Parameters

• new_filepath (string) – filepath to copy on.

• tmp_suffix (string) – temporary suffix during copy (None means no temporary suffix).

• chmod_mode_int (integer) – if set, chmod mode as integer (not octal !) (int(‘0755’, 8) for example to get the integer value of well known ‘0755’ octal value).

Returns

a new Xattrfile corresponding to the copied file.

Raises

• redis.RedisError – if there is a problem with redis.

• IOError – can’t do the copy.

copy_or_nothing(new_filepath, tmp_suffix='.t', chmod_mode_int=None)

Copy a file without raising exceptions.

In case of errors, in contrast to copy() method, no exception is raised.

Parameters

• new_filepath (string) – filepath to copy on.

• tmp_suffix (string) – temporary suffix during copy (None means no temporary suffix).

• chmod_mode_int (integer) – if set, chmod mode as integer (not octal !) (int(‘0755’, 8) for example to get the integer value of well known ‘0755’ octal value).

Returns

True if the copy was ok, False else.

Return type

boolean

copy_tags_on(filepath)

Copy current tags to another file and returns corresponding XattrFile.

The destination filepath must exist. If not, use copy() method.

Note: tags are commited to redis before the copy.

Parameters

filepath (string) – complete filepath to copy tags on.

Returns

Xattrfile corresponding to given filepath

with current tags copied on.

Raises

• redis.RedisError – if there is a problem with redis.

• IOError – if the given path does not exist or is not a file.

delete()

Delete the file and corresponding tags.

Raises

• redis.RedisError – if there is a problem with redis.

• IOError – can’t do the delete at a filesystem level.

delete_or_nothing()

Delete the file and corresponding tags.

In case of errors, in contrast to delete() method, no exception is raised.

Returns

True if the delete was ok, False else.

Return type

boolean

dirname()

Get and return the dirname of the file.

dump_tags_on_logger(logger, lvl)

Dump tags on the given logger with the given log level.

getsize()

Return the size of the file (in bytes).

Returns

int: the size of the file (in bytes) or None in case of problems.

getuid()

Return the uid of the file.

Returns

the uid of the file or None in case of problems.

Return type

int

hard_link(new_filepath, tmp_suffix='.t')

Create a hard link of the file (and its tags).

The temporary suffix is used during the hardlink to get a kind of atomic operation.

Note: tags are commited to redis during the hard link.

Parameters

• new_filepath (string) – filepath for the hard link.

• tmp_suffix (string) – temporary suffix during copy (None means no temporary suffix).

Returns

a new Xattrfile corresponding to the new file/link.

Raises

• redis.RedisError – if there is a problem with redis.

• IOError – can’t do the link at a filesystem level.

hardlink_or_copy(new_filepath, tmp_suffix='.t', chmod_mode_int=None)

Hardlink or copy (only if move failed) without raising exceptions.

The original file (and its tags) is keeped intact and the current object is not modified.

Parameters

• new_filepath (string) – complete new filepath towards harlink/copy.

• tmp_suffix (string) – temporary suffix during copy (None means no temporary suffix).

• chmod_mode_int (integer) – DEPRECATED (do not use).

Returns

first boolean is True if the operation was ok,

False else ; second boolean is True if the operation was done with a hardlink, False if the operation was done with a copy.

Return type

(boolean, boolean)

move_or_copy(new_filepath, tmp_suffix='.t', chmod_mode_int=None)

Move or copy (only if move failed) without any exceptions.

The original file (and its tags) is deleted (whatever move or copy is effectively done) and the current object is renamed to new filepath.

Parameters

• new_filepath (string) – complete new filepath towards move/copy.

• tmp_suffix (string) – temporary suffix during copy (None means no temporary suffix).

• chmod_mode_int (integer) – DEPRECATED (do not use).

Returns

first boolean is True if the operation was ok,

False else ; second boolean is True if the operation was done with a move, False if the operation was done with a copy.

Return type

(boolean, boolean)

rename(new_filepath)

Move file (and its tags) to another path (in the same filesystem).

Tags are preserved and written before the operation.

Parameters

new_filepath (string) – new filepath.

Raises

• redis.RedisError – if there is a problem with redis.

• IOError – can’t do the rename at a filesystem level.

write_tags_in_a_file(filepath)

Write tags in a utf8 file.

Parameters

filepath – filepath of the file to write in.

17.2. Misc

class xattrfile.BytesDictWithDirtyFlag

Dictionnary with modification (dirty) flag for bytes keys/values.

This class overrides DictWithDirtyFlag class. It adds checks and conversions to be sure that both keys and values are bytes strings.

Example (in python3):

>>> d = BytesDictWithDirtyFlag()
>>> d['foo'] = 'bar'
>>> d.dirty
True
>>> d['foo']
b'bar'
>>> d[b'foo']
b'bar'

__init__()

Initialize self. See help(type(self)) for accurate signature.

get(key, default=None)

Return the value for key if key is in the dictionary, else default.