progress

class micvbang.progress.ProgressTracker(it, get_id=None, f=None, flush_freq=0, print_skips_freq=0)

Track and continue progress over iterators, saving state in a file-like object.

Arguments:

it(iterable): Iterable used to to make the iterator to track the progress of. get_id(function): Function mapping values generated from :param it: to unique ids of type string.

Note

param get_id:must return a string value that does not contain newlines.

close()

Close ProgressTracker. This ensures that no iterators created from the instance will progress any further.

iter()

Return an iterator that iterates over the given input iterator and automatically tracks its progress. processed() will be called before each value is returned to the user.

Note

Potential off-by-one error here; all ids are marked as processed before they are returned and will therefore never be returned again. If the program crashes and the id was not in fact processed by user code, it will go unprocessed.

iter_ids()

Return an iterator that yields an (id, data)-tuple. In order to mark an iteration as processed, processed() must be called with the given id.

processed(id)

Mark an id as processed.

This means that values with the given id will not be returned when creating iterators using the same progress file.

class micvbang.progress.ReadAppendFile(open_read, open_append)

Create new instance of ReadAppendFile(open_read, open_append)

open_append

Alias for field number 1

open_read

Alias for field number 0