xyzpy.gen.cropping#

Functions

calc_clean_up_default_res(crop, clean_up, ...)

Logic for choosing whether to automatically clean up a crop, and what, if any, the default all-nan result should be.

check_ready_to_reap(crop, allow_incomplete, wait)

clean_slurm_outputs(job[, directory, ...])

from_pickle(s[, picklelib])

gen_cluster_script(crop, scheduler[, ...])

Generate a cluster script to grow a Crop.

gen_qsub_script(crop[, batch_ids, scheduler])

Generate a qsub script to grow a Crop.

get_picklelib([picklelib])

grow(batch_number[, crop, fn, num_workers, ...])

Automatically process a batch of cases into results.

grow_cluster(crop, scheduler[, batch_ids, ...])

Automagically submit SGE, PBS, or slurm jobs to grow all missing results.

load_crops([directory])

Automatically load all the crops found in the current directory.

manage_slurm_outputs(crop, job[, wait_time])

parse_crop_details(fn, crop_name, crop_parent)

Work out how to structure the sowed data.

parse_fn_farmer(fn, farmer)

qsub_grow(crop[, batch_ids, scheduler])

Automagically submit SGE or PBS jobs to grow all missing results.

read_from_disk(fname)

to_pickle(obj[, picklelib])

write_to_disk(obj, fname)

Classes

Crop(*[, fn, name, parent_dir, save_fn, ...])

Encapsulates all the details describing a single 'crop', that is, its location, name, and batch size/number.

Reaper(crop, num_batches[, wait, default_result])

Class that acts as a stateful function to retrieve already sown and grow results.

Sower(crop)

Class for sowing a 'crop' of batched combos to then 'grow' (on any number of workers sharing the filesystem) and then reap.