xyzpy.gen.combo_runner
xyzpy.gen.combo_runner#
Functions for systematically evaluating a function over all combinations.
Functions

Take a function 



Evaluate a function over all cases and combinations and output to a 

Return the first element from the ndimnested list x. 

Take a nested sequence and find its shape as if it were an array. 

Concatenate a nested list of xarray objects along several dimensions. 

Take a single result of a function evaluation and calculate the same sequence of scalars or arrays but filled entirely with 

Convert the output of combo_runner into a 

Convert the output of combo_runner into a 
 xyzpy.gen.combo_runner.combo_runner(fn, combos=None, *, cases=None, constants=None, split=False, flat=False, shuffle=False, parallel=False, executor=None, num_workers=None, verbosity=1)[source]#
Take a function
fn
and compute it over all combinations of named variables values, optionally showing progress and in parallel. Parameters
fn (callable) – Function to analyse.
combos (dict_like[str, iterable]) – All combinations of each argument to values mapping will be computed. Each argument range thus gets a dimension in the output array(s).
cases (sequence of mappings, optional) – Optional list of specific configurations. If both
combos
andcases
are given, then the function is computed for all subcombinations incombos
for each case incases
, arguments can thus only appear in one or the other. Note that missing combinations of arguments will be represented bynan
if creating a nested array.constants (dict, optional) – Constant function arguments. Unlike
combos
andcases
, these won’t produce dimensions in the output result whenflat=False
.split (bool, optional) – Whether to split (unzip) the outputs of
fn
into multiple output arrays or not.flat (bool, optional) – Whether to return a flat list of results or to return a nested tuple suitable to be supplied to
numpy.array
.shuffle (bool or int, optional) – If given, compute the results in a random order (using
random.seed
andrandom.shuffle
), which can be helpful for distributing resources when not all cases are computationally equal.parallel (bool, optional) – Process combos in parallel, default number of workers picked.
executor (executorlike pool, optional) – Submit all combos to this pool executor. Must have
submit
orapply_async
methods and API matching eitherconcurrent.futures
or anipyparallel
view. Pools frommultiprocessing.pool
are also supported.num_workers (int, optional) – Explicitly choose how many workers to use, None for automatic.
verbosity ({0, 1, 2}, optional) –
How much information to display:
0: nothing,
1: just progress,
2: all information.
 Returns
data – Nested tuple containing all combinations of running
fn
ifflat == False
else a flat list of results. Return type
nested tuple
Examples
>>> def fn(a, b, c, d): ... return str(a) + str(b) + str(c) + str(d)
Run all possible combos:
>>> xyz.combo_runner( ... fn, ... combos={ ... 'a': [1, 2], ... 'b': [3, 4], ... 'c': [5, 6], ... 'd': [7, 8], ... }, ... ) 100%########## 16/16 [00:00<00:00, 84733.41it/s] (((('1357', '1358'), ('1367', '1368')), (('1457', '1458'), ('1467', '1468'))), ((('2357', '2358'), ('2367', '2368')), (('2457', '2458'), ('2467', '2468'))))
Run only a selection of cases:
>>> xyz.combo_runner( ... fn, ... cases=[ ... {'a': 1, 'b': 3, 'c': 5, 'd': 7}, ... {'a': 2, 'b': 4, 'c': 6, 'd': 8}, ... ], ... ) 100%########## 2/2 [00:00<00:00, 31418.01it/s] (((('1357', nan), (nan, nan)), ((nan, nan), (nan, nan))), (((nan, nan), (nan, nan)), ((nan, nan), (nan, '2468'))))
Run only certain cases of some args, but all combinations of others:
>>> xyz.combo_runner( ... fn, ... cases=[ ... {'a': 1, 'b': 3}, ... {'a': 2, 'b': 4}, ... ], ... combos={ ... 'c': [3, 4], ... 'd': [4, 5], ... }, ... ) 100%########## 8/8 [00:00<00:00, 92691.80it/s] (((('1334', '1335'), ('1344', '1345')), ((nan, nan), (nan, nan))), (((nan, nan), (nan, nan)), (('2434', '2435'), ('2444', '2445'))))
 xyzpy.gen.combo_runner.combo_runner_to_df(fn, combos, var_names, *, var_dims=None, var_coords=None, cases=None, constants=None, resources=None, attrs=None, shuffle=False, parse=True, to_df=True, parallel=False, num_workers=None, executor=None, verbosity=1)#
Evaluate a function over all cases and combinations and output to a
xarray.Dataset
. Parameters
fn (callable) – Function to evaluate.
combos (dict_like[str, iterable]) – Mapping of each individual function argument to sequence of values.
var_names (str, sequence of strings, or None) – Variable name(s) of the output(s) of fn, set to None if fn outputs data already labelled in a Dataset or DataArray.
var_dims (sequence of either strings or string sequences, optional) – ‘Internal’ names of dimensions for each variable, the values for each dimension should be contained as a mapping in either var_coords (not needed by fn) or constants (needed by fn).
var_coords (mapping, optional) – Mapping of extra coords the output variables may depend on.
cases (sequence of dicts, optional) – Individual cases to run for some or all function arguments.
constants (mapping, optional) – Arguments to fn which are not iterated over, these will be recorded either as attributes or coordinates if they are named in var_dims.
resources (mapping, optional) – Like constants but they will not be recorded.
attrs (mapping, optional) – Any extra attributes to store.
parallel (bool, optional) – Process combos in parallel, default number of workers picked.
executor (executorlike pool, optional) – Submit all combos to this pool executor. Must have
submit
orapply_async
methods and API matching eitherconcurrent.futures
or anipyparallel
view. Pools frommultiprocessing.pool
are also supported.num_workers (int, optional) – Explicitly choose how many workers to use, None for automatic.
verbosity ({0, 1, 2}, optional) –
How much information to display:
0: nothing,
1: just progress,
2: all information.
 Returns
ds – Multidimensional labelled dataset contatining all the results if
to_df=False
(the default), else a pandas dataframe with results as labelled rows. Return type
xarray.Dataset or pandas.DataFrame
 xyzpy.gen.combo_runner.combo_runner_to_ds(fn, combos, var_names, *, var_dims=None, var_coords=None, cases=None, constants=None, resources=None, attrs=None, shuffle=False, parse=True, to_df=False, parallel=False, num_workers=None, executor=None, verbosity=1)[source]#
Evaluate a function over all cases and combinations and output to a
xarray.Dataset
. Parameters
fn (callable) – Function to evaluate.
combos (dict_like[str, iterable]) – Mapping of each individual function argument to sequence of values.
var_names (str, sequence of strings, or None) – Variable name(s) of the output(s) of fn, set to None if fn outputs data already labelled in a Dataset or DataArray.
var_dims (sequence of either strings or string sequences, optional) – ‘Internal’ names of dimensions for each variable, the values for each dimension should be contained as a mapping in either var_coords (not needed by fn) or constants (needed by fn).
var_coords (mapping, optional) – Mapping of extra coords the output variables may depend on.
cases (sequence of dicts, optional) – Individual cases to run for some or all function arguments.
constants (mapping, optional) – Arguments to fn which are not iterated over, these will be recorded either as attributes or coordinates if they are named in var_dims.
resources (mapping, optional) – Like constants but they will not be recorded.
attrs (mapping, optional) – Any extra attributes to store.
parallel (bool, optional) – Process combos in parallel, default number of workers picked.
executor (executorlike pool, optional) – Submit all combos to this pool executor. Must have
submit
orapply_async
methods and API matching eitherconcurrent.futures
or anipyparallel
view. Pools frommultiprocessing.pool
are also supported.num_workers (int, optional) – Explicitly choose how many workers to use, None for automatic.
verbosity ({0, 1, 2}, optional) –
How much information to display:
0: nothing,
1: just progress,
2: all information.
 Returns
ds – Multidimensional labelled dataset contatining all the results if
to_df=False
(the default), else a pandas dataframe with results as labelled rows. Return type
xarray.Dataset or pandas.DataFrame
 xyzpy.gen.combo_runner.get_ndim_first(x, ndim)[source]#
Return the first element from the ndimnested list x.
 xyzpy.gen.combo_runner.infer_shape(x)[source]#
Take a nested sequence and find its shape as if it were an array.
Examples
>>> x = [[10, 20, 30], [40, 50, 60]] >>> infer_shape(x) (2, 3)
 xyzpy.gen.combo_runner.multi_concat(results, dims)[source]#
Concatenate a nested list of xarray objects along several dimensions.
 xyzpy.gen.combo_runner.nan_like_result(res)[source]#
Take a single result of a function evaluation and calculate the same sequence of scalars or arrays but filled entirely with
nan
.Examples
>>> res = (True, [[10, 20, 30], [40, 50, 60]], 42.0, 'hello') >>> nan_like_result(res) (array(nan), array([[nan, nan, nan], [nan, nan, nan]]), array(nan), None)
 xyzpy.gen.combo_runner.results_to_df(results_linear, settings, attrs, resources, var_names)[source]#
Convert the output of combo_runner into a
pandas.DataFrame
.
 xyzpy.gen.combo_runner.results_to_ds(results, combos, var_names, var_dims, var_coords, constants=None, attrs=None)[source]#
Convert the output of combo_runner into a
xarray.Dataset
.