xyzpy.gen.case_runner ===================== .. py:module:: xyzpy.gen.case_runner .. autoapi-nested-parse:: Functions for systematically evaluating a function over specific cases. Attributes ---------- .. autoapisummary:: xyzpy.gen.case_runner.case_runner_to_df Functions --------- .. autoapisummary:: xyzpy.gen.case_runner.case_runner xyzpy.gen.case_runner.case_runner_to_ds xyzpy.gen.case_runner.is_case_missing xyzpy.gen.case_runner.parse_into_cases xyzpy.gen.case_runner.find_missing_cases Module Contents --------------- .. py:function:: case_runner(fn, fn_args, cases, combos=None, constants=None, split=False, shuffle=False, parse=True, parallel=False, executor=None, num_workers=None, verbosity=1) Simple case runner that outputs the raw tuple of results. :param fn: Function with which to evalute cases with :type fn: callable :param fn_args: Names of case arguments that fn takes, can be ``None`` if each case is a ``dict``. :type fn_args: tuple :param cases: List of specific configurations that ``fn_args`` should take. If ``fn_args`` is ``None``, each case should be a ``dict``. :type cases: iterable[tuple] or iterable[dict] :param combos: Optional specification of sub-combinations. :type combos: dict_like[str, iterable], optional :param constants: Constant function arguments. :type constants: dict, optional :param split: See :func:`~xyzpy.combo_runner`. :type split: bool, optional :param shuffle: If given, compute the results in a random order (using ``random.seed`` and ``random.shuffle``), which can be helpful for distributing resources when not all cases are computationally equal. :type shuffle: bool or int, optional :param parallel: Process combos in parallel, default number of workers picked. :type parallel: bool, optional :param executor: Submit all combos to this pool executor. Must have ``submit`` or ``apply_async`` methods and API matching either ``concurrent.futures`` or an ``ipyparallel`` view. Pools from ``multiprocessing.pool`` are also supported. :type executor: executor-like pool, optional :param num_workers: Explicitly choose how many workers to use, None for automatic. :type num_workers: int, optional :param verbosity: How much information to display: - 0: nothing, - 1: just progress, - 2: all information. :type verbosity: {0, 1, 2}, optional :returns: **results** :rtype: list of fn output for each case .. py:function:: case_runner_to_ds(fn, fn_args, cases, var_names, var_dims=None, var_coords=None, combos=None, constants=None, resources=None, attrs=None, shuffle=False, to_df=False, parse=True, parallel=False, num_workers=None, executor=None, verbosity=1) Takes a list of ``cases`` to run ``fn`` over, possibly in parallel, and outputs a :class:`xarray.Dataset`. :param fn: Function to evaluate. :type fn: callable :param fn_args: Names and order of arguments to ``fn``, can be ``None`` if ``cases`` are supplied as dicts. :type fn_args: str or iterable[str] :param cases: List of configurations used to generate results. :type cases: iterable[tuple] or iterable[dict] :param var_names: Variable name(s) of the output(s) of ``fn``. :type var_names: str or iterable of str :param var_dims: 'Internal' names of dimensions for each variable, the values for each dimension should be contained as a mapping in either `var_coords` (not needed by `fn`) or `constants` (needed by `fn`). :type var_dims: sequence of either strings or string sequences, optional :param var_coords: Mapping of extra coords the output variables may depend on. :type var_coords: mapping, optional :param combos: If specified, run all combinations of some arguments in these mappings. :type combos: dict_like[str, iterable], optional :param constants: Arguments to `fn` which are not iterated over, these will be recorded either as attributes or coordinates if they are named in `var_dims`. :type constants: mapping, optional :param resources: Like `constants` but they will not be recorded. :type resources: mapping, optional :param attrs: Any extra attributes to store. :type attrs: mapping, optional :param shuffle: If given, compute the results in a random order (using ``random.seed`` and ``random.shuffle``), which can be helpful for distributing resources when not all cases are computationally equal. :type shuffle: bool or int, optional :param parse: Whether to perform parsing of the inputs arguments. :type parse: bool, optional :param parallel: Process combos in parallel, default number of workers picked. :type parallel: bool, optional :param executor: Submit all combos to this pool executor. Must have ``submit`` or ``apply_async`` methods and API matching either ``concurrent.futures`` or an ``ipyparallel`` view. Pools from ``multiprocessing.pool`` are also supported. :type executor: executor-like pool, optional :param num_workers: Explicitly choose how many workers to use, None for automatic. :type num_workers: int, optional :param verbosity: How much information to display: - 0: nothing, - 1: just progress, - 2: all information. :type verbosity: {0, 1, 2}, optional :returns: **ds** -- Dataset with minimal covering coordinates and all cases evaluated. :rtype: xarray.Dataset .. py:data:: case_runner_to_df .. py:function:: is_case_missing(ds, setting, method='isnull') Does the dataset or dataarray ``ds`` not contain any non-null data for single location ``setting``? Note that this only returns true if *all* data across *all* variables is completely missing at the location. :param ds: Dataset or dataarray to look in. :type ds: xarray.Dataset or xarray.DataArray :param setting: Coordinate location to check. :type setting: dict[str, hashable] :returns: **missing** :rtype: bool .. py:function:: parse_into_cases(combos=None, cases=None, ds=None, method='isnull') Convert maybe ``combos`` and maybe ``cases`` to a single list of ``cases`` only, also optionally filtering based on whether any data at each location is already present in Dataset or DataArray ``ds``. Note that this only checks whether *all* data across *all* variables is completely missing at the location. To check against a single variable only simply supply a DataArray instead of a Dataset, e.g. ``ds=ds["var_name"]``. :param combos: Parameter combinations. :type combos: dict_like[str, iterable], optional :param cases: Parameter configurations. :type cases: iterable[dict], optional :param ds: Dataset or DataArray in which to check for existing data. :type ds: xarray.Dataset or xarray.DataArray, optional :param method: How to determine whether data is missing when ``ds`` is supplied. "isnull" checks for null/nan values, while "isfinite" checks for all non-finite values (i.e. inf or nan). :type method: {"isnull", "isfinite"}, optional :returns: **new_cases** -- The combined and possibly filtered list of cases. :rtype: iterable[dict] .. py:function:: find_missing_cases(ds, ignore_dims=None, method='isnull') Find all cases in a dataset or DataArray with missing data. :param ds: Dataset or DataArray in which to find missing data :type ds: xarray.Dataset or xarray.DataArray :param ignore_dims: Internal variable dimensions (i.e. to ignore). By default (None) this is set to any dimensions that don't appear on all variables. :type ignore_dims: set, optional :returns: **cases_missing** -- List of cases with missing data, where each case is a dict mapping from dimension name to coordinate value. :rtype: iterable[dict]