# ‘Farming’ with Harvester Example#

This examples demonstrates ‘farming’ runs, that is, using a Harvester to perform and combine succesive, disjoint sets of runs.

First we define a simple trig function to explore and label with a Runner:

%config InlineBackend.figure_formats = ['svg']
import xyzpy as xyz
import numpy as np

def trig(f, x, amp, phi):

trig_fn = getattr(np, f)
err = 0.2 + 0.1 * np.random.randn()

return amp * trig_fn(x - phi), err

r = xyz.Runner(trig, ['f(x)', 'ef(x)'])


The harvester is composed of this Runner and logic for merging for saving each new set of runs.

h = xyz.Harvester(r, data_name='trig.h5')


We perform one set of runs first:

combos_1 = {
'f': ['sin', 'cos'],
'x': np.linspace(-5, 5, 101),
'amp': [1, 2, 3],
'phi': [0.4, 0.8],
}

h.harvest_combos(combos_1)

100%|##########| 1212/1212 [00:00<00:00, 548026.78it/s]


Which we can plot:

fig, axs = h.full_ds.xyz.plot(
x='x',
y='f(x)',
color='f',
col='amp',
row='phi',
err='ef(x)',
)


Then we can define and run a completely disjoint set of runs:

combos_2 = {
'f': ['sin', 'cos'],
'x': np.linspace(-5, 5, 101),
'amp': [4, 5],
'phi': [1.2, 1.6],
}

h.harvest_combos(combos_2)

100%|##########| 808/808 [00:00<00:00, 461464.82it/s]


If we plot the current full_ds we’ll see both sets of runs are present, but there are also blank spaces:

fig, axs = h.full_ds.xyz.plot(
x='x',
y='f(x)',
color='f',
col='amp',
row='phi',
err='ef(x)',
)


Finally let’s add tan into the mix, but only for the central combinations:

combos_3 = {
'f': ['tan'],
'x': np.linspace(-5, 5, 101),
'amp': [2, 3, 4],
'phi': [0.8, 1.2],
}

h.harvest_combos(combos_3)

100%|##########| 606/606 [00:00<00:00, 84516.47it/s]


We’ll specify some plotting limits to avoid the infinities as well:

fig, axs = h.full_ds.xyz.plot(
x='x',
y='f(x)',
color='f',
col='amp',
row='phi',
err='ef(x)',
ylim=(-4, 4),
)


All runs have also been accumulated into the on-disk:

!ls *.h5

trig.h5

xyz.load_ds('trig.h5')

<xarray.Dataset>
Dimensions:  (f: 3, x: 101, amp: 5, phi: 4)
Coordinates:
* f        (f) <U3 'cos' 'sin' 'tan'
* x        (x) float64 -5.0 -4.9 -4.8 -4.7 -4.6 -4.5 ... 4.6 4.7 4.8 4.9 5.0
* amp      (amp) int64 1 2 3 4 5
* phi      (phi) float64 0.4 0.8 1.2 1.6
Data variables:
f(x)     (f, x, amp, phi) float64 0.6347 0.8855 nan nan ... nan nan nan nan
ef(x)    (f, x, amp, phi) float64 0.1428 0.1683 nan nan ... nan nan nan nan

For now just clean this up.

h.delete_ds()