xyzpy.gen.ray_executor
======================

.. py:module:: xyzpy.gen.ray_executor


Attributes
----------

.. autoapisummary::

   xyzpy.gen.ray_executor._unpack_dispatch


Classes
-------

.. autoapisummary::

   xyzpy.gen.ray_executor.RayFuture
   xyzpy.gen.ray_executor.RayExecutor
   xyzpy.gen.ray_executor.RayGPUExecutor


Functions
---------

.. autoapisummary::

   xyzpy.gen.ray_executor.get_ray
   xyzpy.gen.ray_executor._unpack_futures_tuple
   xyzpy.gen.ray_executor._unpack_futures_list
   xyzpy.gen.ray_executor._unpack_futures_dict
   xyzpy.gen.ray_executor._unpack_futures_identity
   xyzpy.gen.ray_executor._unpack_futures
   xyzpy.gen.ray_executor.get_remote_fn
   xyzpy.gen.ray_executor.get_fn_as_remote_object
   xyzpy.gen.ray_executor.get_deploy


Module Contents
---------------

.. py:function:: get_ray()

   Import and return the ``ray`` module (cached).


.. py:class:: RayFuture(obj)

   Basic ``concurrent.futures`` like future wrapping a ray ``ObjectRef``.


   .. py:attribute:: __slots__
      :value: ('_obj', '_cancelled')



   .. py:attribute:: _obj


   .. py:attribute:: _cancelled
      :value: False



   .. py:method:: result(timeout=None)


   .. py:method:: done()


   .. py:method:: cancel()


.. py:function:: _unpack_futures_tuple(x)

.. py:function:: _unpack_futures_list(x)

.. py:function:: _unpack_futures_dict(x)

.. py:function:: _unpack_futures_identity(x)

.. py:data:: _unpack_dispatch

.. py:function:: _unpack_futures(x)

   Allows passing futures by reference - takes e.g. args and kwargs and
   replaces all ``RayFuture`` objects with their underyling ``ObjectRef``
   within all nested tuples, lists and dicts.

   [Subclassing ``ObjectRef`` might avoid needing this.]


.. py:function:: get_remote_fn(fn, **remote_opts)

   Cached retrieval of remote function.


.. py:function:: get_fn_as_remote_object(fn)

   Store ``fn`` in the Ray object store and return an ``ObjectRef``.


.. py:function:: get_deploy(**remote_opts)

   Alternative for 'non-function' callables - e.g. partial
   functions - pass the callable object too.


.. py:class:: RayExecutor(*args, default_remote_opts=None, **kwargs)

   Basic ``concurrent.futures`` like interface using ``ray``.

   Example usage::

       from xyzpy import RayExecutor

       # create a pool that by default requests a single gpu per task
       pool = RayExecutor(
           num_cpus=4,
           num_gpus=4,
           default_remote_opts={"num_gpus": 1},
       )



   .. py:attribute:: default_remote_opts


   .. py:method:: _maybe_inject_remote_opts(remote_opts=None)

      Return the default remote options, possibly overriding some with
      those supplied by a ``submit call``.



   .. py:method:: submit(fn, *args, pure=False, remote_opts=None, **kwargs)

      Remotely run ``fn(*args, **kwargs)``, returning a ``RayFuture``.



   .. py:method:: map(func, *iterables, remote_opts=None)

      Remote map ``func`` over arguments ``iterables``.



   .. py:method:: scatter(data)

      Push ``data`` into the distributed store, returning an ``ObjectRef``
      that can be supplied to ``submit`` calls for example.



   .. py:method:: shutdown()

      Shutdown the parent ray cluster, this ``RayExecutor`` instance
      itself does not need any cleanup.



.. py:class:: RayGPUExecutor(*args, gpus_per_task=1, **kwargs)

   Bases: :py:obj:`RayExecutor`


   A ``RayExecutor`` that by default requests a single gpu per task.