runtime_memory.rst

.. include:: subst.rst

OpenCL Runtime: Memory
======================

.. currentmodule:: pyopencl

.. class:: MemoryObject

    .. attribute:: info

        Lower case versions of the :class:`mem_info` constants
        may be used as attributes on instances of this class
        to directly query info attributes.

    .. attribute:: hostbuf

    .. method:: get_info(param)

        See :class:`mem_info` for values of *param*.

    .. method:: release()

    .. method:: get_host_array(shape, dtype, order="C")

        Return the memory object's associated host memory
        area as a :class:`numpy.ndarray` of the given *shape*,
        *dtype* and *order*.

    .. automethod:: from_int_ptr
    .. autoattribute:: int_ptr

    |comparable|

Memory Migration
----------------

.. function:: enqueue_migrate_mem_objects(queue, mem_objects, flags=0, wait_for=None)

    :param flags: from :class:`mem_migration_flags`

    .. versionadded:: 2011.2

    Only available with CL 1.2.

Buffer
------

.. class:: Buffer(context, flags, size=0, hostbuf=None)

    Create a :class:`Buffer`.
    See :class:`mem_flags` for values of *flags*.
    If *hostbuf* is specified, *size* defaults to the size of
    the specified buffer if it is passed as zero.

    :class:`Buffer` inherits from :class:`MemoryObject`.

    .. note::

        Python also defines a type of `buffer object
        <https://docs.python.org/3/c-api/buffer.html>`__,
        and PyOpenCL interacts with those, too, as the host-side
        target of :func:`enqueue_copy`. Make sure to always be
        clear on whether a :class:`Buffer` or a Python buffer
        object is needed.

    Note that actual memory allocation in OpenCL may be deferred.
    Buffers are attached to a :class:`Context` and are only
    moved to a device once the buffer is used on that device.
    That is also the point when out-of-memory errors will occur.
    If you'd like to be sure that there's enough memory for
    your allocation, either use :func:`enqueue_migrate_mem_objects`
    (if available) or simply perform a small transfer to the
    buffer. See also :class:`pyopencl.tools.ImmediateAllocator`.

    .. method:: get_sub_region(origin, size, flags=0)

        Only available in OpenCL 1.1 and newer.

    .. method:: __getitem__(slc)

        *slc* is a :class:`slice` object indicating from which byte index range
        a sub-buffer is to be created. The *flags* argument of
        :meth:`get_sub_region` is set to the same flags with which *self* was
        created.


.. function:: enqueue_fill_buffer(queue, mem, pattern, offset, size, wait_for=None)

    :arg mem: the on device :class:`Buffer`
    :arg pattern: a buffer object (likely a :class:`numpy.ndarray`, eg.
        ``np.uint32(0)``). The memory associated with *pattern* can be reused or
        freed once the function completes.
    :arg size: The size in bytes of the region to be filled. Must be a multiple of the
        size of the pattern.
    :arg offset: The location in bytes of the region being filled in *mem*.
        Must be a multiple of the size of the pattern.

    Fills a buffer with the provided pattern

    |std-enqueue-blurb|

    Only available with CL 1.2.

    .. versionadded:: 2011.2

.. _svm:

Shared Virtual Memory (SVM)
---------------------------

Shared virtual memory allows the host and the compute device to share
address space, so that pointers on the host and on the device may have
the same meaning. In addition, it allows the same memory to be accessed
by both the host and the device. *Coarse-grain* SVM requires that
buffers be mapped before being accessed on the host, *fine-grain* SVM
does away with that requirement.

.. warning::

    Compared to :class:`Buffer`\ s, SVM brings with it a new concern: the
    synchronization of memory deallocation. Unlike other objects in OpenCL,
    SVM is represented by a plain (C-language) pointer and thus has no ability for
    reference counting.

    As a result, it is perfectly legal to allocate a :class:`Buffer`, enqueue an
    operation on it, and release the buffer, without worrying about whether the
    operation has completed. The OpenCL implementation will keep the buffer alive
    until the operation has completed. This is *not* the case with SVM: Unless
    otherwise specified, memory deallocation is performed immediately when
    requested, and so SVM will be deallocated whenever the Python
    garbage collector sees fit, even if the operation has not completed,
    immediately leading to undefined behavior (i.e., typically, memory corruption and,
    before too long, a crash).

    Version 2022.2 of PyOpenCL offers substantially improved tools
    for dealing with this. In particular, all means for allocating SVM
    allow specifying a :class:`CommandQueue`, so that deallocation
    is enqueued and performed after previously-enqueued operations
    have completed.

SVM requires OpenCL 2.0.

.. _opaque-svm:

Opaque and "Wrapped-:mod:`numpy`" Styles of Referencing SVM
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

When trying to pass SVM pointers to functionality in :mod:`pyopencl`,
two styles are supported:

- First, the opaque style. This style most closely resembles
  :class:`Buffer`-based allocation available in OpenCL 1.x.
  SVM pointers are held in opaque "handle" objects such as :class:`SVMAllocation`.

- Second, the wrapped-:mod:`numpy` style. In this case, a :class:`numpy.ndarray`
  (or another object implementing the  :c:func:`Python buffer protocol
  <PyObject_GetBuffer>`) serves as the reference to an area of SVM.
  This style permits using memory areas with :mod:`pyopencl`'s SVM
  interfaces even if they were allocated outside of :mod:`pyopencl`.

  Since passing a :class:`numpy.ndarray` (or another type of object obeying the
  buffer interface) already has existing semantics in most settings in
  :mod:`pyopencl` (such as when passing arguments to a kernel or calling
  :func:`enqueue_copy`), there exists a wrapper object, :class:`SVM`, that may
  be "wrapped around" these objects to mark them as SVM.

The commonality between the two styles is that both ultimately implement
the :class:`SVMPointer` interface, which :mod:`pyopencl` uses to obtain
the actual SVM pointer.

Note that it is easily possible to obtain a :class:`numpy.ndarray` view of SVM
areas held in the opaque style, see :attr:`SVMPointer.buf`, permitting
transitions from opaque to wrapped-:mod:`numpy` style. The opposite transition
(from wrapped-:mod:`numpy` to opaque) is not necessarily straightforward,
as it would require "fishing" the opaque SVM handle out of a chain of
:attr:`numpy.ndarray.base` attributes (or similar, depending on
the actual object serving as the main SVM reference).

See :ref:`numpy-svm-helpers` for helper functions that ease setting up the
wrapped-:mod:`numpy` structure.

Wrapped-:mod:`numpy` SVM tends to be a good fit for fine-grain SVM because of
the ease of direct host-side access, but the creation of the nested structure
that makes this possible is associated with a certain amount of cost.

By comparison, opaque SVM access tends to be a good fit for coarse-grain
SVM, because direct host access is not possible without mapping the array
anyway, and it has lower setup cost. It is of course entirely possible to use
opaque SVM access with fine-grain SVM.

.. versionchanged:: 2022.2

   This version adds the opaque style of SVM access.

Using SVM with Arrays
^^^^^^^^^^^^^^^^^^^^^

While all types of SVM can be used as the memory backing
:class:`pyopencl.array.Array` objects, ensuring that new arrays returned
by array operations (e.g. arithmetic) also use SVM is easiest to accomplish
by passing an :class:`~pyopencl.tools.SVMAllocator` (or
:class:`~pyopencl.tools.SVMPool`) as the *allocator* parameter in functions
returning new arrays.

SVM Pointers, Allocations, and Maps
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

.. autoclass:: SVMPointer

.. autoclass:: SVMAllocation

.. autoclass:: SVM

.. autoclass:: SVMMap


.. _numpy-svm-helpers:

Helper functions for :mod:`numpy`-based SVM allocation
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

.. autofunction:: svm_empty
.. autofunction:: svm_empty_like
.. autofunction:: csvm_empty
.. autofunction:: csvm_empty_like
.. autofunction:: fsvm_empty
.. autofunction:: fsvm_empty_like

Operations on SVM
^^^^^^^^^^^^^^^^^

(See also :ref:`mem-transfer`.)

.. autofunction:: enqueue_svm_memfill
.. autofunction:: enqueue_svm_migratemem

Image
-----

.. class:: ImageFormat([channel_order, channel_type])

    .. attribute:: channel_order

        See :class:`channel_order` for possible values.

    .. attribute:: channel_data_type

        See :class:`channel_type` for possible values.

    .. attribute:: channel_count

        .. versionadded:: 0.91.5

    .. attribute:: dtype_size

        .. versionadded:: 0.91.5

    .. attribute:: itemsize

        .. versionadded:: 0.91.5

    .. method:: __repr__

        Returns a :class:`str` representation of the image format.

        .. versionadded:: 0.91

    |comparable|

    .. versionchanged:: 0.91

        Constructor arguments added.

    .. versionchanged:: 2013.2

        :class:`ImageFormat` was made comparable and hashable

.. function:: get_supported_image_formats(context, flags, image_type)

    See :class:`mem_flags` for possible values of *flags*
    and :class:`mem_object_type` for possible values of *image_type*.

.. class:: Image(context, flags, format, shape=None, pitches=None, hostbuf=None, is_array=False, buffer=None)

    See :class:`mem_flags` for values of *flags*.
    *shape* is a 2- or 3-tuple. *format* is an instance of :class:`ImageFormat`.
    *pitches* is a 1-tuple for 2D images and a 2-tuple for 3D images, indicating
    the distance in bytes from one scan line to the next, and from one 2D image
    slice to the next.

    If *hostbuf* is given and *shape* is *None*, then *hostbuf.shape* is
    used as the *shape* parameter.

    :class:`Image` inherits from :class:`MemoryObject`.

    .. note::

        If you want to load images from :class:`numpy.ndarray` instances or read images
        back into them, be aware that OpenCL images expect the *x* dimension to vary
        fastest, whereas in the default (C) order of :mod:`numpy` arrays, the last index
        varies fastest. If your array is arranged in the wrong order in memory,
        there are two possible fixes for this:

        * Convert the array to Fortran (column-major) order using :func:`numpy.asarray`.

        * Pass *ary.T.copy()* to the image creation function.

    .. versionadded:: 0.91

    .. versionchanged:: 2011.2
        Added *is_array* and *buffer*, which are only available on CL 1.2 and newer.

    .. attribute:: info

        Lower case versions of the :class:`mem_info`
        and :class:`image_info` constants
        may be used as attributes on instances of this class
        to directly query info attributes.

    .. attribute:: shape

        Return the value of the *shape* constructor argument as a :class:`tuple`.

    .. method:: get_image_info(param)

        See :class:`image_info` for values of *param*.

    .. method:: release()

    |comparable|

.. function:: image_from_array(ctx, ary, num_channels=None, mode="r", norm_int=False)

    Build a 2D or 3D :class:`Image` from the :class:`numpy.ndarray` *ary*.  If
    *num_channels* is greater than one, the last dimension of *ary* must be
    identical to *num_channels*. *ary* must be in C order. If *num_channels* is
    not given, it defaults to 1 for scalar types and the number of entries
    for :ref:`vector-types`.

    The :class:`ImageFormat` is chosen as the first *num_channels* components
    of "RGBA".

    :param mode: "r" or "w" for read/write

    .. note::

        When reading from the image object, the indices passed to ``read_imagef``
        are in the reverse order from what they would be when accessing *ary* from
        Python.

    If *norm_int* is *True*, then the integer values are normalized to a floating
    point scale of 0..1 when read.

    .. versionadded:: 2011.2

.. function:: enqueue_fill_image(queue, mem, color, origin, region, wait_for=None)

    :arg color: a buffer object (likely a :class:`numpy.ndarray`)

    |std-enqueue-blurb|

    Only available with CL 1.2.

    .. versionadded:: 2011.2

.. _mem-transfer:

Transfers
---------

.. autofunction:: enqueue_copy(queue, dest, src, **kwargs)

.. autofunction:: enqueue_fill(queue, dest, src, **kwargs)

Mapping Memory into Host Address Space
--------------------------------------

.. autoclass:: MemoryMap

.. function:: enqueue_map_buffer(queue, buf, flags, offset, shape, dtype, order="C", strides=None, wait_for=None, is_blocking=True)

    |explain-waitfor|
    *shape*, *dtype*, and *order* have the same meaning
    as in :func:`numpy.empty`.
    See :class:`map_flags` for possible values of *flags*.
    *strides*, if given, overrides *order*.

    :return: a tuple *(array, event)*. *array* is a
        :class:`numpy.ndarray` representing the host side
        of the map. Its *.base* member contains a
        :class:`MemoryMap`.

    .. versionchanged:: 2011.1
        *is_blocking* now defaults to True.

    .. versionchanged:: 2013.1
        *order* now defaults to "C".

    .. versionchanged:: 2013.2
        Added *strides* argument.

    Sample usage::

        mapped_buf = cl.enqueue_map_buffer(queue, buf, ...)
        with mapped_buf.base:
            # work with mapped_buf
            ...

        # memory will be unmapped here

.. function:: enqueue_map_image(queue, buf, flags, origin, region, shape, dtype, order="C", strides=None, wait_for=None, is_blocking=True)

    |explain-waitfor|
    *shape*, *dtype*, and *order* have the same meaning
    as in :func:`numpy.empty`.
    See :class:`map_flags` for possible values of *flags*.
    *strides*, if given, overrides *order*.

    :return: a tuple *(array, event)*. *array* is a
        :class:`numpy.ndarray` representing the host side
        of the map. Its *.base* member contains a
        :class:`MemoryMap`.

    .. versionchanged:: 2011.1
        *is_blocking* now defaults to True.

    .. versionchanged:: 2013.1
        *order* now defaults to "C".

    .. versionchanged:: 2013.2
        Added *strides* argument.

Samplers
--------

.. class:: Sampler

    .. method:: __init__(context, normalized_coords, addressing_mode, filter_mode)

        *normalized_coords* is a :class:`bool` indicating whether
        to use coordinates between 0 and 1 (*True*) or the texture's
        natural pixel size (*False*).
        See :class:`addressing_mode` and :class:`filter_mode` for possible
        argument values.

        Also supports an alternate signature ``(context, properties)``.

        :arg properties: a sequence
            of keys and values from :class:`sampler_properties` as accepted
            by :c:func:`clCreateSamplerWithProperties` (see the OpenCL
            spec for details). The trailing *0* is added automatically
            and does not need to be included.

        This signature Requires OpenCL 2 or newer.

        .. versionchanged:: 2018.2

            The properties-based signature was added.

    .. attribute:: info

        Lower case versions of the :class:`sampler_info` constants
        may be used as attributes on instances of this class
        to directly query info attributes.

    .. method:: get_info(param)

        See :class:`sampler_info` for values of *param*.

    .. automethod:: from_int_ptr
    .. autoattribute:: int_ptr

    |comparable|

Pipes
-----

.. class:: Pipe(context, flags, packet_size, max_packets, properties=())

    See :class:`mem_flags` for values of *flags*.

    :arg properties: a sequence
        of keys and values from :class:`pipe_properties` as accepted
        by :c:func:`clCreatePipe`. The trailing *0* is added automatically
        and does not need to be included.
        (This argument must currently be empty.)

    This function requires OpenCL 2 or newer.

    .. versionadded:: 2020.3

    .. versionchanged:: 2021.1.7

        *properties* now defaults to an empty tuple.

    .. method:: get_pipe_info(param)

        See :class:`pipe_info` for values of *param*.

Type aliases
------------

.. currentmodule:: pyopencl._cl

.. class:: Buffer

   See :class:`pyopencl.Buffer`.