-
Andreas Klöckner authoredAndreas Klöckner authored
Built-in Utilities
Memory Pools
The constructor :func:`pyopencl.Buffer` can consume a fairly large amount of processing time if it is invoked very frequently. For example, code based on :class:`pyopencl.array.Array` can easily run into this issue because a fresh memory area is allocated for each intermediate result. Memory pools are a remedy for this problem based on the observation that often many of the block allocations are of the same sizes as previously used ones.
Then, instead of fully returning the memory to the system and incurring the associated reallocation overhead, the pool holds on to the memory and uses it to satisfy future allocations of similarly-sized blocks. The pool reacts appropriately to out-of-memory conditions as long as all memory allocations are made through it. Allocations performed from outside of the pool may run into spurious out-of-memory conditions due to the pool owning much or all of the available memory.
Using :class:`pyopencl.array.Array` instances with a :class:`MemoryPool` is not complicated:
mem_pool = pyopencl.tools.MemoryPool(pyopencl.tools.ImmediateAllocator(queue))
a_dev = cl_array.arange(queue, 2000, dtype=np.float32, allocator=mem_pool)An object representing a :class:`MemoryPool`-based allocation of device memory. Once this object is deleted, its associated device memory is returned to the pool. This supports the same interface as :class:`pyopencl.Buffer`.
mem_flags takes its values from :class:`pyopencl.mem_flags` and corresponds to the flags argument of :class:`pyopencl.Buffer`. DeferredAllocator has the same semantics as regular OpenCL buffer allocation, i.e. it may promise memory to be available that may (in any call to a buffer-using CL function) turn out to not exist later on. (Allocations in CL are bound to contexts, not devices, and memory availability depends on which device the buffer is used with.)
mem_flags takes its values from :class:`pyopencl.mem_flags` and corresponds to the flags argument of :class:`pyopencl.Buffer`. DeferredAllocator has the same semantics as regular OpenCL buffer allocation, i.e. it may promise memory to be available that later on (in any call to a buffer-using CL function).
A memory pool for OpenCL device memory. allocator must be an instance of one of the above classes, and should be an :class:`ImmediateAllocator`. The memory pool assumes that allocation failures are reported by the allocator immediately, and not in the OpenCL-typical deferred manner.