Array memory ownership¶

Every generated binding that returns an array has to answer one question: who owns this memory, and what keeps it alive? This page is the answer, as a rule per layer, with the measurements behind each rule and the history that produced them.

It exists because the question was answered three different ways over about a year, and the first two answers were wrong in ways that took a heap overflow and a 1.5 GB leak to surface. If you are adding a new array-returning shape, read the rules for new shapes at the bottom.

The rules¶

Layer 1 — C. A DSP kernel never allocates an output. Outputs are caller-supplied out-parameters.

Layer 2 — Python. NumPy owns each call's result. Nothing is shared between calls.

Layer 3 — out=. Exists for placement and determinism, not throughput. It is measurably slower on average.

Layer 1 — C: the caller owns every buffer¶

Every generated C kernel takes its output as a pre-allocated pointer:

/* blockwise */
void  comp_steps(comp_state_t *state, const T *in, size_t n, R *out);
/* variable-length */
size_t comp_verb(comp_state_t *state, const T *in, size_t n_in, R *out);
/* records */
size_t comp_verb(comp_state_t *state, rec_t *result, size_t max_results);

The kernel writes and returns a count. It never mallocs something the caller must free. The --perf tier holds to this too: JM_DEFINE_STEPS uses a stack-resident scratch buffer sized at compile time, so the entire SIMD path is allocation-free by construction.

Why it holds. A caller-owned output is the only arrangement where lifetime is not a question. The C caller already knows how long it needs the samples; the kernel cannot know, so it must not decide.

Layer 2 — Python: NumPy owns each result¶

Every array-returning binding allocates its result from NumPy, per call:

npy_intp _adim = (npy_intp)_cap;
PyObject *arr0 = PyArray_SimpleNew(1, &_adim, NPY_COMPLEX64);
R *_d0 = (R *)PyArray_DATA((PyArrayObject *)arr0);
size_t n_out = comp_verb(self->handle, ..., _d0);

No instance buffer, no free-list, no liveness tracking. The returned array owns its memory outright: it survives destroy(), it outlives the object, and two results never alias.

Why not reuse a buffer? Because it makes "is my previous result still valid?" a question the binding has to answer at runtime, and two serious attempts to answer it both failed:

gh-604 is the bill for the second: a 3000-iteration loop that bound its result grew RSS by 1.5 GB (~514 KiB retained per call) and ran 6-8× slower than allocating. Both layers were correct code defending an incorrect premise — that a binding can know whether the caller still holds the previous result.

What this costs¶

Per-call allocation needs the output length before the kernel runs. Where a kernel can return fewer samples than requested, the binding allocates max(max_out(state, n), n) and trims (or, with pass_capacity, exactly max_out(state, n) — see below).

The trim is not a copy. Where the kernel fills the allocation exactly — the generator shape's normal case — a fast path returns the array directly. Where it writes fewer, the array is shrunk in place, which releases the tail.

The thing to watch on a short-writing kernel is memory, not CPU — and the amount at stake is governed entirely by how tight max_out() is. A kernel whose max_out() is a fixed internal cap rather than a function of the input produces a large over-allocation on every call:

max_out() is a sizing contract, nothing else¶

With no instance buffer, max_out() sizes nothing internal. It does two things: bounds the per-call allocation, and validates out=.

Since gh-607, *_max_out() takes the same count the binding is about to pass to the kernel — named to mirror the kernel's own parameter for that method's shape (n_in for an array-arg method, <param>_len for a single-array-param method, n for a generator; an all-scalar-params method has no count to mirror and its max_out() stays zero-arg). A fixed-output kernel takes the parameter and ignores it — (void)n_in; — uniformly, since jm cannot tell from the manifest whether a given block's output is call-independent.

The count default for a void-input method¶

A variable_output method with arg_type = "void" has no input to size from, so the binding gives it a count keyword and seeds it with 1.

Whether that matters depends on how the object produces its output, not on the method's signature. Two shapes look alike in the manifest and want opposite things:

Signature is the wrong predicate here. A void-input pass_capacity method can be either shape, so check what the object actually allocates before adding the key — this is the sweep error to avoid (doppler-dsp/doppler#568, where delay's pre-allocated _ptr_buf meant its accessors never needed it).

For the first shape the right default is the object's natural capacity, which lives in the user's C and cannot be derived from the manifest. Declare it (gh-657):

[[delay.methods]]
name = "ptr"
arg_type = "void"
return_type = "double _Complex"
variable_output = true
pass_capacity = true
count_default = "state->num_taps"

The value is a C expression, evaluated once before argument parsing, and overridden by any count the caller passes. An expression mentioning state gets a local alias for the object's handle. Because it is C rather than a Python literal, the .pyi and the runtime __doc__ both render the default as ....

Without the key the seed stays 1, exactly as before.

It is not a reliable call-independent upper bound even so. A generator's steps(count) writes exactly count samples, which can exceed it. The real contract, without pass_capacity:

n_out <= max(max_out(state, n), n)

0 is an ordinary answer now, not a "the kernel doesn't know" sentinel — a decimator still filling its history productively returns 0 for a short input. Without pass_capacity the binding clamps the allocation to at least what the call needs regardless of what max_out() returns, so a mechanically-migrated return 0; still allocates n and is safe — nothing about max_out() is memory-safety-critical on this path.

With pass_capacity = true, the kernel takes a trailing size_t max_out and the binding trusts the bound exactly: it allocates max_out(state, n) with no clamp, and passes that same capacity to the kernel. The kernel is now the one enforcing the bound — an under-reporting max_out() truncates at the kernel rather than overflowing the allocation. Opt in when a kernel can genuinely bounds-check its own writes; the exact allocation is the entire point of doing so.

Layer 3 — out= is for placement and determinism¶

out= writes into an array you supply. Use it when where the samples land matters — an mmap'd file, a shared-memory segment, a preallocated ring, a buffer another library owns — or when you have a tail-latency budget at large block sizes.

Do not use it expecting throughput. It is slower on average.

Throughput: a fixed cost, always¶

Measured on a generated project, complex64, same kernel both sides:

The overhead is fixed — validation plus building the returned view — not proportional to n. That is why it should be quoted in nanoseconds and never as a percentage: the same ~60 ns is 85% of the call at n=64 and 0.4% at n=65536.

It costs more than the allocation it avoids because a NumPy allocation of a recently-freed block is roughly 130 ns and does not grow with n — the allocator hands back the same warm block.

Determinism: real, but only above a size threshold¶

Per-call latency distribution, 3000 calls at n=65536:

At 64k, out= costs +60 ns at the median and buys 2.6× better p99 and 3× better maximum. That is the allocator occasionally reaching the OS for a large block, removed from the tail.

The threshold matters. At n <= 1024 the same comparison goes the other way — out= measures p99 1,543 ns against 1,172 ns for plain allocation — because the allocator never leaves its free-list at that size, so there is no allocator tail to remove and you are left paying only the fixed overhead.

Use out= for a latency budget at large blocks. It does not improve jitter at small ones.

Alignment: placement and alignment collide¶

The buffers out= exists to serve are exactly the ones prone to misalignment — mmap'd regions, offsets into a shared segment, and any NumPy slice. A misaligned output costs real throughput in vectorised kernels: a measured 16% penalty on an FFT of 4096 against a misaligned out=.

So placement freedom is real but not free:

out= is validated, not coerced¶

An out= buffer must be a writable, C-contiguous ndarray of exactly the output dtype; anything else raises TypeError rather than being converted. Both properties matter for the same reason: the marshal asks NumPy for a contiguous array of the output dtype, and if either is missing NumPy hands back a converted temporary. The kernel then fills the temporary, the temporary is freed, and the call returns a correct-looking result while your array is never touched.

That failure mode is invisible to anyone who only reads the return value, which is why both are hard errors:

big = np.zeros((4, 2), np.float32)
g.steps(x, out=big[:, 0])       # TypeError: out must be a writable,
                                # C-contiguous ndarray of the output dtype

A wrong dtype was the original trigger (gh-581); a strided buffer is the same defect reached from the other side. Note the overlap with the alignment note above — slicing is the common way to arrive at both problems, except a strided slice now raises where a merely misaligned one is quietly slower.

An undersized out= raises ValueError. The requirement is whatever the binding would have allocated for the same call, and it comes from one emitter so the two faces cannot answer differently — an out= that validated more loosely would accept a buffer the internal path considers too small:

So a buffer sized to the request is accepted exactly when the author has said the bound is a per-call one. Where out= demands more than the call could possibly produce, the answer is on the C side — give max_out the count, or assert exact_max_out — not a looser check here, which would only move the silent truncation from the allocation into the caller's own buffer.

Who owns what, by shape¶

Rules for new shapes¶

When you add an array-returning shape to the generator:

1. Name the owner. NumPy, the caller, or the C state. Write it in the shape's comment and add a row to the table above.
2. A borrowed view must pin something. If you return PyArray_SimpleNewFromData over memory you did not allocate, call PyArray_SetBaseObject on whatever keeps that memory alive. A view that pins nothing is a dangling pointer waiting for a del.
3. Never make validity depend on a runtime probe. If the correctness of a returned array depends on the binding guessing what the caller did with the previous one, the design is wrong. This is the specific mistake gh-219 and gh-437 made.
4. Prefer per-call allocation. It is ~130 ns, flat in n. Reuse is an optimisation you must justify with a measurement against the hold case, not the drop case.
5. Trim in place when the array is fresh and unshared (PyArray_DIMS(arr)[0] = n); use a view + SetBaseObject only when the base is the caller's array.

Appendix: allocation cost¶

Why "just allocate" is the default — NumPy allocation, complex64:

The allocator recycles: a freed block of the same size comes straight back, already mapped and warm. This is why per-call allocation costs the same at 1M samples as at 1, and why the reuse buffer's saving was bounded at roughly one allocation — about 130 ns — no matter how large the block.

The retained-result row is the one to understand. Allocation gets 5–11× more expensive when nothing can be recycled — but a program that retains results is paying for that memory because it asked for it. The old reuse buffer paid that cost for memory nobody could reach: retired buffers held until tp_dealloc, on top of a fresh allocation per call.