composer_seams example¶

A kind = "composer" module — the third and largest of the object-of-objects generators — and specifically the seams where it hands work back to you as plain C (gh-998).

A composer emits its CPython types into one .so: a source (Clip), a segment (Track) and the composer itself (Mix), plus a timeline type when [module.X.timeline] type_name is declared, which this example does not. All of that binding is jm's. The seams are not, and they are the interesting part:

All of them are straight C with no CPython in them. jm knows their signatures exactly, so it publishes them in a generated header — native/inc/<pkg>/<mod>/<mod>_bridge.h — and the binding includes that rather than re-declaring them. Before gh-998 they were extern lines buried inside the generated _ext.c, so a C test or benchmark could reach a signature jm owns only by writing a second copy of it. The same header also carries a bit_pattern field's coerce_str_fn (gh-1709) and an owned-pointer field's copy_fn / free_fn / parse_fn / format_fn (gh-1711); this example uses only the three above.

That header is the one file this example is really about, and step 8 uses it the way it is meant to be used: from a translation unit that includes nothing else.

A composer is also manifest-only — there is no jm composer command. You write the table and run just-makeit apply.

TL;DR — see it work first¶

. <(curl -fsSL https://just-buildit.github.io/just-makeit/install.sh)
just-makeit example composer_seams
# composer_seams: PASSED

Prerequisites¶

. <(curl -fsSL https://just-buildit.github.io/just-makeit/install.sh)

Pass a custom path to keep the venv somewhere persistent:

. <(curl -fsSL https://just-buildit.github.io/just-makeit/install.sh) -- ~/my-venv

Or with pip, which installs just-makeit and then builds the toolchain venv at /tmp/jm-venv:

pip install just-makeit && just-makeit install-deps
source /tmp/jm-venv/bin/activate

1. Scaffold the generator being composed¶

just-makeit new studio
cd studio

# The object a source composes into: an ordinary jm component whose
# create/step/steps/reset/destroy are exactly what a composer expects.
just-makeit object clip \
    --state "level:double:0.0" \
    --arg-type void \
    --return-type "float _Complex"

Nothing composer-specific yet — this is a plain jm object. It matters because the composer's defaults are named after it: declaring generator = "clip" makes jm expect studio_clip_state_t, studio_clip_step, studio_clip_steps, studio_clip_reset, studio_clip_destroy and studio/clip/clip_core.h, all of which jm object has just produced. Every one is overridable in the manifest; none of them needs to be here.

--arg-type void --return-type "float _Complex" is what makes it a source: studio_clip_step(state) takes no input and returns a sample, and studio_clip_steps(state, out, n) fills a block.

Give the generator a body. A constant-level source keeps the composed output checkable by eye: in native/inc/studio/clip/clip_core.h, replace the two lines of the studio_clip_step stub's body so that it reads:

static inline float _Complex
studio_clip_step(const studio_clip_state_t *state)
{
    return (float _Complex)state->level;
}

2. Write the backing kernel¶

A composer does not generate its kernel — backing = "playlist" names C that is yours. It lives in a c_deps directory, which is jm's escape hatch for hand-written C: jm emits an add_subdirectory() line for it and never touches anything inside.

native/inc/studio/playlist/playlist_core.h:

/* playlist_core.h — the backing kernel, hand-written.
 *
 * jm never generates this: a composer's `backing` is the project's own C.
 * What jm DOES rely on is the shape below, so the three declarations here
 * are effectively an interface the generated binding calls into.
 */
#ifndef PLAYLIST_CORE_H
#define PLAYLIST_CORE_H

#include <complex.h>
#include <stddef.h>

/* The source config the `Clip` type wraps. One member per
 * `[[module.playlist.source.fields]]` entry — jm marshals Python onto this. */
typedef struct
{
  double gain;
} clip_t;

/* One segment: a set of sources summed over `dur` samples. jm builds a
 * transient array of these from the OO objects and hands it to create().
 * `fs` is always present, whether or not the manifest lists it. */
typedef struct
{
  clip_t *sources;
  size_t  n_sources;
  size_t  dur;
  double  fs;
} track_t;

typedef struct playlist_state playlist_state_t;

playlist_state_t *playlist_create (const track_t *tracks, size_t n, int repeat,
                                   int continuous);
size_t playlist_execute (playlist_state_t *state, float _Complex *out,
                         size_t max);
/* Reflects the RESOLVED spec back, so `Mix.segments` can rebuild OO objects
 * from what the kernel actually holds rather than from what was passed in. */
const track_t *playlist_segments (const playlist_state_t *state, size_t *n,
                                  int *repeat, int *continuous);
void           playlist_destroy (playlist_state_t *state);

#endif /* PLAYLIST_CORE_H */

native/src/backing/playlist_core.c:

#include "studio/playlist/playlist_core.h"
#include <stdlib.h>
#include <string.h>

struct playlist_state
{
  track_t *tracks;
  size_t   n_tracks;
  size_t   track_i;
  size_t   pos;
  int      repeat;
  int      continuous;
};

playlist_state_t *
playlist_create (const track_t *tracks, size_t n, int repeat, int continuous)
{
  playlist_state_t *st;

  if (!tracks || n == 0)
    return NULL;
  st = calloc (1, sizeof *st);
  if (!st)
    return NULL;
  st->tracks = calloc (n, sizeof *st->tracks);
  if (!st->tracks)
    {
      free (st);
      return NULL;
    }
  /* Deep-copy: the array jm passes is transient and its `sources` pointers
   * alias buffers owned by the Python objects. */
  for (size_t i = 0; i < n; i++)
    {
      st->tracks[i]         = tracks[i];
      st->tracks[i].sources = calloc (tracks[i].n_sources, sizeof (clip_t));
      if (!st->tracks[i].sources)
        {
          st->n_tracks = i;
          playlist_destroy (st);
          return NULL;
        }
      memcpy (st->tracks[i].sources, tracks[i].sources,
              tracks[i].n_sources * sizeof (clip_t));
    }
  st->n_tracks   = n;
  st->repeat     = repeat;
  st->continuous = continuous;
  return st;
}

size_t
playlist_execute (playlist_state_t *state, float _Complex *out, size_t max)
{
  size_t n = 0;

  if (!state || !out)
    return 0;
  while (n < max)
    {
      track_t *tr;
      double   sum = 0.0;

      if (state->track_i >= state->n_tracks)
        {
          if (!state->repeat && !state->continuous)
            break; /* a finite spec simply runs out */
          state->track_i = 0;
        }
      tr = &state->tracks[state->track_i];
      for (size_t k = 0; k < tr->n_sources; k++)
        sum += tr->sources[k].gain;
      out[n++] = (float _Complex)sum;
      if (++state->pos >= tr->dur)
        {
          state->pos = 0;
          state->track_i++;
        }
    }
  return n;
}

const track_t *
playlist_segments (const playlist_state_t *state, size_t *n, int *repeat,
                   int *continuous)
{
  *n          = state->n_tracks;
  *repeat     = state->repeat;
  *continuous = state->continuous;
  return state->tracks;
}

void
playlist_destroy (playlist_state_t *state)
{
  if (!state)
    return;
  for (size_t i = 0; i < state->n_tracks; i++)
    free (state->tracks[i].sources);
  free (state->tracks);
  free (state);
}

native/src/backing/CMakeLists.txt:

# Hand-owned. A `c_deps` entry gets an `add_subdirectory()` line from jm and
# nothing else — this file is never regenerated, which is the whole point of
# the escape hatch.
add_library(backing_core OBJECT playlist_core.c playlist_bridge.c)
target_include_directories(backing_core PUBLIC ${CMAKE_SOURCE_DIR}/native/inc)

3. Declare the composer — manifest only¶

"""Declare the composer module. There is no `jm composer` command.

A composer is manifest-only: this table plus `just-makeit apply` is the whole
interface. `c_deps` goes on `[project]`; everything else describes the OO
types jm will emit.
"""

from pathlib import Path

MANIFEST = Path("just-makeit.toml")

COMPOSER = """
[module.playlist]
kind = "composer"
backing = "playlist"
composes = ["clip"]
# `clip_core` is the generator's OBJECT lib; `backing_core` is the c_deps one.
# CMake will not pull either through transitively, so both are named here.
depends_on = [{ name = "clip", link = true }]
extra_link_libs = ["backing_core"]

[module.playlist.source]
object = "clip"
struct = "clip_t"
type_name = "Clip"

[[module.playlist.source.fields]]
name = "gain"
type = "double"
default = "1.0"

# ── seam 1: build the generator from a source config ──────────────────────
# jm emits the binding for Clip.step()/steps(); `clip_from_source` is the
# straight-C function that turns a clip_t into a running studio_clip_state_t.
[module.playlist.source.generates]
generator = "clip"
bridge_fn = "clip_from_source"
# ...and why it refused, when it does: a reason becomes ValueError instead of
# a bare "clip_from_source returned NULL" RuntimeError (gh-1307).
bridge_error_fn = "clip_why_not"

# ── seam 2: a derived read-only property ──────────────────────────────────
# Computed in C on every read, so it cannot go stale when `gain` is
# reassigned -- which a stored field would.
[[module.playlist.source.computed]]
name = "duration"
type = "double"
fn = "clip_duration"

[module.playlist.segment]
type_name = "Track"
struct = "track_t"
sources = "multi"

[[module.playlist.segment.fields]]
name = "dur"
type = "size_t"
default = "4"

[module.playlist.oo]
composer_type_name = "Mix"

# ── a method jm cannot express ────────────────────────────────────────────
# The body is hand-written CPython in native/src/playlist/playlist_ext_extra.c;
# this row is what gives it a place on Mix and a line in the .pyi.
[[module.playlist.extra_methods]]
name = "total_samples"
fn = "Mix_total_samples"
flags = "METH_NOARGS"
doc = "Samples the whole spec lasts: the sum of every track's dur."
returns = "int"
"""


def main() -> None:
    text = MANIFEST.read_text(encoding="utf-8")
    if "[module.playlist]" in text:
        return
    assert "[project]\n" in text, "unexpected manifest layout"
    text = text.replace("[project]\n", '[project]\nc_deps = ["backing"]\n', 1)
    MANIFEST.write_text(text + COMPOSER, encoding="utf-8")


if __name__ == "__main__":
    main()

The two seams are five lines of that table. Everything else describes the types jm will emit, and the fields list is the keystone: one ordered list of {name, type, default} per source and segment drives the C struct marshalling, the getset slots, and the constructor keywords all at once.

4. Apply¶

just-makeit apply

  create  native/inc/studio/playlist/playlist_bridge.h
  create  native/src/playlist/CMakeLists.txt
  create  native/src/playlist/playlist_ext.c
  create  src/studio/playlist/playlist.pyi
  update  CMakeLists.txt

The first of those is the gh-998 file, and it is the only header a composer module emits:

/* Build the composed generator from a source config (source -> generator). */
studio_clip_state_t *clip_from_source(const clip_t *, double);

/* Why clip_from_source() refused: a reason raised as ValueError, or NULL
 * for none (RuntimeError). Same arguments; called only after it returned
 * NULL (gh-1307). */
const char *clip_why_not(const clip_t *, double);

/* Computed read-only property `duration`. */
double clip_duration(const clip_t *);

Self-contained on purpose — it pulls in the backing header and the generator's, so a consumer does not have to work out what to include first. It is emitted only when the source declares at least one seam; a composer with none gets no header at all, because there would be nothing to say.

5. Write the two seams¶

native/src/backing/playlist_bridge.c:

/* The bodies behind playlist_bridge.h.
 *
 * Note what is NOT here: no Python.h, no PyObject, no reference counting.
 * jm owns the binding that calls these; the algorithms are the project's.
 * The include is the generated header, so these definitions are checked
 * against jm's declarations by the compiler rather than by eye.
 */
#include "studio/playlist/playlist_bridge.h"

/* Seam 1 — source config to running generator. A real one would derive
 * increments from `fs`; this one just carries the level across, and refuses
 * a configuration it cannot honour. */
studio_clip_state_t *
clip_from_source (const clip_t *src, double fs)
{
  (void)fs;
  if (src->gain < 0.0)
    return NULL;
  return studio_clip_create (src->gain);
}

/* ...and why. Called only after clip_from_source returned NULL, with the
 * same arguments: a sentence here is raised as ValueError; NULL means "no
 * reason to give" and keeps the RuntimeError. */
const char *
clip_why_not (const clip_t *src, double fs)
{
  (void)fs;
  if (src->gain < 0.0)
    return "a clip's gain must be >= 0";
  return NULL;
}

/* Seam 2 — a quantity derived from the config, never stored beside it. */
double
clip_duration (const clip_t *src)
{
  return src->gain * 2.0;
}

Two things are worth noticing.

The file includes the generated header rather than declaring anything. That is what makes the split safe: if the manifest renames bridge_fn, or a computed property changes type, this file stops compiling instead of quietly linking against a signature that no longer matches.

clip_why_not is the optional third seam (gh-1307). A bridge returns NULL for configuration reasons far more often than for memory, and a bare NULL reached Python as RuntimeError: clip_from_source returned NULL whatever the reason. The error function is asked only after a refusal, with the same arguments, and its sentence is raised as ValueError — the category a refused create() gets. Returning NULL from it keeps the RuntimeError, so a genuine allocation failure still reads as one.

And there is no CPython in it. The seams exist precisely so that the parts only you can write stay in plain C — the marshalling, the type objects, the reference counting and the error translation are all on jm's side of the line.

5b. A method jm cannot express¶

The seams cover what a source is. Now and then a type needs one method jm has no vocabulary for. extra_methods is the escape hatch: the manifest row in step 3 declares the name, the flags and the Python signature, and the body is plain CPython in a file jm never touches.

native/src/playlist/playlist_ext_extra.c:

/* Hand-written: jm includes this file after the generated types and never
 * modifies it. The PyMethodDef row and the .pyi line come from
 * [[module.playlist.extra_methods]]; jm also forward-declares the function
 * with the signature METH_NOARGS implies, so write exactly that one. */

static PyObject *
Mix_total_samples (PyObject *self, PyObject *Py_UNUSED (ignored))
{
  PyObject *segments = PyObject_GetAttrString (self, "segments");
  if (!segments)
    return NULL;

  size_t     total = 0;
  Py_ssize_t n     = PySequence_Size (segments);
  for (Py_ssize_t i = 0; i < n; i++)
    {
      PyObject *track = PySequence_GetItem (segments, i);
      PyObject *dur   = track ? PyObject_GetAttrString (track, "dur") : NULL;
      Py_XDECREF (track);
      if (!dur)
        {
          Py_DECREF (segments);
          return NULL;
        }
      total += PyLong_AsSize_t (dur);
      Py_DECREF (dur);
    }
  Py_DECREF (segments);
  if (PyErr_Occurred ())
    return NULL;
  return PyLong_FromSize_t (total);
}

jm #includes the file after the generated types, so the body can use anything they define. It also forward-declares Mix_total_samples above the method table that names it, with the signature METH_NOARGS implies: write that exact signature, or the file does not compile.

The order does not matter either. The row alone makes the binding include the file, so it can be written before or after the apply that declared it. If it is missing, the build fails naming playlist_ext_extra.c.

6. Build¶

cmake -B build -S . -DCMAKE_BUILD_TYPE=Release \
    -DPython3_EXECUTABLE="$(command -v python3)"
cmake --build build --parallel 4

The composer's .so links clip_core (the generator) and backing_core (the c_deps library) alongside NumPy. CMake will not pull an OBJECT library's objects through another target transitively, which is why both are named in the manifest rather than left to propagate.

7. Drive it from Python¶

"""Drive the three generated OO types, and both seams."""

import sys

sys.path.insert(0, "src")

import numpy as np  # noqa: E402

from studio.playlist.playlist import Clip, Mix, Track  # noqa: E402

# ── seam 2: a computed property ──────────────────────────────────────────
clip = Clip(gain=2.0)
print(f"Clip(gain=2.0).gain      -> {clip.gain}")
print(f"           .duration     -> {clip.duration}   (clip_duration, in C)")
assert clip.duration == 4.0

# Derived on every read, so reassigning the field it depends on is enough.
# A stored attribute would still be reporting 4.0 here.
clip.gain = 5.0
print(f"  after gain = 5.0       -> {clip.duration}  (recomputed, not stored)")
assert clip.duration == 10.0

# It is read-only: there is no field behind it to assign to.
try:
    clip.duration = 1.0
    raise AssertionError("expected a read-only property")
except AttributeError:
    print("           .duration = 1.0 -> AttributeError (read-only)")

# ── seam 1: standalone generation through the bridge ─────────────────────
# Clip.steps() has no kernel of its own; it calls clip_from_source() to build
# a studio_clip_state_t and then drives the generator jm never had to know about.
block = Clip(gain=7.0, fs=1.0).steps(3)
print(f"Clip(gain=7.0).steps(3)  -> {block}   (via clip_from_source)")
assert isinstance(block, np.ndarray)
assert np.allclose(block, [7 + 0j, 7 + 0j, 7 + 0j])

# A bridge that refuses says why: clip_why_not's sentence arrives as a
# ValueError, where it used to be "clip_from_source returned NULL".
try:
    Clip(gain=-1.0).steps(1)
    raise AssertionError("expected the bridge to refuse")
except ValueError as exc:
    print(f"Clip(gain=-1.0).steps(1) -> ValueError: {exc}")
    assert str(exc) == "a clip's gain must be >= 0"

# ...and help() says so up front: the Raises section names the ValueError
# and where its message comes from (gh-1356).
assert "the message is its reason" in " ".join(Clip.steps.__doc__.split())

# ── the composed object-of-objects ───────────────────────────────────────
# Track sums its sources; Mix sequences tracks and runs the backing kernel.
track = Track.sum(Clip(gain=2.0), Clip(gain=3.0), dur=4)
mix = Mix(track)
out = mix.execute(8)
print(f"Mix(Track.sum(2,3,dur=4)).execute(8) -> {out}")
# Four samples of 2+3, then the finite spec runs out -- execute() returns a
# short block rather than padding.
assert np.allclose(out, [5 + 0j] * 4)
assert len(out) == 4

# The resolved spec reflects back as rebuilt OO objects -- read from the
# kernel through playlist_segments(), not cached from what was passed in.
print(
    f"mix.segments             -> {len(mix.segments)} track(s), "
    f"repeat={mix.repeat}, continuous={mix.continuous}"
)
assert len(mix.segments) == 1
assert isinstance(mix.segments[0], Track)
assert mix.repeat is False and mix.continuous is False

# ── the hand-written method ──────────────────────────────────────────────
print(f"mix.total_samples()      -> {mix.total_samples()}   (_ext_extra.c)")
assert mix.total_samples() == 4
two = Mix([Track.sum(Clip(), dur=3), Track.sum(Clip(), dur=5)])
print(f"  over tracks of 3 and 5 -> {two.total_samples()}")
assert two.total_samples() == 8

print("composer_seams demo: PASSED")

Clip(gain=2.0).gain      -> 2.0
           .duration     -> 4.0   (clip_duration, in C)
  after gain = 5.0       -> 10.0  (recomputed, not stored)
           .duration = 1.0 -> AttributeError (read-only)
Clip(gain=7.0).steps(3)  -> [7.+0.j 7.+0.j 7.+0.j]   (via clip_from_source)
Clip(gain=-1.0).steps(1) -> ValueError: a clip's gain must be >= 0
Mix(Track.sum(2,3,dur=4)).execute(8) -> [5.+0.j 5.+0.j 5.+0.j 5.+0.j]
mix.segments             -> 1 track(s), repeat=False, continuous=False
mix.total_samples()      -> 4   (_ext_extra.c)
  over tracks of 3 and 5 -> 8
composer_seams demo: PASSED

duration is the argument for computing rather than storing: reassigning gain changes it, and there is no cache to invalidate because there is no cache. And Clip.steps() is a generator the source type never contained — clip_from_source builds it on demand, which is why a config object can produce samples at all.

8. The point: a C consumer needs only the generated header¶

native/tests/test_bridge.c:

/* A C consumer of the composer's seams.
 *
 * This translation unit includes the generated bridge header and NOTHING
 * else -- no Python, no re-declared externs, no knowledge of where the
 * definitions live. That is the whole of gh-998: before it, the only route
 * to these signatures was to write a second copy of them, and a second copy
 * is what drifts.
 */
#include "studio/playlist/playlist_bridge.h"

#include <stdio.h>

int
main (void)
{
  clip_t               src = { 0 };
  studio_clip_state_t *gen;
  double               d;

  src.gain = 3.0;

  /* Seam 2, called straight from C. */
  d = clip_duration (&src);
  if (d != 6.0)
    {
      fprintf (stderr, "clip_duration: got %f, want 6.0\n", d);
      return 1;
    }

  /* Seam 1, likewise -- and the generator it returns is an ordinary
   * jm component, so the rest of its API is already available here. */
  gen = clip_from_source (&src, 1.0);
  if (!gen)
    {
      fprintf (stderr, "clip_from_source returned NULL\n");
      return 1;
    }
  if (crealf (studio_clip_step (gen)) != 3.0f)
    {
      fprintf (stderr, "studio_clip_step disagrees with the source config\n");
      studio_clip_destroy (gen);
      return 1;
    }
  studio_clip_destroy (gen);

  printf ("bridge consumer: PASSED\n");
  return 0;
}

cc -I native/inc native/tests/test_bridge.c \
   build/native/src/backing/CMakeFiles/backing_core.dir/*.o \
   build/native/src/clip/CMakeFiles/clip_core.dir/*.o -lm -o /tmp/test_bridge
/tmp/test_bridge
# bridge consumer: PASSED

This is what the seams' prototypes moving out of _ext.c bought. That file is a CPython translation unit — including it from a test is not an option — so while the declarations lived there, a C test wanting to assert that the composed path and the standalone path agree could reach only the half with a public header. The real instance downstream had to say so in a comment and cover the other half from Python instead, which is a weaker claim than the one it was trying to make.

One header, one declaration each, checked by the compiler on both sides.