framework-arduinoteensy-cxx20/libraries/Audio/new_objects.md

Start by finding a similar object in Audio.h and copy its definition, and of course change the name. All objects much inherit from AudioStream. If your object has inputs, you must have "audio_block_t *inputQueueArray[num]" in the private section, where "num" is the number of inputs. Your constructor must initialize AudioStream with the number of inputs and that array. You also must have the virtual update() function in your public section. Everything else is optional... just add private variables as needed for whatever you want to do, and add public functions for whatever stuff you want to be accessible from the Arduino sketch. The simplest way to get this "boilerplate" stuff right is to copy an object definition that already works. Maybe I ought to publish a template example?

In your actual code, the only function you must implement is update(). If you don't, the compiler will give a rather unhelpful vtable error.

If your object gets input, in update() you'll call receiveReadOnly(channel) or receiveWritable(channel) to acquire any incoming audio. The library allows shared memory, so receiveReadOnly() is more efficient in if the same audio is fed to other objects. Each input can only source 1 block, so you only need to call either of these one for each input channel. As the name suggests, you choose which function depending on whether you will modify the contents of the audio data. The mixer object, for example, calls receiveWritable() for the first block it acquires, and then receiveReadOnly() for any others. These functions can and do return NULL if there is no input, so you must check for NULL and treat the input as silent.

If you need to create audio, or you just need another block to fill, call allocate(). It too can return NULL, so you must check.

When you obtain audio from allocate(), receiveReadOnly() or receiveWritable(), you own it. You must call release() to free the memory. If you need to buffer audio, you can store block pointers in your object's private variables and use them on future update() runs, but this consumes the precious audio memory, so only keep blocks allocated if you really must.

The pointers you get, type "audio_block_t *" are a struct with 1 member you're mean to access: "data", which is an array of 128 int16_t's with the actual audio data. The struct has a couple other members which you should not touch. If you obtained the block with receiveReadOnly(), do not write to the data[] array. But you still must call release() when you're done with that block.

Once you've got your input blocks, plus any new blocks you need, do whatever your object will do. If you're modifying audio, you'll probably obtain the inputs as writeable blocks and change their data[] arrays directly. If you're synthesizing something, you'll probably get new blocks with allocate() and fill them up. Of course, you can do anything you want, within the limits of your programming skill and the available memory and CPU power.

If your object has outputs, call transmit(block, channel) to send audio out. It's ok to transmit the same block to multiple outputs. You still own that block, even after transmit. You must always call release() for every block you obtain with receiveReadOnly(), receiveWritable() or allocate().

For example, if you wanted to create automatic gain control, you might have 1 input and 1 output. You'd call receiveWritable(), then perhaps compute an average by just summing up the samples (inverting the negative ones, of course). Or maybe you'd square them and add them all up? Then you might adjust your gain slightly if the average is above or below the target, which of course would be a private member variable, and multiply all 128 samples by the gain setting. Then just transmit and release the block.

If your object provides functions to control its operation, you may need to carefully design and data exchange between the functions called by the sketch running in normal context, and the update() function which runs in low priority interrupt context.  Generally, shared variables need to be declared with "volatile", and interrupts may need to be disabled with noInterrupts() while you read or edit shared data.  Also minimize time with interrupts disabled.  Use interrupts() to re-enable.

Every objects gets simple CPU usage tracking (but I need to publish an example), so you can test how much CPU you're using. The Cortex-M4 DSP optimizations can really help. In this AGC example, to compute the average, you could fetch the input samples 2 at a time using a 32 bit pointer. The M4 optimizes multiple reads in a row, where a normal 32 bit read takes 2 cycles, but subsequent back-to-back reads take only 1 more cycle. So you could fetch 8 inputs in only 5 cycles. The DSP instructions feature a dual 16x16 multiply-and-accumulate (allowing inputs from separate halves of 32 bit registers), so you could square and sum each pair of inputs in just 1 cycle. The average could be computed in just 144 cycles without looping overhead. To keep things in perspective, ALL the update functions for all audio objects must complete in under 278528 cycles. The CPU usage functions can tell you what fraction of that total you're using up.

Likewise, the AudioMemory() function at the beginning of setup() creates the pool of memory which provides all audio_block_t data. There are functions to query the current and worst case usage, so you can tell if you've got memory issues. But if you receive or allocate audio blocks and fail to release them, you'll quickly run out of memory and the entire system will go silent, so memory leaks are pretty obvious.

Objects that actually move data on or off the chip have some other requirements, which are what causes every update() function to actually run every 2.9 ms. But you don't need to worry about those if your object runs entirely on-chip, using only receiveReadOnly(), receiveWritable(), allocate(), transmit() and release().