In order to make the playback of sequenced notes in the MIDI editor as accurate and well-timed as possible, some nodes in web synth schedule MIDI events directly on the audio thread. This is referred to as audio thread MIDI scheduling.
This is only relevant for pre-scheduling MIDI events that are created during playback of the [global-beat-counter]. Interactive MIDI events such as those originating from external MIDI keyboards always originate on the UI thread, but since they're dynamic the latency constraints aren't as tight.
Audio thread MIDI scheduling is applicable to both senders and receivers of MIDI events. In order for the full benefit to be realized, both the producer and consumer nodes must support it.
The core components of this implementation are the [event-scheduler], [midi-node], and [global-beat-counter]. They work together to both support the scheduling itself as well as interfacing between events originating on the audio thread and UI thread.
As previously mentioned, there are both producers and consumers of audio thread MIDI events. Notable producers include the [midi-editor] and [looper] and the main consumer at the time of writing is the [synth-designer].
Nodes must opt in to audio thread MIDI scheduling. This is done by setting enableRxAudioThreadScheduling in the [midi-node]'s input callbacks.
The core of the implementation revolves around some globals set up by the [event-scheduler] on the audio thread. Since all [audio-worklet-processor]s run in the same global context, they can all reference and read from these same globals.
The main entry point is globalThis.midiEventMailboxRegistry. For each mailbox, it holds a ring buffer of encoded MIDI events. These events can be written and read using the submitEvent and getEvent methods respectively. Check out EventSchedulerWorkletProcessor.js in the codebase for more details.
Since each source and destination node can send/receive MIDI events on either the UI or audio thread, special handling sometimes needs to be done to determine where to send MIDI events.
When MIDI events originate on the UI thread, the situation is somewhat simpler. The MIDI node will handle checking if this is set on all connected rx nodes' input callbacks and post incoming MIDI events from the UI thread to the audio thread directly. The UI thread callbacks will not be called in that case.
When sending MIDI events from the audio thread, state will have to be kept synchronized as to whether the destination MIDI nodes have the enableRxAudioThreadScheduling prop set. If it is set, then the MIDI event can be posted to the referenced mailbox IDs directly on the audio thread. If it is not set, then the MIDI event will have to be sent to the UI thread and sent to the receiver MIDI node using the onAttack/onRelease/etc. interface.
This is the original way that MIDI events were sent/received. In the case that the receiving MIDI node processes MIDI events on the UI thread, then the onAttack/onRelease functions on its MIDI node must be called directly.
In order to receive MIDI events on the audio thread, enableRxAudioThreadScheduling must first be set on the MIDI node's inputCbs to opt-in. Then, they must register a mailbox with a unique ID by making use of the mailbox infra set up by the [event-scheduler] like this:
globalThis.midiEventMailboxRegistry.addMailbox(processorOptions.mailboxID);
This is done on the audio thread. Once the mailbox is registered, events will be written into the mailbox by tx nodes. The events can be pulled out like this:
let msg;
while ((msg = globalThis.midiEventMailboxRegistry.getEvent(this.mailboxID))) {
const { eventType, param1 } = msg;
switch (eventType) {
// ...
}
}