MAIN GOAL: Figure out if and how we are going to implement the "MIDI sample player" (I kind of want to call it "Sample Synth" but that makes less sense to other people I think)
The goal of this component would be to play samples in response to MIDI events being received. Beyond that, I was imagining two different modes of operation:
My original thought for this was to create a separate node or VC for this. It would allow for a clean-room implementation with a simple and directed design. However, then I thought about what other functionalities would be required. What if we wanted to put envelopes on the samples? Wanted to apply per-voice effects to them? I realized that was all all functionality of the [synth-designer] that already existed and thought that perhaps it would be better to add this sample playing functionality into the synth designer instead. That goes along with the goal of making the synth designer the one-stop shop for MIDI-based synthesis in web synth.
So, what would that look like? Well, it would be pretty similar to wavetable - very similar in fact. We'll add a new "waveform" type or two to the synth designer which corresponds to these new sample synth modes. Then, we'll add some additional state to it to track the loaded samples, configurations, and whatnot. Will of course need to write the engine code to do the synthesis, handle plumbing and initialization code for loading samples and setting them into the backend. And finally, we'll need to create a UI to configure all of this.
All of that is clear to me and should be straightforward enough except for the UI. I kinda really want to build the UI with Svelte, but it's also going to have to be integrated into the current synth designer/FM synth UI. What will the UI look like anyway? The user will need to be able to select mappings from samples to MIDI notes, probably with MIDI learn functionality. I think that it would be good to have a view showing the currently mapped samples on a MIDI keyboard of some kind as well, but that could probably be handled with just a scrollable vertical listing.
Will need some UI for picking samples as well, but we can use existing sample picker dialogs for that. The main thing I'm considering right now is whether to try to fit this into the narrow FM synth UI we have currently or to expand it out into a modal or similar. TBH, I think we'll be able to fit it. If there is something that is difficult to represent or something that only needs to be used non-interactively, we can pop it into a modal flow - I don't really care.
OK well I actually have a pretty concept for this now. Let's build!
OK I barely got started and need to consider some more things. I realized that this functionality may overlap with wavetables more or less completely. Or, maybe not. I'm faily confident that I could make this work by ab/using wavetables. We'd throw away the whole "table" aspect of it though and just have a single dimension with a single waveform. However, as a downside, we'd have to do special-casing for de/serialization to load wavetables from sample manager rather than hard-coding into the serialized synth designer state like we do currently.
I don't think we really gain that much tbh. The alternative would be maintain a separate, special-purpose sample player state. It will be much easier to represent loading states in there and everything, yeah the more I think about this the more I'm sold. It won't be very expensive to add and I really don't think it will add any runtime performance whatsoever. Won't have to special case stuff like non-looping and tuning as well. OK let's just build.