I do appreciate that both of these VR instrument platforms have given some thought to allowing performance data to leave their VR environments, but the implementations are currently rather restrictive and incomplete, especially if you use an Apple computer. I have made or modified some tools that work for my simple needs and might be generally useful. I expect these tools to be rapidly superseded by official tools with greater capabilities. At least, I hope that will be the case.
Patchworld
Patchworld has a suite of internal objects designed to interface with Ableton Live, collected under the ‘Ableton toolkit’ category in the object menu. A range of sliders, toggles, knobs and strange disquieting stretchy interface blobs can be wired to an 8 channel bridge device. This device can send a range of either 0-1 or 1-127 on any of 4 sets of 8 channels. Surprisingly, there is no facility for sending notes or note velocity values, so I made one to use with my drum array.
Information is received by a MAX patch available from maxforlive.com. Although versions 1.0 and 1.5 are available I have yet to find any difference between them, even after opening them in editing mode. The embedded javascript object that is supposed to automatically link the bridge to a running copy of Patchworld has never worked and appears to be missing the code. The bridge will function if you manually enter the IP addresses.
I have addressed the lack of note information by simply sending the MIDI note number and corresponding velocity over two channel pairs previously used for MIDI controllers; 5+6 and 7+8. The remaining 4 channels are still available for MIDI CC. Each bridge can be set one of 4 channels, each with 8 lanes, so the loss of 4 to note values should not be much of an imposition. I originally used only two channels for note transmission but the response time was not fast enough to handle the data generated from two handed playing.
Using the modified Patchworld OSC Bridge
The new sections I have added are:
- Input Detect. This button flashes when receiving note information through channel 5 or 7.
- Note Test. Pressing this button will send a copy of the last note received at a velocity of 64. If no notes have been received this button will send note 64 for channel 5 and 66 for channel 7. It can be difficult to use a VR instrument to send notes when mapping MIDI or tuning parameters inside Ableton, even with good pass-through cameras such as those on the Quest 3. My Quest is always very quick to enter sleep mode when lifted away from the face, making it difficult to ‘peep’ at the screen without having to wait for the headset to wake each time. Use these buttons instead of juggling your headset.
- Test CC in is similar – rather than trying to keep continuous gestural controls engaged in the headset while working inside Ableton, these dials can be used to send the same CC numbers for testing instruments, assignments, etc.
Virtuoso VR
There is a already an official companion application (and a VST3 plugin) that allows Virtuoso to send note and controller information to your computer. Neither of these work on my Mac. Although source code is available, the pre-made executables are Windows platform only and I cannot use them. To address this lack I have made both standalone MAX applications and M4L objects that can connect to a running copy of Virtuoso. They currently only support the harp and marimba instruments, with hand rotation sensing for the marimba (the harp does not send this information for some reason). The standalone application can send MIDI information to whatever port and channel you select. M4L objects have no concept of port or channel (all information is channel 1 no matter what port number you send), so I will, in a future version, allow channel number to be used as a coarse MIDI controller for parameters the do not require fast updating – it may be useful for selecting timbre or physical properties for Collision. This is a work in progress and will be expanded to cover all of the Virtuoso instruments. For instrument builders or tinkerers who have to deal with real-world information flows, Live’s built-in MIDI mapping is still, in 2024, infuriatingly unusable, with no way of directly entering mapping information. To make mapping possible I have taken the MAP buttons from the Patchworld bridge.
Using the Standalone Application and M4L tool
The layout of the two interfaces is nearly identical but some controls have different functions as described below.
- Each instrument has a circular input detector. This will flash when MIDI notes are received for that instrument.
- Each instrument has toggles to accept Note Off messages. Note off is not always needed, especially for percussion sounds where it can cause problems by abnormally terminating sounds. Most continuous-note instruments, such as synthesisers, will need to receive note-off messages or the sounds will never stop.
- Each instrument also has an ‘Accept MIDI Channel’ toggle. In the stand alone application this allows the channel selection tools of the virtual instruments to modify the MIDI transmission channel of the relevant instrument output. M4L has no MIDI channel information, so this will be repurposed into another MIDI controller receiver.
- The left and right tilt channel indicators, the red arcs, show input data and can also be manipulated with the mouse to send controller information for testing purposes. The percentage boxes underneath can be used to set the lower and upper bounds of the controller mapping transmitted to mapped Live targets.
- ‘Send local IP’ sends the computer’s IP address to Virtuoso VR via the network.