The Integration Investigation
Into the wilderness
Developing a bespoke middleware audio engine requires some investigation of current available tools. To begin with I will be looking into MAX/MSP and Pure Data due to readily available support and resources. Previous experience working within these environments will allow me to quickly ascertain which one of these will be a liable option for development.
Other considerations include: Super Collider, Sonic Pi, Chuck and C Sound. Although all of these provide a great amount of creative freedom and control, the goal is to have an integrated audio system operating inside a game engine, Unity. This means the integration process will also need to be taken into consideration when deciding which application to use.
At this point it's worth mentioning the foundations of these audio development tools are based on Digital Signal Processing (DSP).
Myu - Max-Unity Interoperability Toolkit
μ [Myu] was developed at Digital Interactive Sound & Intermedia Studio [DISIS] and released in 2009 for download in 2009. Myμ was designed to "simplify communications" between the two platforms. Myμ utilised TCP network packets using the netsend/netreceive framework, making it Pure Data compatible too. Its ability to integrate external hardware peripherals enabled “rapid prototyping potential with minimal integration overhead.”.
After downloading the Myμ integration package and attempting to run the project, a quick decision was made to leave this approach on the back burner until further investigation could be carried out. It seemed the problem was communication issues between Max and Unity. Multiple errors were thrown up with unstable control between the two platforms.
At this early stage the aim is to find a solution that works quickly and easily. Issues could also stem from lack of maintenance and version control of these integration methods, meaning Myμ could be outdated in relation to Unity engine updates. This is commonly referred to as being "deprecated".
Pure Data is available in 2 versions, Extended (deprecated) and Vanilla. Binding protocols have been developed providing integration with mobile platforms and other applications (such as Unity game engine).
Libpd is an integration approach that provides DSP rendering callbacks for audio and MIDI data signals. Rapid prototyping and comprehensive functionality is provided with the ability to exchange control and MIDI messages between itself and connected client. The Libpd page from the Pure Data website summarises this nicely:
“libpd turns Pd into an embeddable library, so you can use Pd as a sound engine in mobile phone apps, games, web pages, and art projects.”
Previous experience using the LibPD4Unity integration workflow has proved successful but as memory serves, the process was time consuming and highly frustrating. The design and implementation processes used for building the Tanks project will still remain the foundation for using Pure Data as an integrated audio system.
Although this approach has been maintained and updated, another integration option currently available has gained attention in the game audio communities, apparently taking the win for Pure Data and Unity integration. Further investigation into Libpd should help provide decisive information towards its continued use or not.
Heavy from Enzine Audio
Enzine Audio have developed an outstanding new approach to integration. The efficient and flexible platform enables transformation from high level languages to lower level*. The output options allow projects to be translated into a variety of languages compatible with a wide range of platforms.
Although current issues converting some Pure Data object may cause issues in development, the team are working on providing a library of Heavy compatible objects as a work around. Also licensing is required for some conversions in Heavy which comes at a financial cost, in comparison to Libpd which is entirely free.