Playing drones as instruments

We have long been trying to bridge the art world and games. We have been focusing on exploring the fringes of the digital medium. Hacking drones to play music is one such experiment.

There is a zeitgeist surrounding the recent surge of toy quadcopters [toy helicopters with 4 rotors that can stabilise itself], colloquially called drones. We have explored their technical capabilities in summer 2014 when we did the Hack-a-drone Workshop at the Click Festival in Helsingør.

Drones in art?

As the quadcopters are an interesting piece of technology, understandably the tech crowd latched on first. Therefore most early drone projects put the focus on image processing, networking and artificial intelligence. The low price of drones and their hackability, made them particularly attractive to these people.

Drones, a term originally used for pilotless weaponized flying objects, have been used in artistic ways before. However; their seemingly intelligent stabilization capabilities, their “athletic power” and their extreme mobile spying capability make them seem malevolent behind their benign toy exterior. These technical and practical aspects coupled with their explosion in availability without the public having adjusted to them, make them an interesting discussion topic. How do we as a society legislate this? Should we even legislate this? Who is accountable for them if they cause harm when there is a malfunction in the system while they are flying? How do we tackle malignant use? Many more discussion points arise, which makes them interesting for art as well.

We are amongst the artistic experimenters, trying to take the drones a step further than “follow the yellow dot experiments”. I took the drone with me to the Maldives Playmaker event where it, if nothing else, made an impression on the local crowd (the public and especially kids love them). Together with Playfuel and Jolly Wally I created a game called “Everything is awesome”. A pervasive experience about drone surveillance in a dystopian society that outlawed creativity.

Lastly, Tim Garbos and I did a collaboration with a classical composer Anders Monrad. Monrad has a formal degree from the classical conservatory in Copenhagen. He has always loved the beauty of complex compositions in both contemporary and classical music, and has been keen on spreading out this love to the general public. However, it almost requires a university degree to appreciate contemporary pieces, and there seems to be a deed in the the world of composers to make it as unobtainable to the general public as possible. And even though it is frowned upon in that world, Monrad wants to break this barrier by making his compositions more available through mediums that the public understands, thereby slowly introducing them to more advanced compositional music. Games and toys (drones) are such “mediums” that many have a literacy in, need little explanation and are approachable. These are the ideas behind the Drone Concert.

Drones as instruments?

The drones, as discussed, can be perceived as malevolent, but what if they are merely a tool? When we control them ourselves instead of granting them autonomy and when we “play” with them, then their malevolence and social context diminishes and the magic of the flying things creating music in front of us materialises.

Since the drones are instruments, but also occupy the physical space and fly around, they obviously create a physical performance, almost a modern dance, one could say. The drone concert is divided into different acts, where not only the music changes, but so does the performance of each drone. During the concert, 2 drones perform for the audience. Each drone has it’s own personality and that affects the flying style, the narrative and in effect the music. This binds the it all together and creates a wholeness; a Gesamtkunstwerk.

There have been other attempts to bring drones into musical performances. From Drone light painting and  the beautiful Cirque du Soleil [SPARKED] video of the flying lamp shades, where the drones are merely eye candy, a type of physical visualisation, to Elevenplay dance performance, a dance performance with a pre-scripted computer guided system that controls the drones.

By removing  the autonomy of the drones we have essentially made them an extension of ourselves. Just like a musician playing a guitar, drum or violin, we make mistakes and are not perfect. It is this imperfection and humanisation of the tool that, together with the drones’ perceived modernistic science fiction features, makes it an interesting and novel project.

We achieved all this by coding a program that controls the drones with Playstation Move controllers using the UniMove library in the Unity game engine. This software connects to a server script that controls a drone. The drones are sending their current status directly to a music system that interprets their status and creates music correspondingly. Their speed, altitude, battery state etc. all feed into this system. On top of this the concert follows a static composition that makes the drones sound different in the beginning than in the end. This statically composed piece of music is played together with the dynamic system. The whole software setup, except the music composition, is made open source (MIT Licence) and will get separate in-depth technical blog post will explain the nooks and crannies.

What’s the point?

We made a concert where drones fly around.

• So what?
• I didn’t get it.
• What is the point of this?

These are the reactions and questions we get when performing this piece. So what is the point of this?

If you have gotten this far in the post, then you are no doubt interested in experimentation. You might even have seen the inspiration from the Helikopter-Streichquartett, one of Stockhausen’s art pieces, featuring a string quartet with four actual helicopters.

We have created an experience that is novel and, because of the current trend surrounding toy drones, also interesting for a crowd that does not normally experience contemporary classical music. By breaking with the usual socio-cultural context of drone, we show a different future for them. It’s a technical and practical achievement, and if nothing else, it was a lot of fun to do.