6G and the Internet-of-Musical Things (IoMusT)

One of the use cases proposed during early 5G days was a band being able to play live music with team members spread geographically. Mischa Dohler covered this in his various talks, especially one on Internet of Skills. NTT Docomo had a fantastic use case on this that you can read here. The original video is gone but you can see a clip of the video here.

A recent research paper titled, "The Journey Towards 6G: A Digital and Societal Revolution in the Making" introduces the concept of Musical Thing and Internet-of-Musical Things. Quoting from the paper:

Musical Thing (MusT) is “a computing device capable of sensing, acquiring, processing, or actuating, and exchanging data serving a musical purpose”.

Internet-of-Musical Things (IoMusT) are “the ensemble of interfaces, protocols and representations of music-related information that enable services and applications serving a musical purpose based on interactions between humans and Musical Things or between Musical Things themselves, in physical and/or digital realms”

The abstract of the paper is as follows:

While the fifth generation (5G) is bringing an innovative fabric of breakthrough technologies, enabling smart factories, cities, and Internet-of-Things (IoT), the unprecedented strain on communication networks put by these applications, in terms of highly cognitive, agile architectures and the support of massive connectivity, energy efficiency, and extreme ultralow latency, is pushing 5G to their limits. As such, the focus of academic and industrial efforts has shifted toward beyond 5G (B5G) and the conceptualization of sixth generation (6G) systems. This article discusses four main digital and societal use cases (UCs) that will drive the need to reconcile a new breed of network requirements. Based on this, we provide our vision of the fundamental architectural ingredients that will enable the promise of 6G networks of bringing the unification of experiences across the digital, physical, and human worlds. We outline key disruptive technological paradigms that will support 6G materialize a bouquet of unique expectations and redefine how we live and protect our planet. Finally, we adopt the recently envisaged ecosystem of the Internet-of-Musical Things (IoMusT) to depict how the discussed UCs and technological paradigms may be exploited to realize this ecosystem.

The paper explains Internet-of-Musical Things overview and application scenarios as follows:

1) Adaptation to Environmental Knowledge
  • MusTs may reconfigure themselves automatically based on dynamic changes in the structure/layout of their sensor interface.
  • MusTs may sense environmental-related parameters (e.g., humidity level, device energy consumption, temperature) and send the sensed data over the IoMusT network.
2) Enhanced Music E-Learning

  • MusTs may infer in real-time music playing errors made by e-learners. Smart instruments can provide haptic and visual feedaback through connected wearables and VR glasses worn by the learner.
  • Based on progress analytics received from the cloud, MusTs may recongfigure themselves (e.g., sensor-tosound parameter mapping) automatically to suit the learner’s progress needs and abilities to meet the learning program targets 

3) Remote Rehearsals, Intelligent Mixing, and Interaction with the Cloud

  • MusTs may exchange their produced sounds over the IoMust network for real-time remote concert rehearsals. 
  • MusTs record a mix of those sounds and the metadata associated with the instrument’s configurations (e.g., the utilized sound effects chain, synthesizers, drum machines, as well as their presets), the sensors signals, and the score (automatically transcribed) of what they played for feedback from remote producers.

4) Smart Music Therapy

  • Patients' mood and/or health are inferred by servers connected to the IoMust network through data received from patient’s wearables.
  • Inferred mood and/or health conditions are repurposed in control signals for the connected MusT to generate and on-demand customized real-time music therapy
  • IoMusT perform continuous bio-signal monitoring and mapping for provide adaptive and evolved music therapy.

5) Augmented and Immersive Live Concert Experiences

  • Band musicians act on sensor interfaces of their smart instruments to deliver to colocated or remote audience visuals displayed on their smart glasses, vibrations and thermal sensations on their wearables’, haptic stimulation on their smart clothing, etc.
  • IoMusT system predicts the audience’s mood to help musicians decide on the next songs in their playlist and/or help choreographers create real-time live visualizations responding to the mood of the audience.

We are surely going to see a lot more IoMusT use cases once 6G starts to take shape.

Related Posts: 

Posted by on

Comments

Post a Comment