Translating music into light and motion with robots - Robohub

Researchers at the University of Waterloo have developed an innovative system that enables collaboration between humans and a swarm of wheeled robots to create visual art inspired by music. These robots, roughly the size of soccer balls, move within a confined floor area while trailing colored lights that respond dynamically to musical features such as tempo and chord progression. A camera captures the coordinated light trails, effectively turning the floor into a canvas that visually represents the emotional content of the music. The system translates musical emotion into visual elements through variations in light color, intensity, trail width, robot position, and movement speed, allowing for an expressive and adaptive artistic output. The technology also incorporates human input, enabling users to influence the painting process by adjusting light trail width and positioning, fostering a collaborative interaction between humans and robots. The researchers initially focused on developing algorithms to control multiple robots simultaneously, successfully testing with up to 12 units and designing the system to be scalable. Additionally, they created methods to extract and analyze emotional features from music to guide the