Imagine a boombox that follows every move and suggests music that suits your personal dance style. Such an idea “Beat”, one of several projects from the MIT course 4.043/4.044 (Intelligence of Interaction)taught by Marcelo Coelho in the Architecture Department, which were presented at the 38th annual Neurips (Neural Information Processing Systems) conference in December 2024. With over 16,000 participants, it coincides in Vancouver, Neuips is a competitive and prestigious conference on research and science in the field machinery, as well as an undertaking at the exhibition.
The course is investigating the rising field Large language facilitiesAnd how artificial intelligence can be extended to the physical world. While “Beat” transforms creative dance possibilities, other students' applications include disciplines such as music, story story, critical thinking and memory, creating generative experiences and new forms of interaction of human computers. To sum up, these projects illustrate a broader vision of artificial intelligence: one that goes beyond automation to catalyze creativity, transform education and recalculate social interactions.
Be rhythm
“Beat” by Ethan Chang, a student of mechanical and design engineering MIT and Zhixing Chen, a student of mechanical engineering and MIT music, is a boombox with AI drive, which suggests music from the dancer movement. Dance traditionally managed music throughout history and in different cultures, but the concept of dance to create music is rarely studied.
“Beat” creates a space for cooperation between people-Ai in the field of freestyle dance, strengthening dancers to think about traditional dynamics between dance and music. He uses Posenet to describe movements for a large language model, enabling him to analyze the dance style and API inquiries to find music of similar style, energy and pace. Dancers interact with Boombox, who have more control over artistic expression and described Boombox as a new approach to discovering the species of dance and creative choreography.
A secret for you
“A Mystery for You”, author: Mrinalini Singha SM '24, a recent graduate of the art, culture and technology program, and Haoheng Tang, a recent graduate of Harvard University Graduate School of Design, is an educational game to cultivate critical thinking and checking facts in young learning out. The game uses a large language model (LLM) and a tangible interface to create an engaging investigative experience. Players act as perpetrators of the facts of citizens, responding to “information notifications” printed by the game interface. By inserting combinations of cassettes to quickly monitor the “message update”, not ambiguous scenarios move, analyze evidence and weigh contradictory information to make informed decisions.
This experience of interaction between a man-computers questions our habits related to information consumption, eliminating the interfaces of the touch screen, replacing the continuous scrolling and reading with reading roughly with a rich analog device. Combining the slow media with new generative media, the game promotes thoughtful, embodied interaction, while equipping players to better understand and question today's polarized media landscape, in which disinformation and manipulative narratives develop.
Memorcope
“Memorscope”, a collaborator of MIT MEDIA LAB Research Keunwook Kim, is a device that creates collective memories by combining deeply human experience face -to -face interaction with advanced AI technologies. Inspired by how we use microscopes and telescopes for researching and discovering hidden and invisible details, the memorcope allows two users to “look” at each other, using this intimate interaction as a gate to create and explore their common memories.
The device uses AI models, such as Opeli and Midjourney, introducing various aesthetic and emotional interpretations, which causes a dynamic and collective memory space. This space exceeds the limitations of traditional common albums, offering a liquid, interactive environment in which memories are not only static shutter but lively, evolving narratives, shaped by continuous relations between users.
Narratron
“Narratron” Harvard Graduate School of Design Students Xiying (ARIA) Bao and Yubo Zhao, is an interactive projector who co -creates and cooperates stories for children through Puppetry Shadow using large language models. Users can press the shutter to “capture” the characters they want to be in history, and shadows (such as animal shapes) as input for the main characters. The system then develops the story when new shadow characters are introduced. History appears by the projector as a background for Shadow puppet when it is told by a speaker, when users change the crank into “playing” in real time. Combining visual, auditory and bodily interactions in one system, the project aims to cause creativity in telling shadow stories and enable multimodal cooperation of man-Ai.
Perfect syntax
“Perfect syntax” Karyn Nakamura '24 is a work of video art examining syntactic logic behind movement and video. Using artificial intelligence to manipulate video fragments, the project studies how the fluidity of movement and time can be simulated and reproduced the machines. By attracting inspiration from both philosophical research and artistic practice, Nakamura's work interrogates the relationship between perception, technology and movement that shapes our world experience. Re -imagining the video through calculation processes Nakamura examines the complexity of how machines understand and represent the passage of time and movement.
