I worked on this project under Xiao Xiao as a research project (UROP) at the MIT Media Lab in 2014.

From the paper: "Andante visualizes as animated characters walking along the piano keyboard that appear to play the physical keys with each step. Based on a view of music pedagogy that emphasizes expressive, full-body communication early in the learning process, Andante promotes an understanding of the music rooted in the body, taking advantage of walking as one of the most fundamental human rhythms."

In this project, I worked on developing the animated figures and some code in the Processing sketch that powered it. The most interesting logic in this is creating code that can figure out what sequence of frames must be used when - depending on the jump between notes, a figure may need to physically jump, run or walk, or hop again onto the same note it's already on. It may need to turn between notes and travel, or keep moving in one direction and the distance between notes motivates the density of frames that can be used if a constant frame rate is assumed. A large library of frames was needed to create a library that could cover all cases for each figure. The physical scale of the piano was also an important consideration, as its fallboard and keys create constraints on scale that also need to work with the projector's resolution.