Ph.D
Group : Human-Centered Computing

Supporting music composition with interactive paper

Starts on 01/10/2010
Advisor : MACKAY, Wendy

Funding :
Affiliation : Université Paris-Saclay
Laboratory : LRI

Defended on 10/06/2014, committee :

Research activities :

Abstract :
This thesis focuses on the design of interactive paper interfaces for supporting musical creation. Interactive paper technology enables to capture handwritten gestures on paper and creates new opportunities for combining expression on paper and computation. However, designing interactive systems for highly individual creative practitioners that develop personal musical representations on paper and use sophisticated programming tools is challenging.
Based on exploratory studies and participatory design sessions with composers, we introduce Paper Substrates, interactive paper components that provide modular structures for interacting with personal representations of computer-based musical data. We have developed a toolbox and a GUI application to facilitate the creation of interfaces on paper with Paper Substrates and to link them with computer-aided composition tools.

We used these tools to build several interactive systems with interactive paper including handwritten music input, drawing in tone networks or using movable paper elements for musical creation. Among the applications, we have developed and used Polyphony, an interface that integrates interactive paper with other interfaces, to conduct a structured observation study with 12 composers. The study allowed us to systematically observe and compare their compositional processes. We also collaborated with composer Philippe Leroux for the composition of his piece Quid sit musicus. We created interfaces for generating composition material, synthesizing sounds and controlling the spatialization from handwritten gestures from calligraphic gestures over an old manuscript. These studies inform the design of interactive paper tools for music composition and creativity support.

CAUSAL LEARNING FOR DIAGNOSTIC SUPPORT


CAUSAL UNCERTAINTY QUANTIFICATION UNDER PARTIAL KNOWLEDGE AND LOW DATA REGIMES


MICRO VISUALIZATIONS: DESIGN AND ANALYSIS OF VISUALIZATIONS FOR SMALL DISPLAY SPACES
The topic of this habilitation is the study of very small data visualizations, micro visualizations, in display contexts that can only dedicate minimal rendering space for data representations. For several years, together with my collaborators, I have been studying human perception, interaction, and analysis with micro visualizations in multiple contexts. In this document I bring together three of my research streams related to micro visualizations: data glyphs, where my joint research focused on studying the perception of small-multiple micro visualizations, word-scale visualizations, where my joint research focused on small visualizations embedded in text-documents, and small mobile data visualizations for smartwatches or fitness trackers. I consider these types of small visualizations together under the umbrella term ``micro visualizations.'' Micro visualizations are useful in multiple visualization contexts and I have been working towards a better understanding of the complexities involved in designing and using micro visualizations. Here, I define the term micro visualization, summarize my own and other past research and design guidelines and outline several design spaces for different types of micro visualizations based on some of the work I was involved in since my PhD.