Here I present the first draft of an idea to be realized as the final project for Introduction to Physical Computing class. I propose the creation of a physical interface for the manipulation of sounds derived from the sonification of specific sequence contexts contained in the DNA sequence of genes involved in the photoperiodic pathway of rice (Oryza sativa). This project extends previous works I've done in the field of GENOME SONIFICATION and ARTE GAGAISTA.
Genome sonification is the auditory rendering of genetic elements contained in DNA sequences of biological organisms, whereas 'arte GAGAISTA' is a new form of geometric expressionism at the intersection of genome biology. I coined the term GAGAISMO from the words 'Geometric And Genomic AbstractionISM' in October of 2016 during a short residency at the Laboratory for Computerized Languages, Faculty of Fine Arts, University of the Republic of Uruguay.
Links to my previous work can be accessed at the following links:
#GENOME SONIFICATION_
#ARTE GAGAISTA_
https://www.martincalvino.co/single-post/2016/10/12/Building-the-Artistic-Character-of-Genes-Genomes
Overview
A musical box will be constructed as the physical interface for the manipulation of synthetic sounds created by sonificating DNA sequence contexts of interest (CG content or promoter elements) and at the same time provides the user with sense of touch in a tangible form. By physically controlling sound aspects (loudness and pitch for instance) of sonified DNA the user can compose its own abstract sound art piece that its displayed in visual form on a screen as reactive abstract art. The system provides the user with interactivity through touch, sound and visuals (Figure 1).
Figure 1. Sketch summarizing concept of 'Genome AudioPlayer'. The physical interface is composed of a box containing an Arduino microcontroller inside, knobs, switches, sliders and push buttons. Each sonified DNA sequence has a corresponding physical interface on the box, a distinct sound on the speaker, and a visual interface on the screen.
Physical Interface
Although I conceived a box as the physical interface for this project, a pair of gloves with sensors attached to them can also be an interesting approach to be considered. AudioPlayers in box format are not new, however an audio player presented as pair of gloves prompting users to move their fingers as means to control which gene is played and visualized could generate a more surprising result. The use of gloves as physical interface would be a nice example of re-contextualization of an object for a different purpose.
Interestingly, both physical interfaces could be integrated to control sound and visual aspects of sonified genes. For instance, the box could control the sounds being played whereas the gloves could control the visual being projected (or viceversa) (Figure 2). As the user wearing the gloves interact with the box, automatically is affecting the visuals because of the movements of her/his hands. Alternatively only one glove can be created as physical interface so that the user is challenged by controlling the box and sounds with one hand (without glove) and the visuals with the other hand wearing the glove.
The addition of glove as second interface add the dimension of body movement as point of interaction for the user.
Figure 2. Sketch of 'Genome AudioPlayer' with two physical interfaces: glove interface controlling reactive visuals and box interface controlling sonified gene sounds.
Genome Sonification
The choice of plant genes, and in particular genes from rice, provides the user with a sense of synthetical manipulation of crop genomes, or at least their digital representation in visual and audible format. As a result of whole genome sequencing projects, DNA sequence information is stored as strings of letters (A, G, C, T) and made accessible through digital interfaces known as 'genome browsers'. For this project, DNA sequence information will be deprived of its usability context and traditional aesthetics and will be re-purposed or re-contextualized into an art project in which DNA sequence information is simply the raw material for art.
Genes responsible for photoperiodic flowering in plants are interesting examples because they determine the time of flowering and thus influence grain yield in planting areas from different geographies. This allows for the exploration of the impact of global warming on plants. A knob on the box as physical interface controlling 'pretending temperatures' will prompt the user to increase or decrease global temperature and see the sounds and visuality of genes change in response such as plants would respond by triggering or inhibiting the flowering response.
In principle, I will focus on five genes, one for each finger of the hand wearing the glove with flex sensors.
Reactive art
The artwork projected on the screen will have a geometric expressionism aesthetics and will responde to input from the physical interface(s). Each gene will have a distinct visual element on the painting that the user can recognize when changing as result of their action on the physical interface.
Synthetic sound
Sound will be synthesized using Processing's sound library oscillators objects and envelopes. The resulting sound will be played on the computer and recorded as .mp3 or .wav files using Zoom H4zN pro audio recorder. The files will be imported to Adobe Audition for further manipulation if needed. Audio files will be accessible through the physical interface upon user interaction by allowing serial communication between Arduino micro controller and Processing software on a multimedia computer.