To test each of the note mappings and how they react to various setting and algorithm changes I started by creating a few test images. Each of these images were tested against all of the selected ntoe mappings, the videos below run through them in the order of Scriabin, Newton, Octaving then Rimington.

This first test is called “Test Pattern” and is just that, a selection of colours placed to test the algorithm out. In this sample the tempo is fixed and all of the MIDI notes are within one octave as these options have not been implemented. Opening with the solid red bar at the top the notes do not change, instead there’s a prominent rhythm which leads to a section with not much musical interest. As the piece progresses to the gradient section the lightness starts to build intensity and suddenly it is thrown into a patterned section which causes the bass and piano to play a 5 note repeated phrase. These sort of repeating patterns seem to create very musical phrases and would be a great starting point for images created for sole purpose of turning into music.

In this next piece I have extended the functionality to take into account the average brightness of the current notes and alter the tempo to better suit the mood, a higher average brightness will result in an increase in tempo and a lower average brightness will reduce the tempo. During the first run through using Alexander Scriabin’s note-colour relationship multiple octaves were enabled using a technique described by Cretien van Campen in relation to Alexander Rimington’s work. The brighter the colour the higher the note’s octave. (Campen, 2008, p. 49) I found this really opens up the piece and allows the notes to be spread further apart so this feature will be extended to all of the note mappings. Scriabin’s note-colour relationship really shines in this piece with the synthesiser and the piano coming in with a very melodic introduction to the piece. This is in contrast to the other note mappings such as Isaac Newton’s which tend to sound dissonant when similar colours are placed next to one another.

Using ideas developed from the previous two pieces, this piece brings together all of the techniques. Multiple octaves are now part of all note mappings and the tempo is free to change as the brightness and intensity of the piece evolves. Once again Scriabin’s note-colour relationship really shines with this sort of photographic image, during the start of the image there are many pixels of similar colour that when converted into notes using a “sequential” note mapping such as Alexander Rimington’s come out sounding very muddled and dissonant even when multiple octaves are implemented, whereas Scriabin’s mapping spaces these notes out across the circle of fifths resulting in a much more harmonic set of notes.

Small adjustments were made to the algorithm after each test to allow brightness and saturation to interact with the piece on a larger scale, the implementation of tempo modulation and multiple note octaves has resulted in much more musically interesting pieces.

Final Pieces

For my final pieces I have changed some of the instruments to better match the theme of the images and used all of the techniques developed using the test pieces.

Title: Portsmouth at Night
Note Mapping: Alexander Scriabin

Title: Printed Circuit Board
Note Mapping: Direct Octaving

Title: Paris Locks
Note Mapping: Alexander Rimington

Title: Gradient
Note Mapping: Isaac Newton



Campen, C. v., 2008. The Hidden Sense: Synesthesia in Art and Science. Cambridge, Massachusetts: The MIT Press.

Categories: Music

Leave a Reply

Your email address will not be published. Required fields are marked *