loading
Picture of The PolyPHONIC Kitchen
test_kitchen_layout_sketch_2.JPG
video_script_sketch (2).JPG
   

Our TEAM

Noah Pitts
Sara Montoro
Karl Landin
Alice Lee

University of California, Berkeley
CNM290/CS294-85
Critical Making: Materials, Protocols, and Culture

Our MISSION

The activities of the kitchen, once communal practices shared by members of a household, have become increasingly solitary in our fast-paced society. Busy schedules leave little time for family members or roommates to socialize while preparing meals, and kitchen interactions can become cold and uncomfortable. With the PolyPHONIC Kitchen, the sociality of the kitchen space is innovatively reclaimed. Sensors are set up on commonly used kitchen surfaces and, when triggered, play enjoyable music sequences that build on each other, creating a unique song. Opening a cupboard, the refrigerator, or turning on the stove becomes a social interaction, and the shared kitchen space becomes a place where members of a household may connect with each other without changing their routines.

Our HISTORY

The PolyPHONIC Kitchen was born out of our group’s discussion about kitchen silence. One member noted that cooking alone is often quiet and monotonous, and that an activity such as music might be needed to break the silence. Another member noted that in her own kitchen, an awkward silence was overcompensated for due to the presence of housemates who shared the kitchen but did not know each other well. The unfamiliarity between housemates seemed to volumize each sound that was made - cupboards slammed, dishes clashed, and cutting vegetables sounded like chopping down a tree.

We took photos of this “awkward kitchen” and decided to analyze what made sharing it with housemates such an uncomfortable space. Noting the positives of an antisocial kitchen, one group member observed that being uncomfortable with each other often means that roommates are more likely to clean up after themselves, do the dishes, and respect each other’s items. We then compared this experience to living with friends who we knew well, or perhaps knew prior to moving in. The unfriendly atmosphere of the awkward kitchen, we realized, was due to the lack of opportunities for members of the household to interact outside of the kitchen and discover things in common. The house does not have a living room or any other common spaces, and the people who live there are divided by age, area of study, and language.

With this data with we determined the four main qualities of the awkward kitchen - silence, language barriers, close quarters, and a lack of opportunities for social interactions outside of the kitchen - and searched for a solution to overcome them.




 

  
 
Remove these adsRemove these ads by Signing Up

Step 1: Parts List

Picture of Parts List
prototype_details (12).JPG
prototype_2 (2).JPG
prototype_details (4).JPG
The parts used in this prototype include:
5x 10K Potentiometer (with knobs)
4x Magnetic Reed Switch
1x Motion Sensor (not used in this prototype, but we'll suggest how you can incorporate it)
Wire
A computer
Computer Speakers (optional)
Kitchen cabinets and stove range (for this prototype, we fabricated these)

Step 2: Setting Up Your Computer

Picture of Setting Up Your Computer
We'll be using both Arduino and Processing to create a communication channel through which your hardware can tell your computer to play music.

Install the latest version of the Arduino IDE so you can program your arduino and have it communicate through the Serial Port.

Then you'll also need to download Processing. We recommend the 32-bit version if you're on Windows or Linux as the 64-bit version seemed to have trouble with the Minim library we used.

While these are downloading, set up your kitchen with the next step!

Step 3: Hooking Up The Kitchen

Picture of Hooking Up The Kitchen
prototype_details (17).JPG
prototype_details (20).JPG
In this prototype, we used the kitchen cabinets, stove top burners, and oven as triggers for the music.

The AUTOCAD file (polyphonicKitchen.dwg) is attached below if you would like to laser cut a demo version (like the one shown in this instructable) of a stove and cabinet. This file could also be used to lay out dimensions and cut by hand.

To assemble the reed switches, secure the wired end onto the inside on the opening of the cabinet. You should glue/tape/screw this switch at a location where it and its wires will not be hampered with. The very top of your cabinet or a side would be a good place. The other, unwired half should be secured to the door at a place where it will align with the wired half. Make sure that the wires are long enough to reach wherever your Arduino will be placed.

To assemble the potentiometers, replace the knobs of your (preferably fake) stove with the knobs of the potentiometers. Secure them and then solder on appropriate wires to the three pins of the potentiometer. Again, make sure these wires are long enough to reach your Arduino.

Step 4: Hardware Setup

Picture of Hardware Setup
pkitchen_schem.png
sensor_placement_sketch.JPG
sensor_placement_sketch_2.JPG
prototype_details (6).JPG

Every kitchen is somewhat different and therefore it is best to draw up a simple schematic of the kitchen that is intended to be used. There is potential to turn almost anything into some sort of input for the polyphonic kitchen. Cabinets, drawers, knobs, switches, anything really,,, Once a scheme is decided upon that is endemic to that kitchen you can use the arduino code as an initial boilerplate to bring in all of these inputs into processing which will do all of the audio playback. This prototype was presented in a classroom setting and therefore a model was used to simulate a kitchen interface. If you’re interested in recreating this model, you can follow these schematics more closely, otherwise some augmentation is expected.

Wire it all up. Make sure the pins are in the right places. If different pins are used or even different sensors, then the arduino code (in step 5) must be updated to reflect those changes. Once the arduino is programed it will need to remain connected to the computer through the serial port in order to communicate with processing.

 

Step 5: The Code: Arduino

Picture of The Code: Arduino
The Arduino communicates with Processing via Serial ports. To accomplish this, we have to read in the inputs from the Arduino and our sensors. Then format them into a string that is outputted into the Serial port. 

We have hooked up 4 magnetic reed switches and 5 Potentiometers. If you added more/other sensors into your Arduino, you should incorporate them into this code by defining which pins represent which sensors and then outputting them via the Serial port at the end.

One last thing to note: the zip file contains an Arduino sketch placed within a folder. You will need this file hierarchy for Arduino to load the sketch properly. Please place the file within a folder of the same name.

Arduino Code:

/*
The PolyPHONIC Kitchen
Sara Montoro
Karl Landin
Alice Lee
Noah Pitts

University of California, Berkeley
CNM290/CS294-85
Critical Making: Materials, Protocols, and Culture

Leave us a comment if you have suggestions and other ideas!
*/

//set digital pin (mag switch)
const int mag1Pin = 4;
const int mag2Pin = 5;
const int mag3Pin = 6;
const int mag4Pin = 7;

//set analog pin (cap sensor & pots)
const int capSensePin = A0;
const int pot1Pin = A1;
const int pot2Pin = A2;
const int pot3Pin = A3;
const int pot4Pin = A4;
const int pot5Pin = A5;

//set var (mag)
int mag1State, mag2State, mag3State, mag4State;

//set var (pot)
int capSense =0;
int pot1Val = 0;
int pot2Val = 0;
int pot3Val = 0;
int pot4Val = 0;
int pot5Val = 0;

void setup(){
  //start serial connection
  Serial.begin(9600);

  //configure digital input and enable the internal pull-up resistor
  pinMode(mag1Pin, INPUT_PULLUP);
  pinMode(mag2Pin, INPUT_PULLUP);
  pinMode(mag3Pin, INPUT_PULLUP);
  pinMode(mag4Pin, INPUT_PULLUP);
}

void loop(){

  //read the magnet value into a variable
  mag1State = digitalRead(mag1Pin);
  mag2State = digitalRead(mag2Pin);
  mag3State = digitalRead(mag3Pin);
  mag4State = digitalRead(mag4Pin);

  //read and map the pot value into a variable
  pot1Val = analogRead(pot1Pin);
  pot2Val = analogRead(pot2Pin);
  pot3Val = analogRead(pot3Pin);
  pot4Val = analogRead(pot4Pin);
  pot5Val = analogRead(pot5Pin);

  //send to serial: capSense
  Serial.print(capSense);
  Serial.print(',');

  //send to serial: pot1
  Serial.print(pot1Val);
  Serial.print(',');

  //send to serial: pot2
  Serial.print(pot2Val);
  Serial.print(',');

  //send to serial: pot4
  Serial.print(pot4Val);
  Serial.print(',');

  //send to serial: pot5
  Serial.print(pot5Val);
  Serial.print(',');

  //send to serial: pot3;
  Serial.print(pot3Val);
  Serial.print(',');

  //send to serial: mag1
  Serial.print(mag1State);
  Serial.print(',');

  //send to serial: mag2
  Serial.print(mag2State);
  Serial.print(',');

  //send to serial: mag3
  Serial.print(mag3State);
  Serial.print(',');

  //send to serial: mag4
  Serial.print(mag4State);

  //end serial line with * and start new line
  Serial.println('*');

}

Step 6: The Code: Processing

Picture of The Code: Processing
In our Processing code, we hooked up the inputs received from the Serial port to a sound file. If that certain sensor goes off, the sound will start. You may notice a slight delay in the startup of the sound. This is because we've calculated it so that sounds will start on the beat and work with each other.

The zip file has both the data files (sound clips) and the Processing sketch in the correct folder hierarchy  So just download the zip file, extract the folder with its containing files out, and open up the Processing sketch! (If something happened to it, just make sure both the data folder and the .pde file are inside of folder with the same name as the .pde file.)

Notes about the code:
1) The Serial port defined in this code must match the Serial port you have connected your Arduino to. In the setup function, we set the variable port to a Serial object representing the port it will use to read the data. Currently, it is set to the first available port, but if that doesn't work, change the second argument to the name of the port that your Arduino is connected to.
2) We have preloaded two genres of music. To switch between the two, you must set the variable tempo and count to the correct numbers as defined in the comments.


Processing Code:

/*
The PolyPHONIC Kitchen
Sara Montoro
Karl Landin
Alice Lee
Noah Pitts

University of California, Berkeley
CNM290/CS294-85
Critical Making: Materials, Protocols, and Culture

Leave us a comment if you have suggestions and other ideas!
*/

import processing.serial.*;
import ddf.minim.*;

int[] mag = {0,0,0,0};
int[] pot = {0,0,0,0,0}; //The potentiometers are not stored in order. The middle
                         // connected to pin A3 is actually fed into the Serial after
                         // all of the other potentiometers. It will be the last value of
                         // this array (pot[4]).
float volumeCab;

int mot = 0; //motion sensor. (not used)
int cap;

//sound files
String[] songs;
String[] reggae = {"t75bass1.wav","t75drums1.wav","t75horn1.wav","t75guitar1.wav","t75keys1.wav","t75synth1.wav","t75vocals1.wav","t75backup1.wav"};
String[] flamenco = {"t120bass1.wav", "t120drums1.wav", "t120guitar1.wav", "t120guitarSolo1.wav", "t120percusA1.wav", "t120percusB1.wav", "t120piano1.wav", "t120vocals1.wav"};

boolean minimStarted = false;

//IF you wish to choose a different genre, change the below two variables to the ones matching the music you want.
//REGGAE, tempo = 75, count = 1
//FLAMENCO, tempo=120, count=1
int tempo = 75;
int count = 1;

int beat, beatSum, lastMillis, error, mspb, bps;

Serial port;

Minim minim;
AudioPlayer[] players = new AudioPlayer[8];

void setup() {
  mspb = count*60000/tempo;
  bps = tempo/60;
  size(512, 200, P3D);
 
   //NOTE: the second argument in this call must be the name of the Serial Port that
  // your Arduino is communicating with the computer through. (On your Arduino Sketch,
  // go to Tools > Serial Port. The marked port is the one your Arudino is using.)
  // To see a list of available ports, uncomment the below line.
  // println(Serial.list());
  port = new Serial(this, Serial.list()[0], 9600);
 
  //be able to reach the data directory
  minim = new Minim(this);
 
  switch(tempo) {
    case 100:
      println("Tempo: 100 - No Media Currently Available");  // tempo = 100
      break;
     
    case 120:
      // FLAMENCO temp = 120
      songs = flamenco;
      break;
     
    case 140:
      println("Tempo: 140 - No Media Currently Available");  // tempo = 140
      break;
     
    // REGGAE tempo = 75
    case 75:
      songs = reggae;
      break;
  }
 
  //loadfiles from data folder and the sketch folder
  for (int x = 0; x < songs.length; x++){
    players[x] = minim.loadFile(songs[x]);
  }
 
  //Request values right away
  port.write(65);
 
  for (int i = 0; i < 4; i++){
    players[i].loop();
    players[i].setGain(-80);
  }
  //Minim.start(this);
}

void draw() {
  background(0);
  stroke(255);
  frameRate(120);
 
  //change volume of stove
  for (int i = 0; i < 4; i++){
    players[i].setGain(map(float(pot[i]),0,1023,-80,14));
  }
 
  volumeCab = map(float(pot[4]),0,1023,-80,14);
 
  for (int i=0; i< 4; i++){
    players[i+4].setGain(volumeCab);
  }
 
  //keep musical time
  if(millis()>=lastMillis+mspb){
    lastMillis = millis()-6;
    if(beat == 32){
      beat = 0;
    }
    error = lastMillis%mspb;
    print(beat);
    print(", " + beatSum);
    println(", Error: " + error + "ms");
   
    for (int i = 0; i < 4; i++){
      if (mag[i] == 1 && !players[i+4].isPlaying()) {
        //players[1+4].setLoopPoints(0, 5);
        players[i+4].play(0);
        println("player " + (i+4) + " just started");
      }    
      players[i+4].setGain(volumeCab);
    }
   
    //add to beat counts
    beat = beat+count;
    beatSum = beatSum+count;
    println("cap: " + Integer.toString(cap));
  }
}

//close all audio players and minim
void stop() {
  for (AudioPlayer p: players){
    p.close();
  }
  minim.stop();
  super.stop();
}

//called whenever there is something availble to read
void serialEvent(Serial port) {
  //Read the data
  String input = port.readStringUntil('*');
  if (input != null){
    //Splice the String into an array of integers
    int[] vals = int(splitTokens(input, ",*"));
   
    //check all values are found
    if (vals.length != 10) {
      println("ERROR: Only received " + vals.length + " values.");
      return;
    }
   
    //Put values into the variables
    if (cap != int(vals[0])) {
      cap = int(vals[0]);
      //start/stop minim
    }
   
    for (int i = 1; i <= 5; i++){
      if (pot[i-1] != vals[i]) {
        //Watch out! The middle potentiometer is printed out last and will be pot[4]
        // because it was the oven in our prototype and not a stove burner like the
        // other 4 potentionmeters
        pot[i-1] = vals[i];
      }
    }
   
    for (int i = 6; i <= 9; i++){
      if (mag[i-6] != vals[i]) {
        mag[i-6] = vals[i];
      }
    }
  }
  //handshaking
  port.write(65);
}

Step 7: Time to Make Some Music!

Picture of Time to Make Some Music!
Now that you have everything wired up and the software setup, run both the Arduino sketch and the Processing sketch.

When you open a cabinet (or otherwise set off the reed switches), a layer of music will start up. It'll play for a short period of time (or longer depending on how long you've left the cabinet open for). (NOTE: your oven must be on as well for you to hear these layers.)

When you turn on your stove, a layer of music will start up at a volume representative of how high you've turned on your stove to.

When you turn on the oven, the layers of music the cabinets started will play at a volume represented by the temperature your oven is set to.

Step 8: Final Thoughts

Picture of Final Thoughts
While our prototype made use of cabinets and the stove range, this project is highly customizable to whatever your kitchen needs may be. Just put a sensor on whatever it is in your kitchen that is used the most (the fridge, light switch, dishwasher settings, etc.), wire it up to your Arduino, send appropriate outputs through the Serial port, and set it to start whatever music you want in Processing!

Even the type of music is customizable. Just put the sound files into the data folder of the Processing sketch, change the genre to those files, and associate each sound file with a sensor input. We've found that sounds that naturally go together will create a more pleasant ambiance in the kitchen when everything is happening and multiple sounds starting at random times. 

Some future ideas for this project:
1) Setup a motion sensor to start up the system (with soft background music) upon entrance into the kitchen.
2) Using a capacitance resistor to sense distance one's hand is from something to control the frequency or tone of the music.
3) Changing genres of music based on time of day/amount of sunlight in the kitchen.
4) Integrate an Ethernet shield so you won't need your computer right next to the set up.


Disclaimer: The code provided here will not be perfect for every setup and might need modifications.

Feedback and suggestions are welcome!

Thanks!
bobert6102 years ago
what is this jam going on in that video? i must know the title.
smessud2 years ago
The idea is great, but i have a problem with the stove and oven.
Especially the "preferable a fake stove" part...
What are the operations when you use the stove ?

I suggest putting a light detector (led+LDR) at the back of the knob and a gray gradient disc glued on the stove plate. The wires can be run back in the stove through the hole. Reading the values just requires to turn the led on and off before and after reading.

Anyway, great idea and great selection of music.

PS : adding a fire alarm or burnt food detector would be a plus.....
noahpitts (author) 2 years ago
Thanks Dream Dragon, and sorry if the instructions may not have been all that clear. Basically two things are going on here.

First Arduino is reading the state of any/all of the sensors that are installed and sending that information to the pc through the serial port (USB in this case). If you program the arduino with this code (even if you don't hook up any sensors) you'll be able to see the data stream through the serial monitor in the Arduino IDE.

Processing then reads in that data from the serial port and parses it into various variables. Processing also uses the minim audio library to load and play/loop audio loops. For this example, we used some rather generic audio loops. Since we found that if audio didn't come in on a beat it quickly turned into just noise as more loops layered upon each other and sounded rather terrible. In the processing code you must specify the tempo of the audio loops that you are loading in (we used two separate genres, a lower tempo reggae (tempo = 75) and a higher tempo flamenco (tempo =120) to test programs ability to change the variable "mspb" (milliseconds per beat) dependent on the specified tempo. This is used the keep track of the beat that the music is on, and only allow it to come in on beat (we also have the variable "count" defined so that you can change if you would like the loops to come in on every other beat (count = 2), or every fourth beat (count = 4). In that sense triggering a sensor essentially sets a parameter of one of the audio loop channels. the stove top pots adjust the gain of 4 channels while the mag reed switches start to play a channel on the next beat (the gain of those channels is specified by oven pot).

I suppose that became one of the fundamental design considerations. To not necessarily link an action in the kitchen with discrete sound or tone, but to allow that input to serve as a parameter in an audio mixer. Essentially there are infinite possibilities in which sensors in the kitchen setting could or could not be tripped as well as in what order. The idea was to be able to translate that into a potentially unique audio experience every time a person/s are in the kitchen doing what they do naturally, 'touching things'; ideally becoming a social experience. This prototype uses 8 channels and when only considering whether or not a channel is on or off (not considering gain or effects or anything else) there are 256 potential combinations. It wouldn't take much expansion of the audio library and sensor inputs to dramatically increase that combination (6 options on 8 channels is well over 1.5 million combinations). It is interesting to ponder how that experience would be different for people who tend to follow routine as opposed to those who don't, say a routine morning coffee ritual versus a busy night cooking with friends.
It sounds like a really good idea, but not being the sort of person who can read the code like a book, I'm not completely understanding how this works.

Does each trigger have it's own loop, or does it play a tune when the cupboard is open?