loading

Goals

  • Get comfortable using the Edison to prototype a hardware project
  • Build a simple Node.js application
  • Play around with the power of WebSockets and real time networking
  • Take advantage of Cylon.js ease-of-use when interfacing with buttons and sensors
  • Explore digitally synthesizing sounds with Timbre.js

Source code available on GitHub

Step 1: Ya'll Ready for This?

Required Hardware

Before You Get Started

Recommended pre-existing knowledge in your brain

  • Beginner Node.js/JavaScript experience
  • Basic knowledge of how to use microcontrollers with sensors (If you feel comfortable with the Arduino "Blink" sketch, you are mostly there)

Need to brush up on something before getting started? Here are some great tutorials to start with.

Acknowledgements

  • The code for playing sounds, as well as the .wav file we slice up for instrument sounds, is borrowed with love from the BeatBox example for Timbre.js.
  • We use Socket.IO, which makes utilizing WebSockets incredibly simple. Thanks Socket.IO!
  • We rely on Cylon.js quite heavily, which is an excellent tool for people who like to hack on hardware projects. If you are looking to contribute to an open source project, take a look at what The Hybrid Group is up to, because they are rad.

Step 2: Let's Git Edison Ready for Some Quick Iterations.

Note: If you already know your way around git and GitHub, you can skip this section by just installing git on your Edison, then creating and cloning a repo to both your development machine and Edison.


There are numerous ways to deploy code to the Edison. Some of these options include the Intel XDK, the Arduino IDE, SSH, and many others. In order to make it easy for us to keep track of the changes we are making to our code, to provide us a convenient way to update the code running on the Edison, and to give us a safe environment for testing new ideas out, we are going to employ the power of the git version control system. This will allow us to commit changes as we make them, push them to GitHub, and then pull them down to our Edison in a very small amount of time. It also gives us the added bonus of being able to incrementally make changes to our code, with the safety of being able to discard those changes without losing the progress we've made.

First things first, you will need to install the git command line tool on your Edison. SSH into your Edison, then install git using opkg.

 opkg install git

After installation, you should be able to ask git for it's version to confirm it is installed correctly.

git --version

Next, we are going to create a git repository on GitHub, to give us a convenient place to host our code so we can push from our development machine and pull from our Edison. If you don't have a GitHub account, go ahead and create one now. If you haven't used GitHub before, it may be helpful to follow their Hello World tutorial to get yourself familiar with the tools they offer.

Create a repository for your project, choosing to initialize it with a README as well as the Node.js .gitignore. Once created, clone the repo to your development machine. Then, SSH into your Edison, and clone the repo to your user folder (~).

git clone https://github.com/yourusername/yourprojectname.git ~/

Now that you have cloned to both locations, you will be able to push code from your development machine and pull it down on your Edison to get the latest changes to your project.

Step 3: Strap Up Your Boots.

We're going to start by building a simple Node.js application that will establish our socket communication between itself (running on the Edison), and a web browser that is pointed at the Edison's IP address. I prefer to write and commit code on my development machine, and only use the Edison to actually run the application. So, on your development machine, fire up your terminal and change directories to your project folder.

We will use npm to manage our dependencies. Within your project directory, run:

npm init

then follow the prompts until you have initialized your node app.

Now that we have a package.json file to add our dependencies, run:

npm install --save express socket.io

This will install two packages that we will be using, as well as save them to your package.json file so that we can quickly install all dependencies on the Edison when we are ready to do so. It's a good time to commit our changes, so go ahead and run the following commands to commit and push our work to GitHub.

git add .
git commit -am 'added express and socket.io'
git push origin master

Now that we have our ducks, I mean dependencies in a row, we are going to set up a very simple Express application that will give us a live web server and application that we can run on the Edison. Create a file called app.js, or whatever you told npm init that your main entry point file would be called.

touch app.js

Paste in the following code, which is commented to explain what it is responsible for doing.

var express = require('express')

var app = express() // initialize express
var server = require('http').Server(app) // give http module the express server
app.use(express.static(__dirname + '/public')) // tell express to serve anything inside of the public directory
server.listen(8080) // tell express to start listening for requests on port 8080

Now, create a directory within the main project directory called "public".

mkdir public<br>

Create an index.html file in the public folder, which will eventually be responsible for listening to a web socket and playing audio from the sensor data.

touch public/index.html

Paste in the following contents.

<html>
	<body>
		Hello world!
	</body>
</html>

Commit and push your work.

git add .
git commit -am 'created boilerplate express application'
git push origin master

Then, SSH into your Edison, pull down your changes, and install dependencies using npm. (Substitute the IP address with the IP of your Edison).

ssh root@192.168.1.1
cd ~/path/to/your/project
git pull origin master
npm install<br>

Now that we have the scaffolding for our express application, you should be able to run your app, then navigate to your Edison's IP address at port 8080, and you should see Hello World!

To run the app, SSH into your Edison, change directories to your project folder, and run the following command.

node app.js

When you have proven that everything is working as expected, you can quit your running app using Ctrl+C. Be sure you quit your app before trying to run it again, as you will get errors when attempting to re-bind to port 8080 with a second running instance.

Step 4: Let's Rock This Socket!

Now that we have a basic web server to work with, we are going to sprinkle in some WebSocket magic. We are using Socket.IO to manage our WebSocket communication, which plays quite nicely with Express.

Replace the contents of your app.js with the following code. Changes are commented to explain what they are doing.

var express = require('express')
var app = express()
var server = require('http').Server(app)

var io = require('socket.io')(server) // pass our http server to Socket.IO

app.use(express.static(__dirname + '/public'))
server.listen(8080)

io
  .of('/soundsocket') // establish a route to connect to using the Socket.IO client
  .on('connection', function (socket) {
	console.log('client connected') // each time a client establishes a connection, log a message to the console that the app is running on (on the Edison).
  })

Now we will need to integrate Socket.IO into the HTML file we are serving up. Open up your 'public/index.html' file, and paste in the following code, where 0.0.0.0 is the IP of your Edison.

<html>
  <script src="https://cdn.socket.io/socket.io-1.3.4.js"></script>
  <script type="text/javascript">
    var socketConnection = io.connect('http://0.0.0.0:8080/soundsocket')
    socketConnection.on('connect', function () {
      console.log('connected to socket')
    })
  </script>
  <body>
    Hello world!
  </body>
</html>

Note: We are choosing to use Socket.IO's hosted script, which is a convenience for this tutorial. In practice, you would probably want to host this file yourself.

Commit and push your changes, pull them down on your Edison, then run the app again, as directed in the last step.

At this point, you should be able to navigate to your Edison's IP address at port 8080, and should see your Edison's console log that a client has connected. In addition, if you fire up the Web Inspector, you should see a message logged to that console as well.

We have a working socket connection, however we are not actually sending anything over the socket yet. Let's change that now. Open up your 'app.js' file, and add the following code after logging the 'client connected' message.

socket.emit('test_message', 'some data')

In your index.html file, add the following code directly before the script close tag.

socketConnection.on('test_message', function(data) {
      console.log('received data:', data)
})

Commit and push your changes, pull them down on your Edison, then run the app again.

If you reload your page, our application will emit a socket message called 'test_message' with a body of 'some data'. To verify everything is working as expected, open up the Web Inspector, and check to see that our message was logged to the console.

This gives us a foundation for real time communication, and we are now ready to integrate some buttons and sensors into the project.

Step 5: Enter Cylon, the IoT Developer's Swiss Army Knife

The Edison is a powerful platform that can handle interfacing with hardware at a level that is extremely close "to the metal". Though we could use more powerful methods of accessing hardware interfaces on the board, we are going to take advantage of the Cylon.js framework, which will allow us to focus on the meat of this post: turning sensor data into music! Get started with Cylon by installing the npm package, then commit and push your changes.

npm install --save cylon

Then, update your app.js file to bootstrap Cylon.

var express = require('express')
var app = express()
var server = require('http').Server(app)
var cylon = require('cylon')

var io = require('socket.io')(server)

app.use(express.static(__dirname + '/public'))
server.listen(8080)

io
  .of('/soundsocket')
  .on('connection', function (socket) {
    console.log('client connected')
    socket.emit('my_message', 'hello world!')
  })

cylon.robot({
  connections: { // tell Cylon how we will be connecting to our devices
    edison: { adaptor: 'intel-iot' }
  },
  devices: {
	// we don't have devices to add yet, but we will shortly!
  }
}).on('ready', function(my) {
	console.log('cylon ready')
})

cylon.start()

Commit and push your changes. SSH into your Edison, pull down your changes, then install the 'cylon-intel-iot' npm package.

npm install cylon-intel-iot

Note: We now have a dependency that will not install properly on anything but an Edison. This means that you may get failures if you try to run your application on your development machine instead of your Edison, due to requiring compatibility with the MRAA library.

If you run the application on your Edison, you should now see Cylon initialize and set up it's connections, eventually logging "cylon ready" to the console.

Let's add a button! Go ahead and wire up a push button to one of the digital inputs available on the Edison, noting which pin you are going to connect to it with. If you need help wiring a button, check out Intel's excellent tutorial on wiring a momentary push button. Then, make the following change to the Cylon initialization code, replacing the pin number with the pin you have a button connected to.

cylon.robot({
  connections: {
    edison: { adaptor: 'intel-iot' }
  },
  devices: {
    button: { driver: 'button', pin: 2 }
  }
}).on('ready', function(my) {
	console.log('cylon ready')

	my.button.on('push', function() { // when the button is pushed, this callback will be triggered
	  	console.log("Button pushed!")
	})

	my.button.on('release', function() { // when the button is released, this callback will be triggered
		console.log("Button released!")
	})
})

cylon.start()

If you run the app, you should see your log statements on the Edison when pushing and releasing the button.

Step 6: Async-a-what?!

In order to send data received from the Edison over a socket, we have two different parts that need to work together, both of which are asynchronous in nature. In order to obtain a handle on a socket, Socket.IO needs a client to actually connect. This happens when you visit your Edison's IP at port 8080, and we serve up our 'index.html' file. In addition, Cylon starts up in an indeterminate amount of time, since it must initialize various devices and connections before it's ready to be used. What we want to do is start up Cylon when our application server is started, then initialize Socket.IO. Then, each time Socket.IO receives a connection, we need to add a callback to our button to emit a socket message for that specific socket, each time the button events are fired. We will need to refactor some of our existing code. Replace the contents of 'app.js' with the following code.

var express = require('express')
var app = express()
var server = require('http').Server(app)

var io = require('socket.io')(server)
var cylon = require('cylon')

app.use(express.static(__dirname + '/public'))
server.listen(8080)

// this will be called when Cylon is fully initialized, which is when we should open up our WebSocket connection.
var cylonReady = function(my) {
  io
    .of('/soundsocket')
    .on('connection', function (socket) {
      registerSocketHandlers(my, socket);
    })
}

// this will be called each time a socket is opened, so each client will receive their own events when buttons are pushed.
var registerSocketHandlers = function(my, socket) {
  my.button.on('push', function() {
    socket.emit('button', 'push')
  })

  my.button.on('release', function() {
    socket.emit('button', 'release')
  })
}

cylon.robot({
  connections: {
		edison: { adaptor: 'intel-iot' }
	}
  devices: {
		button: { driver: 'button', pin: 2 }
	}
}).on('ready', cylonReady);

cylon.start()

Update your 'index.html' to reflect the following changes:

socketConnection.on('button', function(buttonState) {
	console.log('button state:', buttonState)
})

Commit and push your changes. In order to prove this is working as expected, you can pull the app down and run it on your Edison, then reload the page in your browser. If you watch the Web Inspector console, you should see messages being emitted when you push and release the button.

Step 7: Making Noise With Timbre.js

Now that we have near real-time feedback of button presses being sent over a web socket to our browser, we are ready to turn that into some music. We are going to use a library called Timbre.js to manipulate sounds with our sensor data. Let's start with something simple: making a bass drum kick when the button is pressed.

First, we will need to add Timbre.js, and it's dependency, SubCollider.js, to our 'public/js' folder. From the root of your project folder, run the following commands to copy these files to your public javascripts folder.

mkdir public/js
curl http://mohayonao.github.io/timbre.js/timbre.dev.js -o public/js/timbre.dev.js
curl https://raw.githubusercontent.com/mohayonao/subcollider.js/master/builds/subcollider.js -o public/js/subcollider.js

Now, let's update our index.html file to include Timbre.js and SubCollider. Replace the contents of 'index.html' with the following code.

<html>
  <script src="https://cdn.socket.io/socket.io-1.3.4.js"></script>
  <script src="js/timbre.dev.js"></script>
  <script src="js/subcollider.js"></script>
  <script src="js/playsounds.js"></script>
  <body>
    Hello world!
  </body>
</html>

You'll notice we are also importing 'js/playsounds.js', which does not exist yet. Playsounds.js will be the script we use to listen to our socket messages and pass that data to Timbre.js. Create this file now, and add the following code.

var BD;

T("audio").load("./drumkit.wav", function() {
	drum = T("lowshelf", {freq:110, gain:8, mul:0.6}, BD).play()
	BD = this.slice(0,  500).set({bang:false})
})

var socketConnection = io.connect('http://0.0.0.0:8080/soundsocket')

socketConnection.on('connect', function () {
  console.log('connected to socket')
})

socketConnection.on('button', function (value) {
  if (value == 'push') {
    BD.bang()
	}
})

We have one more file we need to make available to our script, which is 'drumkit.wav'. This is a sample file provided by Timbre.js that we are slicing up to get an instrument sound. Run the following command from your project folder to copy this file to your public folder.

curl http://mohayonao.github.io/timbre.js/misc/audio/drumkit.wav -o public/drumkit.wav

Commit all of your changes, push, then pull down on your Edison.

At this point, you should be able to run your app, reload the page, turn up the volume, and hear a bass drum kick every time you press the button.

Step 8: Tired of Making the Beat Drop? Let's Add Some More Sensors and Sounds.

The joy of making a bass drum kick from a physical button press fades pretty quickly, so let's make it a little more interesting. We can use Cylon to interface with a huge amount of different sensors and devices. Until now, we have stuck to using a single button push event to kick the drum. Now, we are going to refactor our app.js code to make it a little easier to add more buttons, as well as add some analog sensors.

Before making changes to the code, let's add a few more devices to work with.

For this instructable, I chose to wire up a total of 4 buttons, a potentiometer, and a photocell. This will allow us to control more Timbre.js instruments parameters with a large range of values, instead of just the ON/OFF values from our buttons.

Now, let's refactor our code to support more devices. We want to be able to allow our client to listen to specific button presses, and specific sensor readings, instead of using a generic 'button' event. We are going to send two different events, 'button' and 'sensor', with a hash containing an id of our device, as well as the value we want to send. For buttons, we will expect a 0 or a 1 and for analog sensors, we will expect whatever the value the sensor reading is. Make the following changes to your 'app.js' file.

var express = require('express')

var app = express()
var server = require('http').Server(app)
var cylon = require('cylon')

var io = require('socket.io')(server)

app.use(express.static(__dirname + '/public'))

server.listen(8080)

var socket = io
	.of('/soundsocket')
	.on('connection', function (socket) {
		console.log('client connected')
	})

var cylonReady = function(my) {
  io
    .of('/soundsocket')
    .on('connection', function (socket) {
      registerSocketHandlers(my, socket);
    })
}

// this will be called each time a socket is opened, so each client will receive their own events when buttons are pushed.
var registerSocketHandlers = function(my, socket) {
	buttons = [my.button0, my.button1, my.button2, my.button3]
	for (var i = 0; i < buttons.length; i++) {
		var button = buttons[i];
		registerButtonHandler(socket, button, i);
	}

	analogSensors = [my.potentiometer, my.photocell]
	for (var i = 0; i < analogSensors.length; i++) {
		var sensor = analogSensors[i];
		registerAnalogSensorHandler(socket, sensor, i);
	}
}

// convenience for button specific events
var registerButtonHandler = function(socket, button, buttonID) {
	// on push, send button ID and value of 1
	button.on('push', function() {
		socket.emit('button', {
			'id': buttonID,
			'value': 1
		})
	})

	// on push, send button ID and value of 0
	button.on('release', function() {
		socket.emit('button', {
			'id': buttonID,
			'value': 0
		})
	})
}

// convenience for listening to analog read events and emitting sensor data messages
var registerAnalogSensorHandler = function(socket, analogSensor, analogSensorID) {
	// on new data, send sensor ID and value of sensor reading
	analogSensor.on('analogRead', function() {
		// ask the sensor for it's value
		sensorValue = analogSensor.analogRead()

		socket.emit('sensor', {
			'id': analogSensorID,
			'value': sensorValue
		})
	})
}

// tell Cylon which devices we will be interfacing with
var getDevices = function() {
	return {
		button0: { driver: 'button', pin: 2 },
		button1: { driver: 'button', pin: 3 },
		button2: { driver: 'button', pin: 4 },
		button3: { driver: 'button', pin: 5 },
		potentiometer: { driver: 'analogSensor', pin: 0, lowerLimit: 100, upperLimit: 900 },
		photocell: { driver: 'analogSensor', pin: 1, lowerLimit: 100, upperLimit: 900 }
	}
}

// tell Cylon how we will be connecting to our devices
var getConnections = function() {
	return {
		edison: { adaptor: 'intel-iot' }
	}
}

cylon.robot({
	connections: getConnections(), // use our getConnections function to clean this up.
	devices: getDevices() // use our getDevices function to clean this up.
}).on('ready', cylonReady)

cylon.start()

Note: Be sure that your analog sensors are connected to the Analog In ports of your Edison.


In order to consume these changes in our client, make the following changes so that your 'playsounds.js' reflects the following code:

var BD
var SD
var HH1
var HH2
var CYM

var drum
var lead
var env
var arp
var delay
var inv

T("audio").load("./drumkit.wav", function() {
  BD  = this.slice(   0,  500).set({bang:false})
  SD  = this.slice( 500, 1000).set({bang:false})
  HH1 = this.slice(1000, 1500).set({bang:false, mul:0.2})
  HH2 = this.slice(1500, 2000).set({bang:false, mul:0.2})
  CYM = this.slice(2000).set({bang:false, mul:0.2})

  var scale = new sc.Scale([0,1,3,7,8], 12, "Pelog")

  var P1 = [
    [BD, HH1],
    [HH1],
    [HH2],
    [],
    [BD, SD, HH1],
    [HH1],
    [HH2],
    [SD],
  ].wrapExtend(128)

  var P2 = sc.series(16)

  drum = T("lowshelf", {freq:110, gain:8, mul:0.6}, BD, SD, HH1, HH2, CYM).play()
  lead = T("saw", {freq:T("param")})
  env  = T("perc", {r:100})
  arp  = T("OscGen", {wave:"sin(15)", env:env, mul:0.5})

  delay = T("delay", {time:"BPM128 L4", fb:0.65, mix:0.35},
            T("pan", {pos:T("tri", {freq:"BPM64 L1", mul:0.8}).kr()}, arp)
           ).play();

  inv = T("interval", {interval:"BPM128 L16"}, function(count) {
    var i = count % P1.length

    if (i === 0) CYM.bang()

    P1[i].forEach(function(p) {
      p.bang()
    })

    if (Math.random() < 0.015) {
      var j = (Math.random() * P1.length)|0;
      P1.wrapSwap(i, j);
      P2.wrapSwap(i, j);
    }

    var noteNum = scale.wrapAt(P2.wrapAt(count)) + 60;
    if (i % 2 === 0) {
      lead.freq.linTo(noteNum.midicps() * 2, "100ms");
    }
    arp.noteOn(noteNum + 24, 60)
  }).start()
})

var socketConnection = io.connect('http://0.0.0.0:8080/soundsocket')

socketConnection.on('connect', function () {
  console.log('connected to socket')
})

socketConnection.on('button', function (data) {
  buttonID = data['id']
  value = data['value']

  switch (buttonID) {
  case 0:
    if (value == 1) {
      BD.bang()
    }
    break;
  case 1:
    if (value == 1) {
      CYM.bang()
    }
    break;
  case 2:
    value == 1 ? HH1.set({mul: 1}) : HH1.set({mul: 0.2})
    break;
  case 3:
    value == 1 ? HH2.set({mul: 1}) : HH2.set({mul: 0.2})
    break;
  default:
  }
})

socketConnection.on('sensor', function (data) {
  sensorID = data['id']
  value = data['value']

  switch (sensorID) {
  case 0:
    inv.set({interval: value * 2 })
    break;
  case 1:
    arp.set({mul: 10 / value })
    break;
  default:
  }
})

Commit and push your changes, pull them down to the Edison, and run your app. If everything works as expected, you should be able to use the buttons to hit specific instruments, and be able to modify the delay and note offsets using your analog sensors.

This gives you an example of how to consume different types of data, and either play or modify parameters of instruments within Timbre.js.

Step 9: Taking It Further: Glastonbury, Here We Come!

There are so many different types of sensors, input devices, and other electronics that can be wired up to the Edison. There are also an infinite number of possibilities for generating music from that data using Timbre.js.

My goal for making this instructable was to provide a nice foundation for exploring sensors as well as introduce the power of Timbre.js. I will leave it up to the reader to turn these *noises* into something more exciting. Head on over to the Timbre.js introduction to learn more about how to use the framework!

Get creative with the types of sensors you integrate into your project, like accelerometers, gyroscopes, flex sensors, etc, as well as with the types of sounds and effects that you can manipulate using Timbre.js!

If you make something that you think is cool, share it with the rest of us!

<p>I've done something like this with an Arduino before. Great project! </p>

About This Instructable

2,780views

12favorites

License:

More by Jbuchacher:Using Buttons and Sensors to Make Music with the Intel Edison 
Add instructable to: