P.S.

I added to the Arduino code the on board led on pin 13 to the set to HIGH/LOW in the loop and it does turn on and off depending on whether there is any red on screen, so that's a positive sign. Just need to figure out what the servo/serial port issue is all about, and after thats sorted I can see what I can do about mapping the position of the red to the position of the servo.

Serially

Working to get the Arduino and Processing to communicate, and have the Arduino use the tracking to control the servo motor. Right now I'm just working on having the Arduino set the motor on and off depending on whether the colour red is being detected in the frame. While the two do seem to be communicating i can't see what the Arduinos serial monitor is reading, and there look to be bugs with my coding as well. When I attach the servo at all the port that I am using disappears and only reappears when I pull out the firewire and reattach it. The Code im using found below:

PROCESSING:

import processing.serial.*; //This allows us to use serial objects
import JMyron.*;

JMyron m;
Serial port; // Create object from Serial class
int val; // Data received from the serial port

void setup(){
  int w = 320;
  int h = 240;
 
  size(w,h);
  m = new JMyron();
  m.start(320,240);
  m.findGlobs(1);
  println("Myron " + m.version());

String portName = Serial.list()[1]; //The serial port should match the one the Arduino is hooked to
port = new Serial(this, portName, 9600); //Establish the connection rate
}

void draw()
{
  //track colour
  m.trackColor(255,0,0,200);
 
  //minimum pixels in the glob required to result in a box
  m.minDensity(100);
 
  //fetch new data from camera
  m.update();
  //get current image and store it into array of integers
  int[] img = m.image();
 
  //define floats for red green and blue channels
  float ir,ig,ib;
 
  //first draw the camera view onto the screen
  loadPixels();
 
  //itterate through every pixel on the screen
  for(int i=0;i<width*height;i++){
       /*
        //this code inverts all pixels by subtracting their color value from the maximum
        //this is not necessary, but allows you to see better where your globs should occur
        //on your FTIR screen.
        //for some reason, human eyes distinguish white on black better than black on white.
        ir = 255-red(img[i]);
        ig = 255-green(img[i]);
        ib = 255-blue(img[i]);
       
        //add the inverted pixel to an array
        pixels[i] = color(ir,ig,ib);
        */
       
       
        //you can invert the displayed webcam colour by uncommenting the above code and commenting this line)
      pixels[i] = img[i];
  }
  updatePixels();
 
 
  //draw an averaged color block where the mouse is.
  noStroke();
  int c = m.average(mouseX-20,mouseY-20,mouseX+20,mouseY+20);
  fill(red(c),green(c),blue(c));
  rect(mouseX-20,mouseY-20,40,40);

  noFill();
 
  //initiate a 2-d array that will hold blob centers x and y position
  int[][] a;

 //fetch them
  a = m.globCenters();
 
  //draw them
  stroke(255,255,0);
  fill (128,0,128,128);
 
  //for every blob center
  print("i see" +a.length+"blobs\n");

  //itterate through all blobs
  for(int i=0;i<a.length;i++){
   
   
    //output to the console which blob this is
    print("blob"+ i + " : ");
    int[] p = a[i];
    point(p[0],p[1]);
    port.write('H'); // send an H to indicate mouse is over square
    }
  
 
  //draw bounding boxes of globs
  a = m.globBoxes();
  stroke(255,0,0);
  for(int i=0;i<a.length;i++){
    int[] b = a[i];
    rect(b[0], b[1], b[2], b[3]);
  }

  //draw edge points (same as last, but vector based)
  int list[][][] = m.globEdgePoints(20);
  stroke(0,128,0);
 
  for(int i=0;i<list.length;i++){
    int[][] contour = list[i];
    if(contour!=null){
      beginShape(LINE_LOOP);
      for(int j=0;j<contour.length;j++){   
        vertex( contour[j][0]  ,  contour[j][1] );
      }
      endShape();
     }
  }

            port.write('L');

}

public void stop(){
  m.stop();
  super.stop();
 }

ARDUINO:

  #include <Servo.h>

Servo myservo; //create sservo object to control a servo

char val; // Data received from the serial port

void setup()
{
pinMode(9, OUTPUT); // Set pin as OUTPUT
Serial.begin(9600); // Start serial communication at 9600 bps
myservo.attach(9);
}

void loop()
{
if (Serial.available())
{ // If data is available to read,
val = Serial.read(); // read it and store it in val
}

if (val == 'H')
{ // If H was received
digitalWrite(9, HIGH); // turn the LED on
}
else
{
digitalWrite(9, LOW); // Otherwise turn it OFF
}
}

And some visual aid..

Tinkering with Tracking

Playing around with processing code found at http://www.silentlycrashing.net/p5/libs/jmyron/ trying to understand the tracking and get it to do something interesting/useful. Tracking colour, hopefully be able to take this data and communicate it to the arduino to have it run a servo motor. Ideally, when an object of set colour is moved around the screen, the tracking would send its data serially to arduino and have the arduino rotate a continuous servo motor in accordance, echoing the moving of the coloured object.

Tracking Your Progress

So I downloaded Processing to see what I could do with it, and in looking around I found a link to a library for it that lets you track motion called MYRA. So i was wondering if maybe I could use this to track and recognize guitar chord shapes of a hand, and then feed that info to a program to play a chord...



 ...but so much work to do with so little time left. Need to find out how the tracking works, and then program it to recognize the hand shapes, and then feed out that input to another program (maybe arduino?) that will then set off the pre-set chord noises (this could either be through a midi program or even just pre-recorded chord files or even solenoids that will hit notes depending on what the chord requires).

Button Masher

LEDs and Controller fret buttons. Seems overly complicated, so not sure if I'm going to continue down this rout...

 

...though the idea of making the controller into a "lo-fi" midi-synth where the midi is just a xylophone with solenoids or servo motors in it seemed charming and funny enough to be a possible place to take this. I do enjoy the combination of technological digital devices and physical mechanical pieces fusing together.

Tuning in, Turning On

My final project is still not certain, but I want to involve the Guitar Hero controller I picked up for $4 at a second hand store. Below are two videos of involving the controller.

The first I downloaded Miditar Hero from http://miditarhero.wordpress.com/. It allows you to use the controller on your computer as a midi syth, and by downloading LoopBel it can communicate with an audio software program such as Adobe Audition, GarageBand, or Fruity Loops (which I used to test out).



After messing around with that I took apart the controller.....

...and looked at all the parts: