benbritten.com

Ok, So I am a mac guy. Not a zealot mind you, but I develop software for the mac, it is what i do for a living. But, why reinvent the wheel? there is a great library for machine vision stuff already available: touchlib. I suggest if you are doing any multitouch stuff, to start there. . Also, if you are a mac person, you should check out the Mac Touchlib Install Tutorial at nuigroup.

This is where i started. however, step one (download and install darwinports) was taking too long (i am very impatient :-) so i started writing my own blob detector. By the time i got to step 4 I already had a working (albeit very very poorly) blob detector using my crummy modified iSight as the input source. (note: i didn't really start from scratch. I had already done some firewire capture projects based around this excellent and easy to use framework: CocoaSequenceGrabber by Tim Omernick. So really all i did was take a quick background capture, subtract it from the source image, and then do a simple threshold to get my blobs, this took all of about 20 minutes. of course, that is the easy bit :-)

Anyhow, since I was already going down the path of doing it myself, i figured i might as well do it all the way, so I abandonded the touchlib and started my own Obj-C/Cocoa multi-touch project. (which i will post once it is in a form that I feel OK letting into the wild).

Here is a screengrab of a very very early version (let's call it 0.01 beta :-)

All it does right now is does some basic image processing to get the blobs in a nice findable state (ie monochrome, difference matte, threshold) then identifies them with a simple raster/recursive algorithm, and then build a blob event list based on how far they have moved, whether they are 'new' or whatever. (ie if they havent moved too far in one frame they are considered the same blob, otherwise they are new etc..)

I am currently working on a configuration program now that I have the basic blob detection working.

First off, I wanted to do an FTIR surface. I wanted to start small, make an experimental surface just for testing and such. My first surface only measured 280mm x 190mm.

As far as a camera, I was being very very impatient, and since I had some old external iSight cameras laying around, i decided to use one of those. One of the more important parts of the camera setup is the visible light filter. According to many good sources gleaned from the helpful forums at nuigroup.com, one good cheap way to do a visible light filter is to find some old film negatives and use the black bits at the end of the roll where they have basically been processed with no light exposure (ie a full black frame, like taking a picture of the inside of your lens cap). The hardest part of this was finding some old film. I havent used a film camera for 10 years or more. Luckily, my lovely girlfriend is a bit less of a techno-nerd than I so she still had some old photo laying about and I managed to scavenge 3 whole 35mm frames of black goodness. I taped those to the front of the iSight and got ready to go!

Now, unfortunately, while the iSight CCD is in fact quite sensitive to IR, it does have an IR filter inside it's optics. After doing some web searches to see if anyone had any experience removing the IR filter from an iSight i did run across a few people who said that there was one in there, but it was hard to get to (and had i read further i would have also noticed the advice that removing the IR optics bit resulted in an unusably short focal length, but I didnt) NOTE: if you have an external iSight, do NOT (i repeat: DO NOT) use it for FTIR or IR in general. you will be very disappointed.
I did however persevere and (since i have two of the externals and I am too lazy to sell them on eBay) went ahead and dissected my iSight. If for some reason you MUST use your iSight as an IR sensitive camera (don't) then here is how you do it briefly: take the thing apart. It is pretty self-explanatory. Remove lots of tiny screws, pry those little tabs etc.. once you get the actual CCD (the little black cube on the little daughterboard) you need to take the screws off the bottom of that as well, and pull teh little cube apart. There is a tiny chunk of coated glass about 3mm on a side, and about 2mm thick, it has a slight red sheen, that is the IR filter, take it out. (if i havent made it clear that this will really screw up your focus, take note: this will screw up your focus (ie can only focus very very close to the camera)).. once you have removed that, you can re-assemble and you are 'good to go' (ie it will suck)

Ok, on to the lighting. Since the local Bunnings (hardware store in Melbourne, AU) only had plexiglass that is about 2 mils (and you really want something thicker than 5 for FTIR) I had to switch gears mid-prototype and go with a Diffused Illumination scheme.

Here is a very early shot of the small surface with two IR illuminators and the modified iSight with film-negative filter:

Luckily, the small surface was fairly close to the modified iSight, so it isn't too out of focus. It was at this point that I ordered a unibrain fire-i OEM Board camera. I also got the 4.3mm lens without the IR coating and a wide lense (just to see). But while i waited on that, my extracraptacular prototype was ready for me to write some software...

So, awhile back i saw this cool thing by Jef Han. And it turns out the genius of it is that doing that isn't actually that expensive or difficult, so much that there is a budding international community of DYI Multitouchers (sounds kinky, i know).

Since machine vision has always been a fascination of mine (i even started a visual gesture recognition project using the isight so you could do jedi-style gestures to control the computer, but i never finished it) I decided to give this multi-touch thing a go.

This was about a month ago. Now that the second prototype is nearly done (and kinda sorta working) i decided to blather on about it on here so that I maybe my experience will help others who are trying the same things.

Ever since i drew the first 'Cuddly the Cuttlefish' my girlfriend, Leonie has been obsessed with cartoon cuttlefish. It was meant to be a one-off exercise, i kinda picked a random subject: the cuttlefish, and i drew it. The first few i drew from reference pictures (gotta love google images) were fairly realistic, but it leant itself to a nice cartoonization (if i may use that term) so i went ahead and made the original 'Cuddly' below.

This week both Leonie and I got a cold, nothing life threatening, but many many tissues were used. So, after repeated requests at drawing more cuttlefish, the cold became the impetus for the most recent Cuddly. (or should it be Cuttly?

And just for giggles: this is the sketchwork behind the 'finished' piece. (if you could call it finished, since it really is just a sketch in and of itself). All this was done on the wacom pad, in illustrator (cs2, dont have 3 yet) with the simplest round brush, 3 point diameter, and a 2 point pressure variation. I have sorta come to like that setting for most everything. (after much tweaking. it gives me a nice range of sizes, and allows for some nice flowing strokes, but without going too big). Since i mostly like just using pencils on paper when i am going analog, i find this nice minimal setting to be similar in simplicity to pencils. I rarely use anything else on the wacom (when drawing in any case, once the piece is drawn, then i might go back and fiddle with stroke weights and go in a micro-manage the points, but for the initial drawing, all 3-point round brush for me)

Here is todays sketch, it is Cuddly the Cuttlefish! The original pencil sketch is there, and the digital re-draw. I dont use live trace or anything like that (although i have in the past, and it is really cool.) Instead i place the jpg of the pencil sketch into illustrator, then trace it with the paintbrush to get the outline right. With the wacom i can use the pressure and get some nice organic outlines.

For the colors i went to this cool site, and typed in 'fish' into the palette search and found a zillion nice sea-color-palettes.