BBTouch design15 Jun
At this point the BBTouch codebase should be ready to build apps on. There is plenty more work to be done to make it more robust and friendly, but everything should be there for a basic multi-touch system.
I want to talk briefly about the design, and what is going on so that others can build apps on top of this base if they want to.
There are three main objects:
the ConfigurationController
the InputController
the BlobEventController
The Config Controller holds all the 'global' configuration values. this is basically 2 things: the raw video size (which is currently hardcoded at 640x480, but that will change) and the region of interest (in relation to the raw video feed)the config controller's main responsibility is to convert from raw camera coordinates to 'screen' coordinates. Which is currently a normalized value between 0 and 1 relative to the region of interest.
The InputController handles the raw video and makes sure that it gets to the blob detector and that the blob event controller gets any found blobs. The raw video is currently using the SequeceGrabber API, and the image format is NSImage based. The IC is responsible for setting up the camera object and the blob detector object. For every frame from the camera, the IC sends the raw data (in the form of an NSImage) to the blob detector. It then queries the blob detector and sends any 'raw' blob objects to the BlobEventController.
The Blob Event Controller takes the raw blob data and 'tracks' the blobs. it decides whether the raw blobs are continuations of older blobs or new ones. it send messages to it's delegate whenever there is a blob down, a blob drag, or a blob up. It also tells the delegate when the current cycle is finished.
That is the basic design. There are a few more objects that are interesting to note:
The Simple Event Delegate: this is where you would start if you wanted to have the system generate any type of event, or if you wanted to have it send TUIO events down a serial stream, or whatever. Currently the Simple Event Delegate just generates some NSNotifications for each event, effectively passing them onto anyone who wants to listen for them (in the same app).
Along with that is the Simple Event View, which uses the Notifications generated by the Simple Event Delegate to draw the blobs into it's bounds. It is very simple however, and I would suggest that you look to the ProjectionConfigView for a slightly better subclass. (the ProjectionConfigView forces the same aspect ration as the ROI, which is not necessarily what you want, but is an example of how to use the blob data in a slightly different way.
right now there is not a good full-screen projection config. that is what I am going to work on next. However, that is not strictly necessary for you to build your apps.
Hopefully this all helps. Also, I have tried to be very verbose in the code comments, so check those out as well
all code is available at http://code.google.com/p/opentouch.
My full name is Ben Britten Smith.
Good work man!
I just checked out your code from code.google.com.
It seems that BBCIImageBlobDetector.h and BBCIImageBlobDetector.m are missing, could it be that you forgot to check them in?
I am looking very forward to use your app.
hey mike,
oops, sure enough. They are not done yet (not even remotely) and I hadnt added them to the repository yet. I bet they are showing up in the project tho? oops!
I will go ahead and commit them even tho they are just stubs at the moment.
cheers!
-b
Hello Ben,
since the files are already referenced in the project but just stubs, can I remove them from the project in order to compile? I just tried to checkout the files again, but they are still missing.
BTW: I have some problems with the app on my MacBook. Up to now I can configure the camera using the config screen, but in the main window (where the camera picture shows up) there is only a gray window shown :-(
Are there some configuration files stored somewhere in the /Library dirs which needs to be deleted?
Regards and thanks for your work on BBTouch,
Michael.
Hi Ben,
just an addition to my latest post.
When I remove the missing files from th project it compiles and run, but when I click on the Cam Settings button the app just hangs here.
Any Ideas?
Thanks,
Michael.
Hi Michael,
sorry about the stubs. I am committing them to the repository now. Oops!
Also, you should definitely delete the pref file from the old installation. I hadnt updated my bundle ident yet, so it is probably still called com.yourcompany.BBTouch.plist in the Library/Preferences folder. that may solve your problem with the hanging. Also, in the main window, if the ‘show raw video’ is not checked you wont see anything.
I have noticed that the sequence grabber does take a few seconds to initialize itself (on my macbookpro in any case) and if you hit the cam settings before it has done it’s thing, then it can hang. So be sure that you can see the raw video first, then hit the cam settings.
Hopefully that will help!
let me know if you are still having trouble.
oh! Also check the console log. if the SG cant initialize for some reason it will log an error like : “QTNewGWorld() returned -5205″ or something like that. you can then check the CSGCCamera object and see where it is failing.
The last thing you can try if none of that works is to open up iChat and hit the camera prefs, make sure it is on your built-in iSight. Then open photoBooth, it should default to the next camera you have attached (most likely the one you are using for your setup). once you can see an image in photobooth, then quit photobooth and try to run BBTouch. I have found that sometimes when I am doing lots of debugging, sometimes i cant get the camera to come back. (maybe it doesnt get put away properly when i crash or have to force quit, or who knows) but the photobooth trick seems to bring it back. I am still trying to figure out how to do that in my code :-)
hope this all helps!
cheers!
-b
Hi Ben,
deleting com.yourcompany.BBTouch.plist did the trick.
The camera settings dialog also works.
Thank you very much for our help,
Michael.
This thing is beautiful! I especially love the area selection, I have a few different tester MTmini’s that are designed pretty roughly, that helps a lot.
It does hang after awhile though, in case you didn’t know this already:
After adjusting the camera settings, when I go back sometimes BBtouch hangs and I get a warning: ‘BBBlobDetector’ may not respond to ‘-handleGoodPixel’
screengrab of error:
http://www.flickr.com/photos/27013844@N08/2585282128/
osx leopard 10.5.3, xcode 3.0, intel
theScreamingDrills,
Thanks for that. very helpful! Altho the handle good pixel isnt the issue, it looks like the problem is in the line above. I am guessing that your ROI rectangle is perhaps too big?
I need to add a maximum size so that it is always at least one pixel inset from the main frame on all sides.
The image pre processor puts in a line of black pixels all around the ROI. this is necessary for the blob detection algorithm, otherwise it sometimes tries to grab data outside of the image bounds and goes into the weeds (as you have seen) I will try and fix that today.
Cheers and thanks!
-b
Yeah, that’s what caused it. Good to know, I was starting to get frustrated. I’m not sure why I’m messing with this anyway because it’s not like I can write anything to utilize this, but it’s just so damn intriguing!
hehe, well, hopefully soon i will have at least a few sample apps that you can play with so there is more than just ‘hey wow look, my fingers!, but on the screen!”
and then hopefully there will be more people as well, and so on :-)