If you havent checked them out and enjoy some choose-your-own-adventure style fantasy games, then definitely do so. However, that is not what this post is about. For the books, we have an intro bumper video that plays the first time you open the app. Our first book: Assassin in Orlandes had a landscape oriented menu screen and so we made the video a normal landscape type video. However, the book reading bit was portrait orientation. In our in-house testing we didnt go back and forth between the two orientations all that often, but (as is often the case) the users used the app differently and were constantly going back and forth and so the orientation switch was a bit annoying.
For the second book: The Siege of the Necromancer, we switched to a fully portrait oriented app, with the exception of the intro video (it was already made when we decided to change the orientation, and being a poor indie studio we didnt have the cash to re-do the animations in a vertical aspect. However, most people watch the bumper once or twice then never see it again, so that is OK. )

Now with book three, Slaves of Rema, we are finally moving to a fully portrait oriented video. This poses some interesting problems.

Since I am the developer and we are indies, I am also the video codec guy, and often the video editor as well. The animations are outsourced and the other GFX are done in house, but it falls to me to put it all together and get it ready for the iPhone.

What is the problem you ask? Well, on the iPhone you can only play video in landscape and you can really only play landscape formatted videos. On the iPad you can play videos in portrait, but they get all scrunched down. If you want full screen vertical oriented videos then you really need to make your videos sideways.

This can be a bit of a pain to do. One way to do it is to take all of you grafics and rotate them before inserting them into final cut or similar. This means that you also need to tell the guy doing the animations that you need them to be done sideways and all of that. This is a fine way to go about it, but it can make working on the video a pain in the ass. Who wants to work on a sideways video?

Instead you can make the video upright and rotate it after you have done all the editing. I find this to be the best for our workflow. That way all the graphics guys and animation guys are all working with the right orientation in mind and everything looks good. The hard bit is editing a video that is 320 x 480. Final cut, which I love, is total shite at doing anything that is not meant for TV or HD resolutions. What I use is a combination of Screenflow and Quicktime pro.

wait what?

Screenflow is a screencasting program, why would you use it for editing? Well, mostly because I already use it for screencasting, so I have it laying around. And it does a fine job of doing basic editing. Most importantly it allows you to export in any crazy-ass aspect ratio you want, and the export options are excellent. This allows me to edit the video in the aspect it will be seen at, ie portrait.

So, what I do is edit my video in Screenflow, (in the case of the new books these videos are at 640 x 960, so I can do high rez versions for the iPhone4 and the iPad, but it all works the same for 320 x 480, just be sure your final export size is the size you are shooting for)

I export the video at full rez and lossless from Screenflow (make sure that your rez is 320 x 480 or 640 x 960).

next thing is to open it in Quicktime Pro. (note, the quicktime that comes with your mac these days is not qt pro, it is just qt player. QT pro is $30 ish and is an indispensable tool to have in your arsenal if you do any amount of video work. (if you have final cut or FCX, you should already have QT pro).

Once in QT Pro, go to window->show movie properties. Select the video track, and look at the Visual Settings tab.

All you need to do here is hit one of the rotate buttons. I generally thing that rotating left (the rightmost button) is the way to go. That way the home button on the phone will be facing downward when the phone is is standard portrait orientation and the video is playing.

At this point your video should look like the one above that is tipped sideways.

Now for the fun part, the compression.

This may take a little while, and if you have a slow machine it may take a long while. The idea here is that you will probably need to try a half dozen different settings before you get one that looks good and is the size you want.

First off, see if the built-in settings work OK for your video:

Choose File->Export. Then select the Movie to iPhone setting and save it off. This will generate an .m4v file.

Once it is done compressing, open it back up and watch it. If your video still looks fine then you are done. This seems to wrok well for about 90% of the videos I run through it.
However I find that the default ‘iPhone’ settings sometimes produce terrible video artifacting for certain types of video so I have to do it the hard way. (also, if you are trying to really squeeze every last meg out of your app packages, then the you can get similar quality with half (or less) the size by tweaking the settings yourself.

In this case you need to change the export options and select ‘Movie to MPEG-4′

Have a look at all the fields on the MPEG-4 options.

Make sure that the video size is set to what you want, make sure the video format is set to H.264. After that the most important thing on this screen is the Data Rate. but before we get to that, hit the ‘Video Options’ button, and select baseline.

NOTE: if you dont do this, the video will simply not play on your device, not at all.

Ok, back the the main MPEG-4 screen and the data rate. This is where you will want to change the value and make a few different videos until you are happy with the quality and the size. I generally start at 1200 or 1500 kbits/sec (this usually produces a fairly nice quality video, but it is generally pretty beefy in terms of size.) then I go down by 250 until the quality degrades to a point where I would never use it. From there you should have a handful fo videos fo different qualities/sizes to choose from. You can tweak the data rate down as accurate as you want. If the video quality is almost good enough, then just bump up the data rate by 25 or 50 and see what you get.

cheers!
-Ben

Anyhow, it seems like everyone is showing off their clever and not-so-clever custom stands and cases for the iPad, so to day I figure I would throw mine in the ring :-) (plus it is sunday, and I don’t want to work right now)

When I first got the iPad, I had it propped up like how most of the stands work (ie like a picture frame sitting on your desk) However, this put it out past my mouse pad, and was a stretch to reach it. Since I mainly use the iPad to develop games, this solution was not ideal. (After about two hours of going back and forth between Unity, XCode and stretching to tap on the iPad, my shoulders were fooked)

Instead I needed something to get it closer to me. This meant getting it up above my mouse pad. This works well since the mousing area takes up a not insignificant amount of desk space, but not very much height. So I went scrounging in my big piles of junk. looking for a good solution.

Here is what I came up with:

This stand didn’t cost me a single cent. However, if you don’t have esoteric camera support gear lying about the house, then it might be a wee bit more expensive. (like, prohibitively so :-)

You will need:

1 x Magic arm (Manfrotto makes them)

1 x Super Clamp (also by Manfrotto, altho there are plenty of other similar clamps that work just dandy)

You will also need some way to attach the magic arm to your desk (or in this case the shelf that is just above my desk. )

I have chosen the tried and true method of: find a clamp in the shed, and clamp the bastard to the shelf. However, if you are starting from scratch there are actually many better ways of doing this (for instance, maybe get two super clamps, and just use that. It would definitely look less terrible, but again, this was just what I had around the house)

The last thing you need is some spare bits of wood. I used a bit of 1×2 and some plywood.

The design is very simple, just a single cross piece screwed to the plywood (with a nice groove cut out where the charging cable fits) and a few bits on the back to give me some clamping options.

(note I am lifting the iPad so you can see the groove, it doesnt sit up like that)

(I find the top one give me lots more ‘reach’ with the arm, but is a bit wobbly. The center clamp point is more stable)

The stand works well in both portrait and landscape mode:

As you can see, it hovers right over the mouseing area. This means I can go from typing/mousing to iPad testing by just lifting my hand. Much more ergonomic! This is a perfect solution to keep the iPad in a useful position for coding and testing. At some point I might go out and spend the extra cash for a better mounting solution (either a mounting plate, or another super clamp)

Cheers!
-Ben

• a basic intro to OpenAL on the iPhone
• A rant about OpenAL and sample code
• how to properly handle interruptions in openAL on the iPhone
• how to play lots of sounds all at once in OpenAL

One thing that people keep asking me is how to playback gigantic files in OpenAL. Things like songs or audio books. (This post has been slowly growing over the past few months and I am finally going to put the final bits in and post it!)

The answer to this problem (in case you didn’t read the title of the post) is streaming.

First off, lets define what I really mean when I say streaming. In all of the above code samples, the sound buffers were fully resident in memory. This works great for smaller sounds like button click sounds and swords clanging sounds for games and whatnot. However, for big files like full quality songs and really anything over about a dozen seconds long, putting it all into memory (especially on the iPhone) becomes a bit of a problem.

To solve this we read the data off of the disk in chunks, filling small buffers one at a time and then playing them back one after the other. All the while, in the background we are removing the old buffers and replacing them with new buffers so that we only ever have a few seconds of sound in memory at one time.

Here is a crappy diagram that I spent way too much time on, and it is still crappy:

OK, at the top we have the ‘standard’ way to do thing (the way described in all those posts linked above). Basically:

• Step1: Load the sound data from a file into an OpenAL Buffer
• Step2: Connect the OpenAL Buffer to an OpenAL source with something like: alSourcei(sourceID, AL_BUFFER, bufferID);
• step 3: Start source playing with with alSourcePlay(sourceID);

Pretty simple.

Streaming is a bit more complicated, but still pretty simple (in concept). The first three steps are really very similar to the ‘standard’ way above.

• Step A: Load in a few chunks of the big file into all the waiting OpenAL buffers.
• Step B: Queue up all the newly filled buffers into a single OpenAL source via alSourceQueueBuffers(sourceID, 1, &bufferID);
• Step C: Start the source playing with alSourcePlay(sourceID); This will begin to consume the queued up buffers.
• Step D: In a background thread, check to see if there are any used buffers, and if so then load the next chunk of the sound file into the used buffer
• Step E: Queue up this newly filled buffer to the source with alSourceQueueBuffers(sourceID, 1, &bufferID); As long as there are still bufferes queued up, the source will continue to play them

So there it is, five easy steps. Lets look at them one at a time:

Step A: Load in a few chunks of the big file into all the waiting OpenAL buffers.

Ok first thing we need to do is to prepare a place for all this state data to live. This will be kinda like preloading the streaming file, execept that we dont load any sound into memory just yet, we are just going to get ready to do that.

// this queues up the specified file for streaming
-(NSMutableDictionary*)initializeStreamFromFile:(NSString*)fileName format:(ALenum)format freq:(ALsizei)freq
{
	// first, open the file
	AudioFileID fileID = [self _openAudioFile:fileName];

	// find out how big the actual audio data is
	UInt32 fileSize = [self _audioFileSize:fileID];

First thing, we open the file. This doesnt load it into memory, it just gives us a handle to some data that we need. Specifically the file size. We will need this to calculate the number of buffers we need to play the whole file.

	UInt32 bufferSize = OPENAL_STREAMING_BUFFER_SIZE;
	UInt32 bufferIndex = 0;

	// ok, now we build a data record for this streaming file
	// before, with straight sounds this is just a soundID
	// but with the streaming sound, we need more info
	NSMutableDictionary * record = [NSMutableDictionary dictionary];
	[record setObject:fileName forKey:@"fileName"];
	[record setObject:[NSNumber numberWithUnsignedInteger:fileSize] forKey:@"fileSize"];
	[record setObject:[NSNumber numberWithUnsignedInteger:bufferSize] forKey:@"bufferSize"];
	[record setObject:[NSNumber numberWithUnsignedInteger:bufferIndex] forKey:@"bufferIndex"];
	[record setObject:[NSNumber numberWithInteger:format] forKey:@"format"];
	[record setObject:[NSNumber numberWithInteger:freq] forKey:@"freq"];
	[record setObject:[NSNumber numberWithBool:NO] forKey:@"isPlaying"];

Next we are going to make a streaming sound record. This dictionary will hold all of the state for this stream.

The OPENAL_STREAMING_BUFFER_SIZE is how many bytes of data we want in each buffer chunk. I have had good luck with 48000, this is about a second at high quality so I have plenty of time to grab the next chunk.

The bufferIndex is the current buffer that is full. For now that is zero of course.

Format and Freq we will use when we eventually setup our source. and isPlaying will let us know if we need to be refilling the buffers or not.

	// this will hold our buffer IDs
	NSMutableArray * bufferList = [NSMutableArray array];
	int i;
	for (i = 0; i < 3; i++) {
		NSUInteger bufferID;
		// grab a buffer ID from openAL
		alGenBuffers(1, &bufferID);

		[bufferList addObject:[NSNumber numberWithUnsignedInteger:bufferID]];
	}	

	[record setObject:bufferList forKey:@"bufferList"];

This is where the actual buffer references will live. In theory you can get away with just two buffers. One to play and one to fill while the other is playing. However If you get any stitch in your background thread, or do any heavy lifting on the processor that delays that refill, then you will get some nasty skipping. Instead I tend to use three buffers. This takes up more memory, but affords me a bit more robust playback.

	// close the file
	AudioFileClose(fileID);

	return record;
}

Finally close our audio file and return the record. Take that record returned from the above method and store it into a big NSMutableDictionary called soundLibrary with some key, like the name of the sound. (or in whatever data structure you want, that is how I do it, and that is how the sample code works)

We are still in Step A (well, pre step A even). We haven't loaded any buffers, but we are ready when that time comes.

Once you are ready to play your sound, we move to:

Steps A, B and C in one fell swoop

OK, lets define a play streaming sound method that does A, B and C for us. (and kick off Step D)

We would call this method when we want to actually begin playing the streaming sound, the key is whatever key you used to store the record into the sound library.

This method returns the sourceID so that the calling object can use it to stop the sound.

- (NSUInteger)playStream:(NSString*)soundKey gain:(ALfloat)gain pitch:(ALfloat)pitch loops:(BOOL)loops
{
	// if we are not active, then dont do anything
	if (!active) return 0;

	ALenum err = alGetError(); // clear error code

Just some housekeeping. I have a big boolean called 'active' that shuts off all sounds. Makes it nice and easy. Second we clear out the OpenAL errors so that anything in there will be a result of what we do in this method.

	// generally the 'play sound method' whoudl be called for all sounds
	// however if someone did call this one in error, it is nice to be able to handle it
	if ([[soundLibrary objectForKey:soundKey] isKindOfClass:[NSNumber class]]) {
		return [self playSound:soundKey gain:1.0 pitch:1.0 loops:loops];
	}

If you followed along on the other tutorials you would know that previously I had simply stored the NSNumber value for the buffer in the soundLibrary. This is a way to handle that if the streaming method gets called with the wrong key (ie a non-streaming sound) It just punts to the standard playSound method. Similarly in my playsound method I check to see if the record is an NSDictionary, and if so then it calls this method. That way you can just use the playSounds method with either streaming sounds or regular sounds and it all works dandy.

	// get our keyed sound record
	NSMutableDictionary * record = [soundLibrary objectForKey:soundKey];

	// first off, check to see if this sound is already playing
	if ([[record objectForKey:@"isPlaying"] boolValue]) return 0;

Ok, we grab the record and start to go through the state. If the sound is already playing then we get out early.

	// first, find the buffer we want to play
	NSArray * bufferList = [record objectForKey:@"bufferList"];

	// now find an available source
	NSUInteger sourceID = [self nextAvailableSource];
	alSourcei(sourceID, AL_BUFFER, 0);

	// reset the buffer index to 0
	[record setObject:[NSNumber numberWithUnsignedInteger:0] forKey:@"bufferIndex"];

Now we are going to move into Step A proper. Grab the buffer list, and get an available source. the Method nextAvailableSource simply goes through a big list of premade sources and finds one that is not being used currrently. I think I went over that in the 'lots of sounds' tutorial linked above.

Then we reset the bufferindex to 0. This is basically setting the playhead to the beginning of the sound. next, we fill the buffers

	// queue up the first 3 buffers on the source
	for (NSNumber * bufferNumber in bufferList) {
		NSUInteger bufferID = [bufferNumber unsignedIntegerValue];
		[self loadNextStreamingBufferForSound:soundKey intoBuffer:bufferID];
		alSourceQueueBuffers(sourceID, 1, &bufferID);
		err = alGetError();
		if (err != 0) [self _error:err note:@"Error alSourceQueueBuffers!"];
	}

Ok, this is pretty simple looking but there is the one magic method: loadNextStreamingBufferForSound: intoBuffer: I will get to this in a minute, but basically it grabs a chunk of the audio file based on the bufferIndex and loads it into the buffer. then it increments the bufferIndex so that the next time I call this method I will get the next chunk.
We load a chunk into every buffer in the buffer list (which in our case will be three buffers)
And here is the important part: (this would be the Step B part of the diagram)

alSourceQueueBuffers(sourceID, 1, &bufferID);

This is the magic OpenAL function call that makes this source a streaming source instead of a single buffer source. Basically it will continue to play as long as there are buffers queued up.

	// set the pitch and gain of the source
	alSourcef(sourceID, AL_PITCH, pitch);
	err = alGetError();
	if (err != 0) [self _error:err note:@"Error AL_PITCH!"];
	alSourcef(sourceID, AL_GAIN, gain);
	err = alGetError();
	if (err != 0) [self _error:err note:@"Error AL_GAIN!"];
	// streams should not be looping
	// we will handle that in the buffer refill code
	alSourcei(sourceID, AL_LOOPING, AL_FALSE);
	err = alGetError();
	if (err != 0) [self _error:err note:@"Error AL_LOOPING!"];

With our buffers loaded and queued on the source, we just need to set up the source with all the properties that were passed in. This is exactly like you would do it for a single buffer sound.

	// everything is queued, start the buffer playing
	alSourcePlay(sourceID);
	// check to see if there are any errors
	err = alGetError();
	if (err != 0) {
		[self _error:err note:@"Error Playing Stream!"];
		return 0;
	}

Ok, finally we move to Step C: and we start the source playing. From this point on, we are on the clock to keep the buffers filled up.

	// set up some state
	[record setObject:[NSNumber numberWithBool:YES] forKey:@"isPlaying"];
	[record setObject:[NSNumber numberWithBool:loops] forKey:@"loops"];
	[record setObject:[NSNumber numberWithUnsignedInteger:sourceID] forKey:@"sourceID"];

	// kick off the refill methods
	[NSThread detachNewThreadSelector:@selector(rotateBufferThread:) toTarget:self withObject:soundKey];
	return sourceID;
}

This last bit sets up the state we need to keep the buffers full, and kicks off a new thread to run in the background to keep our buffers full.

OK, before we move onto Step D lets have a look at our buffer loader method

loadNextStreamingBufferForSound: intoBuffer:

This is roughly equivalent to the method that you would use to load an entire file into a buffer for standard sound playback, only we are only going to be grabbing a small bit of the file. Luckily for us this is a pretty common thing you would want to do, so mostly all we have to worry about is keeping the state set properly.

// this takes the stream record, figures out where we are in the file
// and loads the next chunk into the specified buffer
-(BOOL)loadNextStreamingBufferForSound:(NSString*)key intoBuffer:(NSUInteger)bufferID
{
	// check some escape conditions
	if ([soundLibrary objectForKey:key] == nil) return NO;
	if (![[soundLibrary objectForKey:key] isKindOfClass:[NSDictionary class]]) return NO;

First off just some simple checks to make sure I am not trying to load a non-existent sound file, or a non-streaming file.

	// get the record
	NSMutableDictionary * record = [soundLibrary objectForKey:key];

	// open the file
	AudioFileID fileID = [self _openAudioFile:[record objectForKey:@"fileName"]];

	// now we need to calculate where we are in the file
	UInt32 fileSize = [[record objectForKey:@"fileSize"] unsignedIntegerValue];
	UInt32 bufferSize = [[record objectForKey:@"bufferSize"] unsignedIntegerValue];
	UInt32 bufferIndex = [[record objectForKey:@"bufferIndex"] unsignedIntegerValue];;

Grab the record that has all of our state information, and set up all of our variables.

	// how many chunks does the file have total?
	NSInteger totalChunks = fileSize/bufferSize;

	// are we past the end? if so get out
	if (bufferIndex > totalChunks) return NO;

	// this is where we need to start reading from the file
	NSUInteger startOffset = bufferIndex * bufferSize;

	// are we in the last chunk? it might not be the same size as all the others
	if (bufferIndex == totalChunks) {
		NSInteger leftOverBytes = fileSize - (bufferSize * totalChunks);
		bufferSize = leftOverBytes;
	}

Here we are just using our state info to figure out where in the file to look and how big of a chunk to take. If we are at the last chunk, then it may not be a full sized chunk, so we have to take that into account and change our data size.

	// this is where the audio data will live for the moment
	unsigned char * outData = malloc(bufferSize);

	// this where we actually get the bytes from the file and put them
	// into the data buffer
	UInt32 bytesToRead = bufferSize;
	OSStatus result = noErr;
	result = AudioFileReadBytes(fileID, false, startOffset, &bytesToRead, outData);
	if (result != 0) NSLog(@"cannot load stream: %@",[record objectForKey:@"fileName"]);

	// if we are past the end, and no bytes were read, then no need to Q a buffer
        // this should not happen if the math above is correct, but to be sae we will add it
	if (bytesToRead == 0) {
            free(outData);
            return NO; // no more file!
        }

Ok, here we do the actual meat of the method. We alloc some memory, and then use the AudioFileReadBytes() function to grab our desired slice of data from the big file. This loads our chunk of sound data into the outData memory. At this point we will proceed exactly like we would with a single-buffer sound.

	ALsizei freq = [[record objectForKey:@"freq"] intValue];
	ALenum format = [[record objectForKey:@"format"] intValue];

	// jam the audio data into the supplied buffer
	alBufferData(bufferID,format,outData,bytesToRead,freq);

Load out sound into our OpenAL buffer. easy.

	// clean up the buffer
	if (outData)
	{
		free(outData);
		outData = NULL;
	}

	AudioFileClose(fileID);

Do some cleanup.

	// increment the index so that next time we get the next chunk
	bufferIndex ++;
	// are we looping? if so then flip back to 0
	if ((bufferIndex > totalChunks) && ([[record objectForKey:@"loops"] boolValue])) {
		bufferIndex = 0;
	}
	[record setObject:[NSNumber numberWithUnsignedInteger:bufferIndex] forKey:@"bufferIndex"];
	return YES;
}

Finally we increment the bufferIndex so that the net time we call this method we get the next chunk of data in the sequence. If we are looping we reset the index to 0 at the end.

So, that is steps A, B, C, and the beginning of D.

Lets look more closely at our background thread now.

Step D (and E): The background thread to refill our buffers

OK, you may recall, like ten pages ago, that we kicked off a thread to refill the buffers in the background. Lets look at that:

-(void)rotateBufferThread:(NSString*)soundKey
{
	NSAutoreleasePool * apool = [[NSAutoreleasePool alloc] init];
	BOOL stillPlaying = YES;
	while (stillPlaying) {
		stillPlaying = [self rotateBufferForStreamingSound:soundKey];
		if (interrupted) 	{
			// slow down our thread during interruptions
			[NSThread sleepForTimeInterval:kBufferRefreshDelay * 3];
		} else {
			// normal thread delay
			[NSThread sleepForTimeInterval:kBufferRefreshDelay];
		}
	}
	[apool release];
}

This is a pretty simple method. Remember we are in a new thread, so we need to set up our own pool. Once we are no longer playing, then the thread ends. We make one call basically to rotateBufferForStreamingSound:. Finally if we are interrupted then we dont need to be refilling but our thread will still run (until we are terminated if that happens). To be good citizens we will decrease the amount of time we are checking the thread.

Otherwise we just come back in kBufferRefreshDelay seconds. Setting this number can be a bit tricky. If you set it too close to the actual time it takes to play your individual buffers (as you would think) then if it lags at all then you will fall behind and never be able to catch up. You want it to be more than once during every chunk, incase you need to load more than one chunk because of a slow thread. However, run it too often and you are wasting cycles. I have mine set to 0.25 seconds based on the 48000 byte buffer. I dont even remember why i came to these numbers and they might be terrible. but they do work. Feel free to tune them to your heart's desire.

Next up, the actual buffer rotator:

// this checks to see if there is a buffer that has been used up.
// if it finds one then it loads the next bit of the sound into that buffer
// and puts it into the back of the queue
-(BOOL)rotateBufferForStreamingSound:(NSString*)soundKey
{
	// make sure we arent trying to stream a normal sound
	if (![[soundLibrary objectForKey:soundKey] isKindOfClass:[NSDictionary class]]) return NO;
	if (interrupted) return YES; // we are still 'playing' but we arent loading new buffers

	// get the keyed record
	NSMutableDictionary * record = [soundLibrary objectForKey:soundKey];
	NSUInteger sourceID = [[record objectForKey:@"sourceID"] unsignedIntegerValue];	

First some defensive programming, if we are getting called with the wrong key then get out, if we are interrupted then we are not loading any new buffers, so get out (but return YES because we want to keep the thread alive)
Then we grab our ubiquitous record and start to fill in some variables.

	// check to see if we are stopped
	NSInteger sourceState;
	alGetSourcei(sourceID, AL_SOURCE_STATE, &sourceState);
	if (sourceState != AL_PLAYING) {
		[record setObject:[NSNumber numberWithBool:NO] forKey:@"isPlaying"];
		return NO; // we are stopped, do not load any more buffers
	}

First up: check to see if this source that we are meant to be loading buffers into is stopped. If it is, then we dont need to load any new buffers, and we want to return NO so that the thread finishes as well.

	// get the processed buffer count
	NSInteger buffersProcessed = 0;
	alGetSourcei(sourceID, AL_BUFFERS_PROCESSED, &buffersProcessed);

	// check to see if we have a buffer to deQ
	if (buffersProcessed > 0) {
		// great! deQ a buffer and re-fill it
		NSUInteger bufferID;
		// remove the buffer form the source
		alSourceUnqueueBuffers(sourceID, 1, &bufferID);
		// fill the buffer up and reQ!
		// if we cant fill it up then we are finished
		// in which case we dont need to re-Q
		// return NO if we dont have mroe buffers to Q
		if (![self loadNextStreamingBufferForSound:soundKey intoBuffer:bufferID]) return NO;
		// Q the loaded buffer
		alSourceQueueBuffers(sourceID, 1, &bufferID);
	}

Next up, the big event: we see how many buffers the source has processed since our last check. For every buffer that has been processed we should fill in a new one and queue it up.
Before we can queue up a new buffer, we need to dequeue the old one. We do this with a call to alSourceUnqueueBuffers(). This fills in our bufferID variable, we can use this buffer now for whatever we want. In this case we want to fill it up with the next chunk of sound data and then put it at the end of the queue.

We call our loadNextStreamingBufferForSound:intoBuffer: method and if it returns NO, then we are all done and we can get out.
If it returns YES then we can queue up the newly filled buffer.

	return YES;
}

Finally, if we have made it this far then it was a successful buffer load and we return a YES to keep our loop going.

That is about it. The way this is all built, once you cann alSourceStop() on your streaming sound ID, everything sorta cleans itself up. The buffers stop rotating, and the thread stops. You still have buffers in memory tho, so if you want to clean those up too, be sure to add that code.

To Sum Up

OK, so there are the five-ish steps to streaming sound glory with OpenAL.

Obviously I have left out all the other code you need to get this running, have a look at the articles linked at the top, they have most of the rest of it.

Also I should mention that all of this code comes from an earlier, simpler version of what I generally use as my 'sound engine' nowadays. (basically I have been working on finishing this tutorial for awhile now and my working code has evolved since then :-)
This is not to say the sample code here is necessarily inferior, in fact my current code does almost exactly the same things, only there is a bit more optimised state handling and some other things that make it faster/easier, but not as easy to explain in an already very long post. So feel free to take this code and make it better in your own way :-) (for instance, instead of using dictionaries for state, I now have some proper sound classes that hold all that state for me etc..)

Also, i should say that if this code crashes your machine, or bricks your phone or makes your cat lose all it's hair, it is not my fault, you have been warned.

Cheers!
-B

Assassin in Orlandes is nearly finished, we are planning for an early January submission to the App Store, so look for it in mid to late January!

Cheers!
-B

Chris and I just got back from the Global Connect Asia Pacific game dev conference here in Melbourne. We entered Mole into the GDAA Independent Games Awards 2009 and got selected as one of the finalists.

As a GDAA finalist we were invited to put up a table and show off our game at the conference. Chris whipped up a few posters and I grabbed a couple of the stray laptops that I have laying around and we hit the expo.



GCAP is not a gamer expo, it is a game developer expo, so all the people there are somehow involved in the creation of games. Over the two day expo, we spoke to a zillion talented and creative people and they all played Mole, and let us know what they think.

The response was overwhelmingly positive. We had people coming back two and three times to just sit and play Mole during the downtimes. We had people who just sat down to have a quick look end up staying for 30 minutes trying to dig deeper and deeper.

Best of all however was the ability to watch people play the game. We spent a huge amount of time before the conference making sure that the core game play was solid and that the game was fun, and we were richly rewarded for that effort. However, watching people play the game highlighted the areas of the user interface that were lacking. It made apparent the places where we need to change the words on the menus or add new buttons. For instance, when you go to the upgrade shop from the end-of-dig scorecard, it is not at all obvious how to get back. Similarly, when people play it for the very first time, they run out of air and get back to the surface and don’t realize that they need to hit the ‘finish digging’ button to end the dig so they can start a new one. We even found a few gameplay bugs that we will iron out.

We are currently going through and tweaking all these little bits and pieces to try to hone the user experience before we release it to the app store. But we want people to play the game so I have put up the GCAP version online so that anyone can play it for free.

If you are into it: head over to the Mole page and hit the ‘play now’ link. You will need the Unity3d player, and you can get it from the game page.

We are planning on keeping the online version free to use, and we will be updating it alongside the iPhone version, so if you come back and notice some new tweaks here and there, that is why.

We love to hear what you have to say about the game, so send me an email, or leave a comment here!

This is pretty huge news for the game development community. It is, as they say (and forgive the pun) a Game Changer.

If you don’t already know, Unity is a very powerful tool for rapidly building games. (it is billed as a 3d game engine, but it also works quite well for 2d games) One of the many features of Unity is the ability to take the game project you have built, and with the click of the mouse, build a version for the mac, a version for the pc and a version that runs in a web browser. This is excellent for distributing your game but it has other side benefits as well. I collaborate with people all over the world, and we use the in browser games to share our ideas quickly and easily, not to mention it is much easier to get your testers to test out your prototypes if all they have to do is open a browser window.

Before now, the indie version was not very expensive, less than $200 or so, but now that it is free, I think we will see a flood of new and interesting games come out in the coming months.

What does this mean for indie game development?

Well, the obvious thing is that we will see a ton more web based titles, probably lots of crap, but there will always be a few gems in there.

I think that Flash development will start to see a big challenge from the Unity guys. I know a ton of flash guys who have been eyeing unity for awhile now and this will most likely push them over the edge. dont get me wrong, flash is a great tool, and it is great for making games, but it is not a game-building-tool. Unity is designed from the ground up to make games. Here is a good post to read if you a flash dev who is on the fence: http://diamondtearz.org/2009/01/14/10-reasons-for-flash-developers-to-learn-unity-3d/

The even bigger news for me as an iPhone developer is that I think many many more people will start to use unity to build games for the iPhone.

Now, Unity iPhone Basic (the cheapest iPhone license) is still about $400. And some of the zero budget indie developers might have to think twice about that kind of money (it is totally worth it, trust me) But now you can download the Unity Free and prototype your game, or release a web version or just get to know unity and then be ready to port it over to the iPhone version. We are doing that right now with ‘Mole’ and I will be blogging about what it takes to move a prototype level game from the desktop version of unity into a working iPhone version.

I will also be interested to see how this effects the other iPhone game-focused APIs like cocoas2d. Cocoas2d and the like are still free (so still $400 cheaper than using unity to develop your game) and if you are building a 2d game it makes a lot of sense to start with cocoas2d, but I can tell you from my personal experience that building a 2d game with unity is very easy and if your time is worth anything to you then the $400 investment to upgrade to iPhone Basic is well worth the money. Also, I cant stress this enough, you can now prototype your game in Unity for free. Even if you then decide to use cocoas2d to build the deployment version (a few reasons you might want to do this, more on that in a second) you can still do a rapid prototype with Unity to figure out all your gameplay mechanics and decide if the game is fun or not.

So, Unity is great, but what are the downsides?

Well, in terms of desktop or web development, there aren’t any real downsides. Go get a copy and start making games.

In terms of iPhone development there are a few caveats.

First off, as i have mentioned, it is not free. But $400 is very cheap for what you get.

The biggest issue with using Unity Basic (and to some extent this applies to the pro version as well) to develop iPhone games is that your app size will not be under 10M (which is the size limit for apps to be able to be downloaded over the cell network). For the most part this is not a big deal, most games with any amount of depth tend to be bigger than 10M no matter the development tools used.

However, the casual games that you are trying to sell for $0.99 and you want to be an impulse buy, and so you want them to be less than 10M, these are nigh impossible to make with Unity. Why is that? Well, the unity engine is very capable, comes with all sorts of great things like built in physics, scripting, shaders, and lots of other goodies, but all that comes at the cost of size. If you spring for iPhone Advanced then you can strip out parts of the engine you are not using, and possibly get under the 10M mark, but this is a very hard thing to do.

My advice: even if you are looking to build small casual games for the iPhone, I would still suggest you get the now free Unity and prototype with it. Then once your game is working fall back on one of the other frameworks like cocoas2d to build it. However, if you have even an inkling that your game will exceed the 10M limit, then just get the iPhone version.

Cheers!
-B

http://www.snowferno.com/2009/09/23/playhaven-unity-and-snowferno/

Check it out if that sounds like something you might want to do.

Cheers!
-B

So I went through the snippets he sent me very quickly and threw out some advice. Ken came back with a few more questions and I went over the code again and noticed this:

          // use the static buffer data API
          alBufferDataStaticProc(buffer, format, data, size, freq);

Ah-HA! I said, that is probably your problem right there! alBufferDataStatic should be avoided unless you need it. When do you know if you will need it? Here is a rule of thumb: If you dont know the answer to that question then you dont need it.

(I don’t want to seem to be picking on Ken, this problem is so common that I decided I needed to do a post about it (similar to my rant about never using the sample code from Apple in a shipping app))

OK, so why do I think you should avoid it? I am sure you have heard that if you want the best performance then you should be using alBufferDataStatic() right?! No.

Well, what is alBufferDataStatic() anyway? alBufferDataStatic puts the onus of memory management for all your buffer data onto your code.

Let me repeat that: When using alBufferDataStatic() YOU are responsible for all the memory management.

That is it. That is your big performance boost.

Wait! Really? that is all it does?

Yes.

The iPhone’s shared memory space and the way it handles memory and the way your app gets jettisoned if you use too much is a very unique situation. A situation that requires that you often have to really closely handle your memory. So! That means that if you are trying to squeeze every single last byte of memory out of the iPhone so that your fully immersive 3d first-person-shooter can keep a consistent 30FPS then you will probably want to spend a few weeks tweaking your sound code so that you have very close control of your memory at all times.

For the other %99.9999 percent of apps that might want to use OpenAL (even if you are using lots of sounds) you are just adding extra headaches to your code by using alBufferDataStatic(). Here is another rule of thumb: If you are not employing a developer whose sole job is to handle the sound code in your app, then you don’t need alBufferDataStatic().

Have a quick read of the technote that mentions alBufferDataStatic().

What does this really mean?

It means this: when you use alBufferData() you load your sound data into a local buffer (one that you have to malloc) then when you call alBufferData(), OpenAL copies all those bytes into it’s own buffer that it deals with. Then you free your local storage and OpenAL now owns that buffer. The downside: for a brief moment, you have 2 copies of your sound in memory. The upside: very very easy memory management.

If you are getting jettisoned right here, during this buffer copy, then you may need alBufferDataStatic(). But more likely you have done a crap job of handling your memory elsewhere. (also it is good to note that if you are loading big sounds into memory you are doing it wrong anyway, those big sounds should be streamed).

Ok, you are absolutely, positively sure that you need to manage your own sound memory? Maybe you should be using alBufferDataStatic(), but first: here is yet another rule of thumb: if you are reading this blog post to see if you need to use it; you don’t. go to your code right now and take it out. Just take it out.

Obviously I am being a bit hyperbolic in an attempt to make a point. I dont know about your code, but here is a small list of the most common bugs I find in my code:

• Forgot to release an object properly and it leaks and eventually my app is jettisoned
• Accidentally released an autoreleased object, causing a crash
• Accidentally released an object owned by some other object, causing it to crash much much later
• malloced some space and forgot to free it, causing a leak and eventual jettison
• freed some malloced space and then tried to use it, causing my app to go batshit crazy

The list goes on. But can you see a common thread? Memory. Why would I want to add yet more memory management responsibilities when I dont need it? Why do developers always insist on making their own lives so much more difficult than they have to?

Here are a few platitudes:

• Less code == Less bugs. More code == More bugs. Therefore Less Code is always better.
• Do not add any code you dont need right now. (this is very common, you think to yourself: Hmm I am already here in this object, and I could see that I might need a method that does X in the future so I will just write it now so I dont have to write it then. Wrong. In that ‘future’ when you do need it, it will need to do Y, not X and you will have to re-write it. Or, more likely you will never need a method that does X, and you just wasted a bunch of time and added more code
• Do not prematurely optimize. Get the code working first. Then go back and identify the places you need to smooth out. If you try to optimize as you go then you will end up spending all your time making that one method totally awesome (only to realize later that method only gets called once every few seconds, so really it could be the least efficient bit of code ever and nobody would ever notice.)

So, by using alBufferDataStatic() you are breaking all three of my lovely platitudes.

Dont do it.

You are only causing yourself grief and making your code buggy and shitty. Use alBufferData(). It works. It is easy. The highly experienced engineers who work on OpenAL have made sure that there are no memory issues with alBufferData(). If you decide to port your code, alBufferData() will still work! alBufferData() makes your breath smell minty! alBufferData() will totally pull chicks.

I have put OpenAL code into about 6 apps now. A few games and a few utilities. And one ‘sound board’ style app that played 32 sounds simultaneously (which was the max at the time, probably still is) and was able to handle gigantic sounds files (via streaming) without ever using alBufferDataStatic(). Was this some herculean task of super-awesome mad coding skillzzzz?

No. Just the opposite actually. I keep all my code as absolutely bare-minimum simple as I can.

Keep your code simple, don’t use stuff you don’t need.

Cheers!
-B

and now we wait.

..

..waiting…

still waiting…

However! good news this morning! We got our first game site review!

http://nodpad.com/2009/07/24/exclusive-hands-on-with-snowferno/

Here is a choice quote: (that will most likely become our pullquote from this review:

I found that the earlier levels were quite easy, but once the game progressed, it became sheer challenge and sheer awesomeness.

Thanks Daniel and nodpad.com!

cheers!
-B

http://www.snowferno.com/2009/07/22/rc1-crasher-rc2-now-released-to-testers/

woo!