More openAL tidbits for iPhone

Hey All,

my last post (which was a zillion years ago it seems) was about openAL. It has gotten quite a bit of traffic in the past few months, so apparently openAL on the iPhone is something people are interested in. So i am going to post this which is a few more tidbits that I have discovered over the intervening months about openAL.

First off, i have noticed that the vast majority of people are taking apple’s sample code ‘SoundEngine’ and plopping it directly into their code, and then spending countless hours and days trying to get it to work just right for them.

Here is my advice:

Dont.

Look, I love the apple engineers, I love the work they do, but sample code is just that; a sample. It is meant to show you all the various different ways you can do something. the SoundEngine code does way more than you should ever really need. Your best bet is to start from scratch, figure out what your sound needs are and build you own sound controller object. If you do this you will be in a much better place to debug (because you will know how it all works) and you will be able to add only the code your app needs, and less code means less bugs.

What you say? What about code re-use?! I shouldnt reinvent the wheel, I dont fall into the ‘not invented here’ trap!! Heresy!

to that I say: you are re-using code. you are re-using the openAL codebase. but it turns out at the level that you are gluing openAL into your app, you should strive to make your own implementation geared towards what you need.

This is especially true for the iPhone. If there is any code in your app that you can get rid of by pre-formatting your sounds then you should be doing that. There is absolutely no reason that your sounds should be in more than 2 formats (ie an uncompressed format for your short sounds and a compressed format for anything longer) if you are checking the type and format of your sound files at runtime you are doing it wrong. Pick a sample frequency and format (stereo/mono) that works with all your sounds and make them all the same. This will simplify your code immensely and simpler code == less bugs == better performance.

Anyway, that was a bit of a rant. But I see so many posts to the apple dev forums that are like: “I am using SoundEngine from the crashLanding app and i am having this problem…”

Really, openAL is not that hard. I spent about 2 hours with the programmer guide and I got my simple sound controller working and wrote a tutorial since i was such an expert by then. That and the resulting code was about a tenth the size and complexity as soundEngine and I have had very few problems with it (and the problems I have had were easy to track down and debug because there just is not that much code there.

Anyhow, I really did mean to have some technical advice in here, and not just rant, however this is getting long, so it may have to wait until another post. :-)

-b

This entry was posted in code, iPhone, openAL. Bookmark the permalink.

6 Responses to More openAL tidbits for iPhone

  1. illgb says:

    I am one of the many who used the CrashLanding source to build out openAL support for my app. Worked great until I moved off of the simulator and onto an actual device :).

    I plan on rewriting my audio implementation using the guidance you provided in the previous post (and the philosophy you provided in this post :)).

    Thanks for both.

  2. Ben says:

    Hey illgb,

    you are very welcome, and I hope it goes well!

    Cheers!
    -b

  3. illgb says:

    ben,

    hey, it’s been going somewhat well – my big problem now is clicking / popping when i start and stop the sounds in openAL.

    i’ve tried shenanigans like adjusting the AL_GAIN over time, moving the sound away via AL_POSITION over time, but none of them are producing satisfactory results – and i want to avoid thread/sleep nonsense if i can.

    any thoughts on how to get clean, pop-free action with alSourceStart and alSourceStop/Pause ?

  4. Ben says:

    Hey Illgb,

    When everything is set up and working you should have no trouble with popping or clicking. there is no bug or issue within openAL that I am aware of that will cause that. Popping and clicking (in my experience) are usually issues with either the actual sound file or with the way you are loading the data into the buffer. (and ultimately they are caused because the buffer has non-audio data in it for some reason, and openAL is more than happy to try and play that data which sounds like popping and clicking.)

    One way to tell if this is happening is to make a sound with silence at the beginning for a few seconds then actually print out the first couple hundred bytes from the buffer and see if they are not zero or close to zero. Most likely there will be a handful of bytes that are just garbage at the beginning of the buffer, this would point to improper parsing of your audio file.

    In my first foray into openAL (and audio in general, having not done anything this low level before in the audio world) I was loading the sound files basically directly into the buffer. (as raw NSData from the file) this actually worked and the sound played, but the files have non-audio data at the head and this was also getting played as if it were audio data. well, this sounded a lot like a pop at the beginning of the sound. this may or may not be your issue. Have a look again at the tutorial I posted awhile back (linked above) and make sure you are following the instructions on loading sounds into the buffers (ie get the file, then find the data size, then grab the audio data etc..)

    the other thing to check is to make sure you are using afconvert properly and the values you are specifying the same values in the openAL code when you tell it the frequency and the format.

    Hopefully this helps!
    Cheers!
    -b

  5. JohannesVonLuck says:

    illgb,

    I’ve been programming with OpenAL over the course of several years, without any hardships, and now, on the iPhone, I can and _will_ say that the implementation is broken (popping/crackling), and I know (somewhat basically) why.

    One thing that is happening is that OpenAL is relying too much on the NSMainLoop thread to do its mixing, in addition to all the other normal iPhone operations (like talking to the OS). When you start working with a multi-threaded environment, or even a process intensive single-threaded environment, you can block the main thread that NSMainLoop is using and your audio will immediately die. Try it yourself is you doubt me.

    This is not very well modeled on the simulator (in fact, I never get a pop or crackle on it), but happens all the time on the device itself. The difference is the implementation of the ARM’s NSMainLoop/OpenAL.

    When you have too much work going on in the other threads (especially with GLES content or other process-heavy work), then popping and crackling noises start to occur, regardless of anything you’re doing. It doesn’t matter if you’re reading from a WAV (thus not on-the-fly decoding), using alBufferDataStatic (which, btw, docs don’t do a good job of mentioning, but this function relies on you managing the memory segment passed in yourself (i.e. rather than doing a memcpy, the pointer you pass in is copied (essentially a retain) and thus must stay persistent with such data)), modulating gain up and down, or anything else. What matters is that NSMainLoop’s thread isn’t being bombarded with crap and blocking execution from proceeding. Without that thread’s ability to loop, your audio _will_ suffer.

    I’m still looking into seeing if a fix is available, but it appears that the only way to alleviate things, as of now that I have discovered, is to jump the main thread’s priority level to >= 0.8, and/or use smaller mixing buffers (via size, either by sampling frequency or content). Hopefully OS 3.0 fixes these issues.

    It’s funny to mention that Apple’s original 2.0 simulator didn’t even support OpenAL. You had to continually change it to 2.1, and then the bug started happening where you would have to change it on the debugger to actually get the 2.1 simulator used (even if you had the correct one selected in the main XCode window). Context creation would fail consistency in these cases.

    Another thing the OpenAL doc doesn’t mention that well, but the ALC_REFRESH parameter is used to tell how many passes AL does, per second (thus in Hz), to mix in new sounds. So, for example, if you start playing a sound and you only have 1 pass per second set as ALC_REFRESH, your sound could possibly start up to a second later. For audio intensive apps, tuning this value up will produce more immediate results, but for other apps (including games) this can be a low value, like 5 to 15.

    On the side, if you are looking at using a decoder for a compressed audio stream, I’ve been using the Tremor decoder (integer based ogg/vorbis decoder) on the iPhone will good results (especially if you build with ‘_ARM_ASSEM_’ defined, which enables hand crafted ASM routines that speed things up a good bit). FYI, the usual ogg/vorbis library is incredibly slow, and I would continually get buffer underruns while using it.

    Ben,

    – “I am using SoundEngine from the crashLanding app and i am having this problem…”

    Like you make great mention of, anybody who actually uses source code from samples is going to get bit in the butt. Anybody who has spent enough time working in industry knows right well to never rely on sample code for any main implementations. Why? Because the sample code is build just barely enough to just barely enough work. Script kiddies are easily distracted and think that “oh well it worked for this small app, it will work for my super MMORPG!”, well, it won’t. When you half ass your engineering, you get half assed results that just barely work, and if at all work is a simple matter of in-this-particular-case-only (which is absolutely fine for small test apps), not will-do-everything-you-want-and-more (which is needed for any truly useful app). People who use sample code, regardless then, are, without a better word to describe it, stupid, and deserve the problems.

    – “as raw NSData from the file”

    Or better yet, actual raw data, in the form a char* (or similar). Reason being is that if you’re in an audio decoder, chances are your in the 80/20 tier and you want to avoid any extraneous Objective-C message dispatches like the plague.

    – “One way to tell if this is happening is to make a sound with silence at the beginning for a few seconds then actually print out the first couple hundred bytes from the buffer and see if they are not zero or close to zero. Most likely there will be a handful of bytes that are just garbage at the beginning of the buffer, this would point to improper parsing of your audio file.”

    This is a good point, but this only shows that your decoder or read routine is faulty, which if it were would be immediately noticeable in the case of audio. Besides, the data here is meaningless – those zeroes may very well _be_ a part of the sound. The only way to truly test is to dump to a raw file and open in a program, like Audacity, that supports raw header-less binary data reading.

    – “the other thing to check is to make sure you are using afconvert properly and the values you are specifying the same values in the openAL code when you tell it the frequency and the format.”

    One should really be appropriately doing this at design/export time when building your audio, not during runtime.

  6. Ben says:

    Hey Johannes!

    Thanks for that very long and well thought out reply! probably just doubled the usefulness of this blog post :-)
    First off I want to say that you probably have wayyy more openAL experience than I have :-) My background is more on the Cocoa/Mac side of things, so dont take any of the following as me trying to disagree with you :-)

    A few things:

    “- “as raw NSData from the file”

    Or better yet, actual raw data, in the form a char* (or similar). Reason being is that if you’re in an audio decoder, chances are your in the 80/20 tier and you want to avoid any extraneous Objective-C message dispatches like the plague.”

    same thing really, NSData is a just a nice Obj-C wrapper for a void* data buffer, as long as you are accessing the data via the (void*)bytes method, then it is equivalent to mallocing and filling your own data buffers via C calls. (and, in my opinion much easier to use :-) but ultimately the same thing :-). I use the C malloc stuff in my openAL code, but I didnt want to give anyone the wrong idea that NSData isnt just raw data.


    – “the other thing to check is to make sure you are using afconvert properly and the values you are specifying the same values in the openAL code when you tell it the frequency and the format.”

    One should really be appropriately doing this at design/export time when building your audio, not during runtime.

    I absolutely agree with this, and that is what I meant in the original post, so yes, absolutely this is meant as an exercise for the developer and not the runtime code :-)

    On the topic of popping and crackling (this isnt meant so much as a response to Johannes, more of a general anecdote of my personal experience): I really have not had any issue with this, and I have done same fairly intensive openGL process during playback for one project, and on some other projects; some heavy core animation pixel pushing which brought everything else to it’s knees but did not affect my sound quality.

    The two times I have fought with the popping and stuttering were: first when there was a bug in my file parsing code and I was reading in the header info as audio data (which, when played back sounded like a ‘pop’ right at the beginning of the sound (and obviously at every loop)).
    and second when I was doing some streaming playback and my buffer refill code was, let’s say; less than optimal :-)

    I fixed the streaming issue by using 3 buffers instead of 2, making sure my refill code was not running on the main thread, and heavily overpolling. (so, if my buffers were half second buffers, I would have my buffer refill thread check in every third of a second with a very fast shunt if there were no buffers to fill. this way I was (almost) always at least one full buffer ahead. that way whenever the main thread got hairy then my buffer refiller could be blocked for almost a second and a half (in the above scenario) and not have any issues. (which was rare, but not unheard of)

    I also ‘tune’ my buffer sizes to the particular time and sound that I am playing (so if I am trying to play a big streaming sound and I know that a whole lot of other things will be going on at the same time, I tend to make the buffers that much bigger and pre-fill them when possible) and other times I try to keep them slim and svelte to keep my memory footprint down.

    This may or may not have anything at all to do with the popping you are experiencing, but after doing those few things I have had no issues at all with openAL.

    The other thing that may be different is that as a general practice on the iPhone I make sure that all my sounds are absolutely the lowest quality I can get away with. (i know this sounds horrible! :-) I am as much of an audiophile as the next guy, but rarely if ever do you need your sound effects to be stereo 48k full quality (and this is one of the big mistakes that many people make), so I tend to set them to 8k mono first then slowly increase the quality till I (and my testers) are happy.
    Also, I never play any ‘songs’ though openAL. I do agree that the ‘soundtrack’ songs for the apps should probably be 48k stereo (otherwise your composers get shirty :-) but I always design so that any big musical numbers play via the hardware decompressor, since that has it’s own little world and doesn’t affect the running of my app (very much) (and generally have a ‘turn music off, turn ipod on’ kinda thing to allow people to play their own tunes if it makes sense for the application).

    As far as the main thread blocking openAL mixing, I do believe you, and I will have to try it, but it has not been a problem for me as of yet. (could be that I have just been lucky :-)

    Cheers! and thanks again for the very informative post!
    -B

Leave a Reply