It seems like it has something to do with the rework audio to get the 2.0 downmixing working that was commited on Feb 27th.
A built of the Feb 24th with just the remote code addition doesn't exhibit this problem. The audio does a better job of playing right away after your complete your fast forward.
This is what I noticed, if you fastfwd at 2x, the audio is silent during the fastfwd. But if you continue clicking to go into 4x, 8x, 16x, 32x, the audio returns, but is stuck in a loop. I think this makes extra work for XBMC in trying to sync the audio to the video frame, saturating the CPU cycle and/or buffer.
Once you click play again, it takes a few secs before the video and audio syncs up again. During this time, you see slow video frame rate. For the most part, on my ATV2, it last a few seconds...and then resume as normal. For others that may have slow networks, smb issues, or other ailments, it freezes and never returns.
BTW, these problems only exist on the nightly, unofficial build. So they shouldn't be added to the FAQ thread IMHO.
Contrast all this to the Feb 24th build, I can fastfwd at 2x, 4x, 8x, 16x, 32x and upon pressing play, picture and audio plays right away...all synced up. Just perfect.
Of course, the downmixing problem is still there.
Anyways, those are my observations so far. Anyone else try fastfwding at only 2x and see if it works without the rebuffering/slow framerate issue?
BTW, this particular issue only affect the unofficial nightly build. So I don't think it should be added to the stickied FAQ thread.