[XBOX] HOW-TO encode videos in H.264 to be able to achieve playback on the Xbox

Ok, so after a little research on skipping frames, I found a fix!

It involves modifying the "mplayer.conf" file (located in: system\players\mplayer\)

Adding the line: "lavdopts=skiploopfilter=all" removes all deblocking from any file encoded with deblocking turned on (deblock=1:X:X -- X represents any value). This will produce visual artifacts upon playback, but solves the problem of not dropping frames in complex scenes!!! In most cases, this will make the video visually worse. However, files that were encoded with the "--nf" setting (or deblock=0:0:0) are unaffected and play back normal with zero quality drop.

An even better setting is adding the line: "lavdopts=skiploopfilter=bidir" -- With files encoded with Deblocking, this produces a lot less artifacting and better quality, but you can still run into a little skipping here and there. What this is doing is skipping the Deblocking filter on the B-Frames only.

And finally, one other setting is: "lavdopts=skiploopfilter=nonref". I couldn't find much info about it other than it helping with playback. I didn't test it because I read that "bidir" is the best, but I'm guessing that this setting removes deblocking on the Reference Frames (or non referenced frames?) I don't know... but give it a test!

All I did was open the mplayer.conf file into Notepad, added the line at the very bottom (under lavdopts=lowres=1,1400), saved, and uploaded to my box (replacing the old file).

Again, this seems to be only affecting Deblocked videos, but in one instance I dropped less frames on a video that wasn't Deblocked! So if you are willing to tradeoff quality for smooth playback, play with these settings! And please report back your results... maybe we can even get these to become changeable through the GUI!!!

JPSiemer, I would use cabac option, but there is a remote possibility that I might use these same encoded videos(without reencoding) on my 'future' ipod/iphone, and I heard that its better to turn cabac off for those devices.
That's why I am not using cabac.

And BTW, IMO this is the BEST thread with REAL solutions for making H264 playback smooth in XBMC. Keep it up! You guys rock.

kparuchuri,

If you haven't bought an iPhone yet, just wait a little longer. Steve Jobs is about to introduce the new iPhone model and rumors are going around that it will be faster and might even support higher resolution video. So it quite possibly might be able to handle CABAC with ease on the newer model! Don't take my word for it tho, just wait and see -- it's only about a week away...

kparuchuri,

By the way, I decided to go with vbv-bufsize=1500 for all my encodes now. Before, I was using 768 (don't ask why), and it was working out fine, until I realized that MeGUI was giving me WARNING messages in the logfile everytime. It was telling me that my buffersize was too low. So what it was doing was increasing the buffersize on the 2nd pass everytime. I don't know if this could potentially cause any problems, but I decided that 1500 was a safer number to use and I don't get these errors anymore. Plus, people on other forums thought I was crazy when I told them I was using a buffsize of 768!

Siemer,
Can you tell me what this vbv-bufsize affects?
I encode with a bitrate of 1400 or so, how will vbv-bufsize improve things?

Also, I recently removed the vbv-bufsize and vbv-maxrate settings from my encoding settings, because on one movie, I saw that the video was out of sync with audio.

So then I reverted back from handbrake latest SVN to handbrake 0.92 and removed these vbv settings, and it fixed the problem.

I am still not sure whether the older HB version fixed it, or removing these vbv options fixed it.

I've had out of sync audio before, as well. When that happened I usually just encoded the video with another application and it worked out fine. It probably is more likely a bug in the application than the settings, since the settings have worked out on every other video why wouldn't they work in this case?

As for what vbv-buffsize affects... I haven't the slightest clue -- I got that setting from reading a couple different threads of people experimenting with H.264 encoding for XBMC. In fact, half of the settings I am using is from extensive research of what other people have done. If some of the settings are outdated or are pointless, then please let me know and I will look into it.

However, I think I remember doing some tests with and without this setting, and I think setting it the maxrate to 5000 solved major skipping problems at high resolutions (when I say high resolutions, I mean the maximum safe resolution that XBMC can handle H.264 files -- 720x400, 720x384, 704x400, etc..)

Anyways, you could always do a test encode with 0.92 with the vbv settings and compare it to your encode you did without the settings. That would answer your question...

JPSiemer,
I added these two options back in my handbrake 0.92 settings, and it did solve one small issue I was seeing.
vbv-maxrate=5000:vbv-bufsize=1500

Without these two settings, whenever I skip to next chapter in VLC(windows), there is some pixelization/tearing for a second or so, and then everything looks fine. Specifying those two options solved this problem.

Now XBMC doesn't have this problem as it doesn't support chapters in mkv (yet).

so I decided to stick with those two settings.
Here are my final handbrake settings, that work very well, with no stuttering on ANY video.

-e x264 -b 1400 -B 128 -R 48 -E faac -f mkv -m -p -2 -T -x level=30:bframes=3:ref=3:me=umh:subq=5:analyse=all:direct=auto:vbv-maxrate=5000:vbv-bufsize=1500:nf

kparuchuri,

great! thanks for posting back. looks like some very basic, but powerful settings you have there...

one question tho... why do you use "level=30"? is there a specific advantage that i do not know about? i always wanted to play around with this but I skipped over it during my testing of the profile...

also, if you like handbrake, you should try out that RipBot264 program i posted about... it's so simple to use and works very well. however, the only thing that it is missing is the ability to turn off deblocking (--nf) and the ability to do a full 2-Pass encode... i cant convince the author that this would be beneficial to the program's users, but if other people were to tell him it is important to them it might just get added... if you are interested, this is an awesome little program and is updated constantly -- check out the thread...

http://forum.doom9.org/showthread.php?t=127611

This is a great program and it's close to replacing MeGUI and Handbrake for me, but to be honest, I can't until these two (very basic) options are added.

JP,
My knowledge is somewhat limited with levels, but from what I remember, the 'level' setting does not affect the encode itself, but it will appear in the metadata of the mp4 that gives the video clients (that playback the video) and indication of how complex the encoded video is.

I use level=30, which is really level '3', not 30. By definition, level 3 means:
52x480@61.4 (12)
352x576@51.1 (10)
720x480@30.0 (6)
720x576@25.0 (5)

Check the 'level' section here:
http://en.wikipedia.org/wiki/H.264

I will try the RipBot264 program and report the feedback...thanks for the tip.

My handbrake settings above produce beautiful videos(identical to original DVD) and I do recommed them to any XBMC user.

Also, those settings are a compromise between quality and encoding speed, I get about 30-35fps in the second encoding pass on my Thinkpad T60 laptop. So multicore desktops should be even faster.