2011-11-21, 18:16
... continuing a thread from HERE...
hw video decoder can run standalone, thankfully. We'd just use a normal sw audio decode, there should be plenty of MHz to go around for that..
When you talk of dropping frames, are you meaning *before* the decoder? The video decoder hw on omap4 is pretty strong, very high bitrate/profile 1080p h264 still has headroom to spare.. so decoding itself is not likely to be a bottleneck.
There are a couple options here.. and rendering is probably the first thing to figure out. Decoding could probably be handled by some existing gst patches (https://github.com/topfs2/xbmc/commits/gstreamer) with a few enhancements, assuming those aren't too much out of date.
For rendering, does auto-render on separate layer/overlay mean I have to do my own A/V sync? Or is xbmc just going to call me with the frame (handle?) to display at the right point in time? It is possible for hw to scale and blend YUV with an ARGB gfx layer. If the renderer class is just passed OSD or whatever should appear in front of video as a separate surface and allowed to combine the two layers as it sees fit, this could be an option.
If going with GL based rendering, instead of hw video overlay, I probably prefer some way to use IMG texture streaming (rather than generic GLES shader) for best performance. Either way, ideally I'd be passing NV12 (not I420) from decoder straight to renderer, along with some cropping coordinates. Raw buffer from decoder will have "codec borders" that need to be cropped out in display.
In worst case, there should be enough memory bandwidth to make an extra pass thru memory with the GPU converting YUV->RGB into something that can be used as a texture.. the existing x11/dri2 path for rendering video in totem involves more copies (when running windowed). And I guess most aren't too concerned about running off a battery..
btw, completely unrelated question.. is there some reason why xbmc appears to want to recompile the world every time I run 'make'?
davilla Wrote:There are a few ways to handle hw decode depending on how integrated the hw decoders are.
If the audio and video hw decoders are tied together via a common pts clock, then the best way to handle this is with a new internal player (aka what dvdplayer does). It's just too hard to try to integrate this into the existing dvdplayer model. Take a look at what boxee had to do for the intel cex41xx. Icky, icky, ifdef city.
If the video hw decoder can run standalone, then clone crystalhd, vda or vtb. Those are all hw video codecs that dvdplayer can use. One thing to note is that to fit the dvdplayer model, you must be able to drop picture frames or dvdplayer will not be able to keep sync with audio...
hw video decoder can run standalone, thankfully. We'd just use a normal sw audio decode, there should be plenty of MHz to go around for that..
When you talk of dropping frames, are you meaning *before* the decoder? The video decoder hw on omap4 is pretty strong, very high bitrate/profile 1080p h264 still has headroom to spare.. so decoding itself is not likely to be a bottleneck.
davilla Wrote:You can either pass up decoded picture frames or you can 'bypass' this by various means, iOS does a 'bypass' using a opaque corevideoref which is rendered in LinuxRendererGLES.
Another swing point is, are hw decoded pictures auto-rendered on a separate video plane or are they just decoded to argb/yuv and then GLES needs to render it. GLES shader convert of yuv to argb can be very slow depending on GPU, so you might have to use neon to do a yuv to argb. There's existing code to do that (for iOS).
There are a couple options here.. and rendering is probably the first thing to figure out. Decoding could probably be handled by some existing gst patches (https://github.com/topfs2/xbmc/commits/gstreamer) with a few enhancements, assuming those aren't too much out of date.
For rendering, does auto-render on separate layer/overlay mean I have to do my own A/V sync? Or is xbmc just going to call me with the frame (handle?) to display at the right point in time? It is possible for hw to scale and blend YUV with an ARGB gfx layer. If the renderer class is just passed OSD or whatever should appear in front of video as a separate surface and allowed to combine the two layers as it sees fit, this could be an option.
If going with GL based rendering, instead of hw video overlay, I probably prefer some way to use IMG texture streaming (rather than generic GLES shader) for best performance. Either way, ideally I'd be passing NV12 (not I420) from decoder straight to renderer, along with some cropping coordinates. Raw buffer from decoder will have "codec borders" that need to be cropped out in display.
In worst case, there should be enough memory bandwidth to make an extra pass thru memory with the GPU converting YUV->RGB into something that can be used as a texture.. the existing x11/dri2 path for rendering video in totem involves more copies (when running windowed). And I guess most aren't too concerned about running off a battery..
davilla Wrote:If you are really serious about this, please create a new thread in the development section of the xbmc forums. I'm sure others will chime in.
I've worked on a few 'embedded' flavors of xbmc. iOS is one, sigma is another and there are a few more than I can't talk about yet. Some use a hw decoder together with the dvdplayer model, others have dvdplayer replaced with a custom internal player to handle the hw playback details. If I would know more about exactly what panda can do with respect to hw video decode, then I can point you into the right direction.
btw, completely unrelated question.. is there some reason why xbmc appears to want to recompile the world every time I run 'make'?