[GSoC] GPU hardware assisted H.264 decoding via OpenGL GLSL shaders - developers only

malloc Wrote:At more than 10x the word count, I'd say no, it wasn't easier. That any the information I provided was readily available to anyone with the slightest knowledge of svn, a tool that anyone in a _developer only_ forum should possess a great amount of knowledge about. And since it's obviously not clear to those who have become lost here, let me quote the description of this forum:

i guess it's a good thing i didn't ask for any support then...

In the developer only forum, yes, no support given there.
Although there are plenty of support forums for all the various platforms. Just hit the main page, and you will see all of the support forums available there.

How about letting this thread go to sleep until there's actually some news about the topic?

Sorry to wake up this thread again but I also remembered details of a guy at MS once claiming to have implemented real time HD decoding on the xbox1 (of HD WMV material) - his name is Guobin (Jacky) Shen.

The original page is no longer available, but the link to a google cache of his web page with details is: http://209.85.173.132/search?q=cache:oyrfgSEQfbsJ:icassp98.microsoft.com/~jackysh/+xbox+generation+1+wmv+gpu+acceleration&hl=en&ct=clnk&cd=1&gl=au

Also see: http://research.microsoft.com/en-us/people/jackysh/ and http://research.microsoft.com/en-us/peop...jects.aspx

Appears to be published. If anyone on a University network has access to journal publications they can google "Accelerate Video Decoding with Generic GPU" and read up on it.

i have read the article. it's all generalities, hand waving, "experiences" and no details (guess why).

no results section with pretty graphs?

oh sorry forgot about those; yeah, pretty graphs are included

I saw a presentation on GPU assisted video decoding at GDC many years back where there were plenty of awesome graphs about how fast it could be. Most of the presentation was about how the guy wrote the DCT in a shader, the main problem of the time being that fragment programs were limited to 28 instructions. Then the rest was presentation was theory on how GPUs could totally do it some day. Waste of an hour was what it was.

Why is it every article about this never ends in "and here's the code to do it!"?

Gamester17 Wrote:This other GSoC student (for the X.org project) is this summer trying to implement GPU hardware accelerated video decoding of MPEG-2 by adding XvMC front-end support to the Gallium 3D framework, the end-result when finished should be that any hardware-specific Gallium 3D back-end device-driver that supports XvMC will be able to take advantage this as long as the MPEG-2 software video decoder features support for XvMC. You should keep up with his blog for reference:

http://www.bitblit.org/gsoc/gallium3d_xvmc.shtml

Sounds as if Younes Manton (the GSoC student for the X.org project) is making quite good progress with his project to use OpenGL GLSL shaders in order to accelerate video decoding, checkout his blog post from the day before yesterday:
http://bitblitter.blogspot.com/2009/01/y...using.html

Quote:Yes I'm still decoding video using shaders

It's been a while since I've said much about my video decoding efforts, but there are two pieces of good news to share. Both are improvements to Nouveau in general, not specific to video decoding.

First, we can now load 1080p clips. Thanks to a very small addition to Gallium and a few lines of code in the Nouveau winsys, a lot of brittle code was removed from the state tracker and memory allocations for incoming data are now dynamic and only done as necessary. The basic situation is we allocate a frame-sized buffer, map it, fill it, unmap it, and use it. On the next frame we map it again, fill it again, and so on. But what if the GPU is still processing the first frame? The second time we attempt to map it the driver will have to stall and wait until the GPU is done before it can let us overwrite the contents of the buffer.

But do we have to wait? Not really, we don't need the previous contents of the buffer, we're going to overwrite the whole thing anyway, so we just need a buffer that we can map immediately. To get around this we were allocating N buffers at startup and rotating between them; filling buffer 0, then 1, and so on, which reduced the likelyhood of hitting a busy buffer. The problem with that is obvious, for high res video we need a ton of extra space, most of it not being used most of the time. Now if we try to map a busy buffer, the driver will allocate a new buffer under the covers if possible and point our buffer to it, deleting the old buffer when the GPU is done with it. If the GPU is fast enough and processes buffers before you attempt to map them again, everything is good and you'll have the minimum number of buffers at any given time. If not, you'll get new buffers as necessary, in the worst case until you run out of memory, in which case you'll get stalls when mapping. The best of both worlds.

The second bit of good news is that we've managed to figure out how to use swizzled surfaces, which gave a very large performance boost. Up to now we've been using linear surfaces everywhere, which are not very cache or prefetch friendly. Rendering to swizzled surfaces during the motion compensation stage lets my modest AthonXP 1.5 GHz + GeForce 6200 machine handle 720p with plenty of CPU to spare. 1080p still bogs the GPU down, but the reason for that is pretty clear: we still render to a linear back buffer and copy to a linear front buffer. We can't swizzle our back or front buffers, so the next step will be to figure out how to get tiled surfaces working, which are similar, but can be used for back and front buffers. Hopefully soon we can tile the X front buffer and DRI back buffers and get a good speed boost everywhere, but because of the way tiled surfaces seem to work (on NV40 at least) I suspect it will require a complete memory manager to do it neatly.

http://nouveau.freedesktop.org/wiki/Surface_Layouts

Beyond that there are still a few big optimizations that we can implement for video decoding (conditional tex fetching, optimized block copying, smarter vertex pos/texcoord generation, etc), but the big boost we got from swizzling gives me a lot of optimism that using shaders for at least part of the decoding process can be a big win. It probably won't beat dedicated hardware, but for formats not supported by hardware, or for decoding more than one stream at a time, we can probably do a lot of neat things in time.

I've also been looking at VDPAU, which seems like a nice API but will require a lot of work to support on cards that don't have dedicated hardware. More on that later maybe.

...and in related news it now also seams like the Gallium 3D framework will be merged into Mesa 3D mainline code sooner rather than latter:

http://www.phoronix.com/scan.php?page=ne...&px=Njk4OA

Quote:Gallium3D To Enter Mainline Mesa Code
Posted by Michael Larabel on January 12, 2009

As we shared late last week, Mesa 7.3 is getting ready for release with the first release candidate having arrived. Mesa 7.3 will feature improved GLSL 1.20 support, support for the Graphics Execution Manager, and Direct Rendering Infrastructure 2 integration
. The stabilized version of Mesa 7.3 will then go to make Mesa 7.4.

Beyond Mesa 7.4 we have learned some details as to what's next: merging Gallium3D to Mesa's master branch. Gallium3D, the new graphics architecture developed by Tungsten Graphics, has been in development for quite a while but is nearing a point of stabilization. If all goes according to plan, Gallium3D will see the light of day in Mesa 7.5. Brian Paul announced on the Mesa3D development mailing list that the gallium-0.2 branch will be merged to master following the Mesa 7.4 branching.