[sarbes]

Please excuse me if this thread belongs in the "Feature Suggestions" area.

I've made an custom blend filter for FFmpeg/libavfilter, which allows me to remove transparent logos from videos. I don't know how much processing power is needed to run this filter in real-time, but the re-encoding of an 1080p video results in about 25fps on my machine. The result can be seen here. The filter-mask needs small adjustments, but the filter itself works fine. The equation itself is not overly complicated:

Code:

C = ((A - B) * 255) / (256 - B)
A: original frame
B: mask
C: result

I know it has its limitations, but I think it works on most sources. I have also ideas for a logo detection and mask generation.

I have some questions I'd like to ask:
1. What do you think?
2. Would it be complicated to add this to XBMC (maybe even for the next release)?
3. Could this be done in OpenGL to reduce CPU usage?

I don't like logos, they distract and cover viewing area. They had their place in times of "dumb" receivers, but not today anymore. They could easily be embedded in the DVB-stream, but there is no standard for it. It would be possible to remove the logos from 2/3 of the channels in my country. I could not find this feature in other mediacenter software, it would be unique for XBMC.

I planned to clean up the code and submit it to the FFmpeg project. It would be cool if a dev could implement this, because I don't have the experience/knowledge to do this.

[Memphiz]

If ffmpeg would support it - it would be easy to add support for this in xbmc i guess...

[sarbes]

The first link to the image didn't work, here is it hosted at Imageshack: http://imageshack.us/photo/my-images/197/8dgy.jpg/

[bossanova808]

That's really quite cool...never seen it done on the fly. Would be great if done in a shader with minimal impact...but have you tried other logos - they may not all be as easy as that one...

[sarbes]

I think so too. A shader would be most efficient.

I've counted the channel logos of german tv stations which could be removed.

HD channels:
8x logo completly removable
6x logo partly removable
2x logo not removable

SD channels:
13x logo completly removable
2x logo partly removable
9x logo not removable

The HD channels will show the best results. SD channels with low bitrates may be a problem, the Logo could bleed into the picture.

The mask has to be perfect. Every imperfection will result in an degeneration of the image quality. I will try other channels in the next days.

[sarbes]

I've made an almost perfect mask for "WDR HD". The logo has a similar opacity as the first one, but it uses an slightly different algorithm. It tends to distort some colors, but this is fixable. Here is a picture with the "WDR HD" logo removed: http://imageshack.us/f/543/q1r9.jpg/. Try to find it .

[tripkip]

Cool !!

[sarbes]

I'm one step closer to the shader. I've made a lookup table which removes the need to do calculations: http://imageshack.us/photo/my-images/822/dxaf.png/. It accepts the pixel components of the mask (U coordinates) and the original frame (V coordinates) and spits out the resulting color component.

The fragment shader will be just a few lines of code. I don't have any experience in programming OpenGL stuff, so this could take a while.

I would appreciate the help of a developer, because I have no clue how to pass the LUT and the mask to the shader. Also I would love to get some additional input.

[sarbes]

I've wrote my first shader and it works pretty good already. Except a little glitch which is fixable (I think), it acts like my FFmpeg-filter. Here is a screenshot: http://imageshack.us/photo/my-images/819/w0ko.png/.

[bossanova808]

Maybe ask FernetMenta - he's a gun with these sorts of things.

Also - how will masks get associated with particular videos/channels?

[sarbes]

Maybe ask FernetMenta - he's a gun with these sorts of things.

Thanks for the info, I will do that in the next few days.

Also - how will masks get associated with particular videos/channels?

I think a directory with all masks will do. The name of a mask could be "channel-name_maskN.png", so the masks can be associated with the channel. There could be multiple masks per channel and the user could cycle between them. Masks could also be stored in the same dir as the video. In addition, some logos need a custom LUT.


I have identified the problem of my shader. The texture alignment is a bit off due to the mapping process. I'll fix that in the next few days.

[bossanova808]

I think for endcoded video (TV rips or whatever) - it should happen at the time of encoding but if there were a library of good masks available to the encoders plus support in the encoder software, that would make it pretty easy for them...but for live channels that would indeed make sense and be cool.

[sarbes]

That is my plan. I will submit the filter to the FFmpeg guys soon. In the meantime I'm working on the shader. My current method is to modify the YUV2RGB shader, because I have no clue how to add new shaders to the rendering pipeline.

[sarbes]

I would like to resurrect this project in the next weeks.

The GL-shader itself is ready, but I have trouble integrating it. As far as I understand it, "xbmc/cores/VideoRenderers/LinuxRendererGL.cpp" is the right spot to add a hook into the pipeline. The shader would be executed in "xbmc/cores/VideoRenderers/VideoShaders/MyShader.cpp" and could be included directly or in "system/shaders/".


Functions that have to be added to "xbmc/cores/VideoRenderers/LinuxRendererGL.cpp"

On init:
- check if option is turned on
- load LUT(s)
- load mask(s)
- load position of mask(s) (optional, I'll do a fullscreen version first)
- calculate size of mask on frame (optional)
- compile shader

On rendering of a frame (before scaling):
- check if option is turned on
- get the frame (from fbo?)
- apply the shader (MyShader)
- render it (to fbo?)

On end:
- free mask(s), LUT(s) and mask position(s)

The Shader itself is really simple. Mask and frame form the input of a 256x256 LUT. The result of it is the "clean" picture. Take a look at my profile picture for an example LUT.


I don't have much experience, so I could use some help (I wont blame you if you want to get Gotham out of the door first ).

[sarbes]

This project is not forgotten, but I don't have much time for it. I had some problems with the logo masks, because I didn't notice the slight variation through different shows (on the same channel). Unfortunately, I cant get my shader test setup to compile anymore, any additions have to wait.

Here is the code for the fragment shader:

Code:

#version 130
uniform sampler2D img;//COL
uniform sampler2D img2;//MASK
uniform sampler2D img3;//LUT
varying vec2 texcoord;


void main()
{
    vec4 texcolor=texture(img,texcoord);
    vec4 texcolor2=texture(img2,texcoord);
    gl_FragColor = vec4(texture2D(img3,vec2(texcolor2.r*0.9960938+0.00195313,texcolor.r*0.9960938+0.00195313)).r, texture2D(img3,vec2(texcolor2.g*0.9960938+0.00195313,texcolor.g*0.9960938+0.00195313)).g, texture2D(img3,vec2(texcolor2.b*0.9960938+0.00195313,texcolor.b*0.9960938+0.00195313)).b, 0.0);
}

Is someone interested and wants to help me?