[magao]

Mazui/UTW Toaru Kagaku no Railgun S ep. 23 720p - many of the subtitles in the ending (karaoke and dialog) are replaced by blocks. I've seen this happen in the past, but it's becoming more prevalent. I don't have the subtitle expertise to determine what's triggering the corruption.

Can anyone confirm if this is happening with other versions of XBMC other than OpenELEC 3.2.0?

[magao]

I've narrowed the issue down to interactions with the ASS \blur tag. Changing all \blur tags to the closest equivalent \be tag resulted in working subtitles.

It looks like OpenELEC is using libass 0.10.1 (the latest) so I'll raise an issue against libass.

[Ned Scott]

I looked it up and it looks like UTW and/or Mazui screwed up the subs themselves.

[magao]

I looked it up and it looks like UTW and/or Mazui screwed up the subs themselves.

Unfortunately it does play properly with MPC-HC + xy-VSFilter, so as far as they (and most anime fansubbers) are concerned, the subtitles are fine and I expect them to continue to use whatever it is that has screwed up. Looking through the subs I didn't see anything which is invalid according to the AegisSubs docs http://docs.aegisub.org/manual/ASS_Tags (although they use plenty of constructions that are noted as possibly being unsafe for soft-subbing).

BTW I also tried to play back with Windows XBMC 12.2 and had multiple issues. Firstly, a lot of the borders and blurring covered way too large an area, and playback dropped to <1FPS at the point that I was getting corruption with OpenELEC. I was debugging on an i7 so I wasn't expecting that. Anyway, even if the symptoms weren't identical, there were definitely issues.

I have a python script which will do the replacement of \blur to an appropriate \be - would there be any way if it were an addon to intercept the subtitles before libass gets its hands on them and perform this processing? The code (currently a quick hack) is below.

Code:

# Currently only tested with Python 3.3
# We operate on bytes - that way we don't change line endings, remove UTF-8 BOM if present, etc
BLUR_RE = re.compile(rb'\\blur([0-9]+)(?:\.([0-9]+))?')

def blur_repl(m):
    i = int(m.group(1), 10)
    j = 0

    if m.group(2):
        j = float(rb'0.' + m.group(2))

    # If exactly zero, produce \be0 otherwise round to nearest integer with a minimum of 1
    if not i and not j:
        be = 0
    else:
        be = max(1, int(i + j + 0.5))

    return rb'\be' + bytes(str(be), 'US-ASCII')

def blur_to_be(data_in):
    return BLUR_RE.sub(blur_repl, data_in)

[Ned Scott]

Doh, my bad, I was looking at the SSA tags and not the ASS tags.

[magao]

Libass issue: http://code.google.com/p/libass/issues/detail?id=110

According to a libass dev, playback is fine in Mplayer2 using libass 0.10.1. On OSX he sees similar symptoms to me with Windows - lagging down to <1FPS and the borders around the text not rendered correctly, but readable. Totally different to how it's rendered on OpenELEC.

OSX vs MPlayer2: http://check2pic.ru/compare/31512/ (OSX first - mouse over to see Mplayer2 which is how it should be rendered)
OpenELEC: http://libass.googlecode.com/issues/atta...3&inline=1

The unusable playback speed on Windows is not evident on OpenELEC. It's not a CPU issue - CPU usage at the time is very low. There are a lot of overlapping blurred subtitles (edit - not precisely overlapping - they're clipped) starting at timestamp 22:06.20 so maybe something is forcing them to be processed serially or go through a slow code path on Windows and OSX.

[Chortos-2]

Hello, I’m the aforementioned dev.

As far as I can see, the subs are fine, and indeed they play correctly in both mplayer2 and VLC on OS X. I thought this could be some sort of older libass bug that had been fixed between the release of XBMC 12.2 and now, but I couldn’t reproduce it in mplayer2 even with an appropriately old libass.

What makes it weirder is that the symptoms vary between platforms: as magao said, on Windows and OS X the player almost freezes when it encounters the first affected line (even after the video is paused) and apparently renders all not-fully-transparent pixels as fully opaque*, while on Linux it doesn’t freeze but shows some amazing block art instead of Latin letters. There are screenshots of both the OS X and the Linux rendering on the libass issue tracker.
* This effect seems to fade away along with the affected line during the animated fade, \fad.

I also observed a huge increase in memory usage on OS X when the first affected line was encountered and a slower huge increase over the rest of the file.

Other potentially useful information: there is a whole bunch of blurred lines in the same frame, but they are clipped to non-overlapping rectangles. They also have different colours to achieve a vertical gradient, and on OS X, I can see the gradient in XBMC, so the clipping seems to work fine. Meanwhile, in the Linux screenshot, I’m inclined to say I don’t see the gradient, which would suggest that clipping is somehow broken. (Which is weird per se, because clipping is done internally in libass.)


And magao, \be and \blur arguments are far from equivalent. As it happens, just recently I did some (mathematical) experiments with \be and \blur, and here’s the conversion formula I obtained: \be argument = (\blur argument)^2 * 8 / log(256). So for \blur3 you actually want \be13!

[magao]

And magao, \be and \blur arguments are far from equivalent. As it happens, just recently I did some (mathematical) experiments with \be and \blur, and here’s the conversion formula I obtained: \be argument = (\blur argument)^2 * 8 / log(256). So for \blur3 you actually want \be13!

Hey - I only learned about \be and \blur yesterday

I'll change my snippet to use the algorithm you've supplied. It appears you mean the natural logarithm of 256. Does it also hold for \blur arguments < 1 e.g. \blur0.5 ~= \be0.35 (which would be converted to \be1 since \be needs to be integer and any non-zero \blur should become a non-zero \be)?

[magao]

Interesting. I used the suggested algorithm for converting \blur -> \be and I started getting the same issues with \be tags (e.g. \be13). The other thing I've discovered is that it's in combination with vector clipping that the issue occurs - if I change all the vector clipping to rectangular clipping (where the vector clip actually is a rectangle) the corruption goes away on OpenELEC (Windows still has the huge aura and slowdown) e.g.

Dialogue: 4,0:22:06.20,0:22:10.17,ED4-Romaji,,0,0,0,,{\pos(60,32)\blur3\bord3\fad(480,320)\clip(m 0 30 l 1280 30 1280 32 0 32 0 30)\c&HEEC69B&\3c&HEEC69B&}kawaita kaze ni sakebu

to a rectangular clip:

Dialogue: 4,0:22:06.20,0:22:10.17,ED4-Romaji,,0,0,0,,{\pos(60,32)\blur3\bord3\fad(480,320)\clip(0, 30, 1280, 32)\c&HEEC69B&\3c&HEEC69B&}kawaita kaze ni sakebu

It also seems to require multiple clipped lines for the problem to occur, but I'm not sure about that.

BTW I also tried removing all the lines with clipping and the slowdown on Windows went away, so that's definitely an issue with rendering that many subtitles (with that many transformations) per frame.

[magao]

I grabbed the XBMC source and built master on Windows. And whilst I was able to replicate a lot of slowdown, it didn't get to the <1FPS that I had before (lowest I saw was about 4FPS). I also was unable to replicate the weird aura that was occurring on Windows as is visible in http://check2pic.ru/compare/31512/. This was using a debug build. I've tried going back to 12.2-Frodo to replicate there but it doesn't want to build - think I'll need a new clone.

I'd noticed that CDVDSubtitlesLibass:ecodeDemuxPkt() called ass_process_chunk with a single line, and thought that maybe as a result it was doing more processing than it needed to (libass might have been missing some opportunities for optimisation). However, I changed CDVDSubtitlesLibass:ecodeDemuxPkt() to instead call ass_process_data and CDVDOverlayCodecSSA:ecode() to pass all lines with the same start and end timestamp in the packet as a single string to CDVDSubtitlesLibass:ecodeDemuxPkt() - and no speedup at all.

Going to have to leave it there for now ... I'll keep poking at it as I get the time.

[Chortos-2]

It appears you mean the natural logarithm of 256.

Yes. I’m not quite sure how widely the ‘ln’ notation is used, so I went with ‘log’ this time.

Does it also hold for \blur arguments < 1 e.g. \blur0.5 ~= \be0.35 (which would be converted to \be1 since \be needs to be integer and any non-zero \blur should become a non-zero \be)?

It gets messy for low levels of \be, because \be is discrete. What \beX does is apply a [1,2,1] convolution kernel to the rasterized bitmap X times in a row, while what \blurX does is apply a Gaussian blur with σ=2X/sqrt(log(256)), at least in libass. Actually, I’m trying to understand whether this matches what VSFilter does and not having much success so far. Anyway. For large arguments, \be approximates a Gaussian curve of σ=sqrt(0.5 \be argument) quite well. But for very small arguments, the curves diverge a bit, the \be one being flatter and fatter; so low-argument \be corresponds to a slightly higher \blur radius than given by my formula. Hm… Maybe I can find an additional coefficient that will make the formula fit small arguments as well. In any case, for a blur radius as low as 0.5, even \be1 is too strong—but sometimes having any blur at all matters more than it being a close match to the original.

[magao]

In any case, for a blur radius as low as 0.5, even \be1 is too strong—but sometimes having any blur at all matters more than it being a close match to the original.

Yeah - that's exactly why if there was any blur I set a minimum \be of 1. Similarly, whilst your algorithm gives a more accurate match, I'm finding that just having \blurX -> \beX gives a good enough result and is much less likely to result in corruption, so I'll probably stick with doing that for the files that need it.

Probably not going to be able to spend a lot of time debugging this during the week ... I'll install the latest version of OpenELEC that came out today but I'm not expecting it to make any difference.

[magao]

Comment on libass tracker: http://code.google.com/p/libass/issues/detail?id=110

Comment #15 on issue 110 by [email protected]: Corrupted subtitles with \blur tag
http://code.google.com/p/libass/issues/detail?id=110

Sounds like xbmc is blending the ASS_Images incorrectly.

Assuming xbmc renders with OpenGL, are you using glBlendFunc(GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA)?

Here's mpv's implementation, if it helps (SUBBITMAP_LIBASS pretty much corresponds to ASS_Image): https://github.com/mpv-player/mpv/blob/m...t/gl_osd.c

I don't know anything whatsoever about programming for OpenGL (or DirectX or anything like that).

I did a search of the codebase for glBlendFunc - possibly relevant places I found are:

Code:

"xbmc/xbmc/cores/VideoRenderers/LinuxRendererGL.cpp"(656,5):    glBlendFunc(GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA);
"xbmc/xbmc/cores/VideoRenderers/LinuxRendererGLES.cpp"(429,5):    glBlendFunc(GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA);
"xbmc/xbmc/cores/VideoRenderers/LinuxRendererGLES.cpp"(480,5):    glBlendFunc(GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA);
"xbmc/xbmc/cores/VideoRenderers/OverlayRendererGL.cpp"(384,3):  glBlendFunc(GL_SRC_ALPHA,GL_ONE_MINUS_SRC_ALPHA);
"xbmc/xbmc/cores/VideoRenderers/OverlayRendererGL.cpp"(491,5):    glBlendFunc(GL_ONE, GL_ONE_MINUS_SRC_ALPHA);
"xbmc/xbmc/cores/VideoRenderers/OverlayRendererGL.cpp"(493,5):    glBlendFunc(GL_SRC_ALPHA,GL_ONE_MINUS_SRC_ALPHA);
"xbmc/xbmc/guilib/GUIFontTTFGL.cpp"(82,5):    glBlendFuncSeparate(GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA, GL_ONE_MINUS_DST_ALPHA, GL_ONE);
"xbmc/xbmc/guilib/GUITextureGL.cpp"(51,3):  glBlendFunc(GL_SRC_ALPHA,GL_ONE_MINUS_SRC_ALPHA);
"xbmc/xbmc/guilib/GUITextureGL.cpp"(154,3):  glBlendFunc(GL_SRC_ALPHA,GL_ONE_MINUS_SRC_ALPHA);
"xbmc/xbmc/guilib/GUITextureGLES.cpp"(107,5):    glBlendFuncSeparate(GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA, GL_ONE_MINUS_DST_ALPHA, GL_ONE);
"xbmc/xbmc/guilib/GUITextureGLES.cpp"(219,3):  glBlendFunc(GL_SRC_ALPHA,GL_ONE_MINUS_SRC_ALPHA);
"xbmc/xbmc/pictures/SlideShowPicture.cpp"(784,5):    glBlendFunc(GL_SRC_ALPHA,GL_ONE_MINUS_SRC_ALPHA);
"xbmc/xbmc/pictures/SlideShowPicture.cpp"(835,5):    glBlendFunc(GL_SRC_ALPHA,GL_ONE_MINUS_SRC_ALPHA);
"xbmc/xbmc/rendering/gl/RenderSystemGL.cpp"(231,3):  glBlendFunc(GL_SRC_ALPHA, GL_ONE);
"xbmc/xbmc/rendering/gles/RenderSystemGLES.cpp"(150,3):  glBlendFunc(GL_SRC_ALPHA, GL_ONE);

[Chortos-2]

TL;DR I know what the OS X/Windows issue is.

I did some more testing today, and the conclusion is... uhh. The corruption on OS X, presumably Windows and possibly Linux only occurs when playing an external subtitle file.

I can’t remember how exactly I did my testing last time, but when I went back to the same files today, I had an external ASS file next to the MKV. At first I tried various things and was very confused when corruption appeared and disappeared depending on such factors as whether I opened the file through the GUI or passed it as a command-line argument, whether the file was a symbolic link or a regular file, and whether it’s the original file or a remuxed copy. I thought I established that XBMC completely ignored my external ASS file at one point, but after exhausting my other options I tried testing it once more. Sure enough, it does use the external file, and moreover, it goes crazy when the external file is present and works correctly (albeit with an occasional slight stutter) when it’s absent.

And from reading the XBMC source code, I know just what the problem is. (I actually read the code and saw this as a potential problem—“unless it’s handled somewhere else”—before I figured out that it was indeed the entire issue. Remember, I thought it was ignoring my external subtitles.) CDVDSubtitleParserSSA creates a separate overlay for each ASS event, even when there are multiple events on screen at the same time. CDVDOverlayContainer::Add then checks for overlapping events but doesn’t do anything because it explicitly avoids touching events that share an exact same start time. So all these duplicate events are rendered, and each blends the entire ASS frame onto the video. So you end up with the same subtitles blended 53 times over, which looks, well, like my screenshot!