[danconti]

Hi guys. I work on boxee and have been debugging this issue (sync problems with DTS in mkv) for a bit and wanted to share some of the things i found.

First off, i have yet to find a tool that encodes the proper timing data for dts in MKV. The cluster/block timing in matroska is consistently inaccurate. What i have seen in every clip is the timing progressively creeps forward until eventually it hits a 'pinch', where there is too much audio data in too short a period of time. This is where the sync issues come in.

Additionally, the patches you guys are working with to try and fix up ffmpeg are not getting you the right duration value for DTS frames. This is because ffmpeg is calculating the duration using the stream time scale, which is reduced too far to get the proper acccuracy. I.e. usually the frame size is 512 and the sample rate is 48000 and the frame duration should be 512 / 48000 = .010667, but instead it gets rounded out to .010000.

To fix this for the boxee box, where we were seeing this same issue, i ended up doing two things:

1) I ignore pts/dts for DTS streams in mkv. I just assume it is always wrong. We have some edge cases that require a timestamp for sync after seek, that's the only exception
2) I pull the stream frame size and sample rate in DVDDemuxFFmpeg.cpp and recalculate the duration with double accuracy.

So the audio clock moves based on packet durations now and not packet timestamps when using DTS in mkv. With this setup i seem to be able to play all DTS/DTS-HD in mkv without a hitch.

I can't provide a patch due to code drift but i'd be happy to help you guys out with as much detail as you'd like. Will check back here or you can mail me - dan at boxee dot tv

Cheers,
-Dan

[erhnam]

Hi guys. I work on boxee and have been debugging this issue (sync problems with DTS in mkv) for a bit and wanted to share some of the things i found.

I can't provide a patch due to code drift but i'd be happy to help you guys out with as much detail as you'd like. Will check back here or you can mail me - dan at boxee dot tv

Cheers,
-Dan

Nice one!! Thanks

[bobo1on1]

I would surely like to see code for that

[danconti]

I would surely like to see code for that

Try this. In CDVDDemuxFFmpeg::Read after calculating the timestamps and the duration from the demux packet, we have a block similar to this. This gets you an audio stream with no timestamps, but with proper duration. You need the existing dca in mkv patches for ffmpeg in place.

On the box we actually require a timestamp after a seek and can't recover if we don't get it from audio (sync problem). So our change is slightly more involved

Code:

+        // Matroska embedded DTS-HD has improper timing.  So here we try to fix things. We anchor
+        // the timing off one pts value - needed for sync after seek - then ignore the rest
+        // of the timing data and just supply the duration
+        if(m_bMatroska && stream->codec && stream->codec->codec_id == CODEC_ID_DTS)
+        {
+          // calculate the proper frame duration since ffmpeg doesn't
+          double duration = ((double)stream->codec->frame_size / stream->codec->sample_rate);

+          pPacket->dts = DVD_NOPTS_VALUE;
+          pPacket->pts = DVD_NOPTS_VALUE;
+          pPacket->duration = duration * DVD_TIME_BASE;
+        }

[a11599]

Ok, I tried this and it seems to fix those glitches. But guess what, I can not seek anymore (audio is lost with lots of CDVDPlayer::CheckPlayerInit - dropping packet type:1 dts:-4503599627370496.000000 to get to start point at 191600000.000000
log entries). I am absolutely not comfortable with A/V playback and how it is implemented in XBMC at all, so I hope a more skilled developer will jump in and make some magic here.

[bobo1on1]

-4503599627370496.000000 is DVD_NOPTS_VALUE.

[a11599]

How can I determine if stream syncing after a seek operation is done?

I tried to send the first 2000 packets with original pts/dts/duration after sync operations before kicking danconti's patch in, and it seemed to fix sync behavior. However, it would be nice to kick the patch in right after syncing is ready.

[danconti]

How can I determine if stream syncing after a seek operation is done?

I tried to send the first 2000 packets with original pts/dts/duration after sync operations before kicking danconti's patch in, and it seemed to fix sync behavior. However, it would be nice to kick the patch in right after syncing is ready.

Guess xbmc will need at least one timestamp after a seek too to regain sync. This is the problem, you don't know if the timestamp you get is accurate. So it's possible that using this method will cause a little bit of drift/sync loss. But this is about as good as you can do.

add a double to DVDDemuxFFmpeg called m_fLastPts
set it to DVD_NOPTS_VALUE in the constructor and after a seek op (both time seek and chapter seek)

in the added code below do something like:

Code:

if( m_fLastPTS == DVD_NOPTS_VALUE )
{
  m_fLastPTS = pPacket->pts;
}
else
{
  pPacket->pts = DVD_NOPTS_VALUE;
  pPacket->dts = DVD_NOPTS_VALUE;
}
pPacket->duration = ... ; // duration calculation as in the previous patch - use the stream info and multiply by DVD_BASE_TIME

Also, you don't need 2K packets. What happens is you feed one timestamp in, dvdplayeraudio will resync to that timestamp, then it will start moving the audio clock forward by the duration in the packets, and the player will try to maintain sync against the audio clock based on that.

One other thought. I believe dvdplayeraudio in xbmc calculations the duration to move the audio clock forward using a calculation that assumes the data is PCM. If you are using the passthrough codepath (i.e. bitstream over spdif) this is no good. I think we addressed this by using the original packet duration if we are doing passthrough, but i'm not certain.

[a11599]

Well, I think I am on track now, although I solved it a little bit different: I keep returning pts/dts/duration from original stream until CDVDPlayer::CheckPlayerInit() returns true. It seems to work, but still there are a few things to try (passthrough, sync to display). I can not test these features at the moment (especially not passthrough) since I am not at home.

Anyways, here is the patch in its current state against SVN 35616 for the brave:
http://pastebin.com/Ki4cLRMQ

Thanks for your help so far.

[a11599]

I have tested the patch at home and it works with S/PDIF passthrough and sync to display as well. I have submitted an official patch now: http://trac.xbmc.org/ticket/10891. I hope others will also test it and someday it can get into trunk.

[elupus]

Posted a less hackish solution to the issue in that trac. Will need to discuss it abit with some other devs thou.

[a11599]

It does not eliminate the glitch in my test video entirely. It is smaller, but still there.

(I really don't see how is that less hackish. This patch is still not a fix for the root cause. It does not ignore dts/pts, but that data is corrupt anyways, so why shouldn't we ignore it?)

[elupus]

It's less hackish since the boxee solution would create a remaining lipsync error if some dts samples are corrupt, or the file has been muxed in a fashing where audio timestamps drift over longer periods of time.

It will also mess with MKV's that have been muxed correctly. even if those are rare at the moment they will hopefully become more common.

My change will still trust timestamps from the mkv, it will just lowpass filter the error compared to timestamps as calculated from decoded samples.

[ashlar]

elupus, sorry for the end-user question but will this find its way into Dharma or it will be for later releases?

[ralob]

elupus, sorry for the end-user question but will this find its way into Dharma or it will be for later releases?

While I do not speak for the team, I think this will probably be put in Eden unless it is thoroughly tested quickly and/or a very simple fix. Dharma is just about to be released, and I am sure the team doesn't want any complications.