Android -  how to add hardware video decoding support

  Thread Rating:
  • 0 Votes - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Post Reply
davilla Offline
Retired-Team-XBMC Developer
Posts: 11,508
Joined: Feb 2008
Reputation: 64
Post: #31
dvdplayer/amcodec solutions has added tricks to keep a/v sync. But until we see how HISPlayer does this, it would be hard to recommend how to do the same.


MediaInfo : http://mediainfo.sourceforge.net/
Do not e-mail XBMC-Team members directly asking for support. Read/follow the forum rules.
find quote
sonach Offline
Junior Member
Posts: 35
Joined: Jan 2012
Reputation: 0
Post: #32
(2012-12-23 11:54)Memphiz Wrote:  you got it right - davilla implemented an amlcodec for usage with the dvdplayer Smile
Cool! This provides a good guideline for us!

(2012-12-23 18:02)davilla Wrote:  dvdplayer/amcodec solutions has added tricks to keep a/v sync. But until we see how HISPlayer does this, it would be hard to recommend how to do the same.

davilla,
I think "until we see how HISPlayer does this" should mean "until we see what the vendor(in our case, it is hisilicon) provides "?

your dvdplayer/amcodec solution does give us another direction to do our work. we will look into this when doing further work and we are appreciated to know your progress Smile
(This post was last modified: 2012-12-24 03:05 by sonach.)
find quote
sonach Offline
Junior Member
Posts: 35
Joined: Jan 2012
Reputation: 0
Post: #33
(2012-12-23 18:02)davilla Wrote:  dvdplayer/amcodec solutions has added tricks to keep a/v sync. But until we see how HISPlayer does this, it would be hard to recommend how to do the same.

another question.
Since you have implemented an amcodec, what do you do with the decoded data(put it directly to EGL texture?)? where do you render it?
AFAIK, one of the difficulty of using hw video decode is how to handle the decoded data(some vendors even doe not privde the API to get the decoded data). So in HISPlayer situation, we just don't care about the decoded data, just let it be in the black-box Tongue
find quote
davilla Offline
Retired-Team-XBMC Developer
Posts: 11,508
Joined: Feb 2008
Reputation: 64
Post: #34
Since aml decodes to a separate video plane that is not accessible by OpenGLES, dvdplayer and renderer runs in a bypass mode where it knows where the video is set to appear and makes a hole for it. Otherwise, it does nothing to render it as it does not handle the actual rendering of the video frames.

Most embedded solutions work the same way with hardware decoders, video is actually decoded to a separate video plane that is under the framebuffer/opengles layer and they are blended together with a hw scaler.


MediaInfo : http://mediainfo.sourceforge.net/
Do not e-mail XBMC-Team members directly asking for support. Read/follow the forum rules.
(This post was last modified: 2012-12-24 04:31 by davilla.)
find quote
sonach Offline
Junior Member
Posts: 35
Joined: Jan 2012
Reputation: 0
Post: #35
(2012-12-24 04:31)davilla Wrote:  Since aml decodes to a separate video plane that is not accessible by OpenGLES, dvdplayer and renderer runs in a bypass mode where it knows where the video is set to appear and makes a hole for it. Otherwise, it does nothing to render it as it does not handle the actual rendering of the video frames.
Most embedded solutions work the same way with hardware decoders, video is actually decoded to a separate video plane that is under the framebuffer/opengles layer and they are blended together with a hw scaler.
(1) Yeah, "dvdplayer and renderer runs in a bypass" that is a keypoint! hisilicon also decodes video to a seperate video plane called VO Smile
(2) the question is, what is the advantage of writing a delicated amcodec over writing a delicated AMLPlayer? both are hardware specific... Maybe the former solution can share some code with other hardware(for example, hisilicon)?
find quote
davilla Offline
Retired-Team-XBMC Developer
Posts: 11,508
Joined: Feb 2008
Reputation: 64
Post: #36
The advantage is the ability to handle the various input streams that dvdplayer can handle such as hts (tvheadend), pvr and rtmp. These are a pain to duplicate and is what lead me to look again at dvdplayer+amcodec. Subtitle handling as well as HD audio too.


MediaInfo : http://mediainfo.sourceforge.net/
Do not e-mail XBMC-Team members directly asking for support. Read/follow the forum rules.
find quote
sonach Offline
Junior Member
Posts: 35
Joined: Jan 2012
Reputation: 0
Post: #37
davilla, got your point!
Maybe this solution is by far the best for STB hardwareSmile
find quote
adolph Offline
Junior Member
Posts: 14
Joined: Oct 2012
Reputation: 0
Post: #38
(2012-12-23 18:02)davilla Wrote:  dvdplayer/amcodec solutions has added tricks to keep a/v sync. But until we see how HISPlayer does this, it would be hard to recommend how to do the same.

hi davilla, I want to know some details about your dvdplayer/amlcodec, and could you please offer the link to your source code ?
find quote
davilla Offline
Retired-Team-XBMC Developer
Posts: 11,508
Joined: Feb 2008
Reputation: 64
Post: #39
you blind ? Smile

https://github.com/pivosgroup
https://github.com/Pivosgroup/xbmc/blob/...LCodec.cpp
https://github.com/Pivosgroup/xbmc/blob/...mlogic.cpp


MediaInfo : http://mediainfo.sourceforge.net/
Do not e-mail XBMC-Team members directly asking for support. Read/follow the forum rules.
(This post was last modified: 2013-01-06 15:51 by davilla.)
find quote
adolph Offline
Junior Member
Posts: 14
Joined: Oct 2012
Reputation: 0
Post: #40
(2013-01-06 15:50)davilla Wrote:  you blind ? Smile

https://github.com/pivosgroup
https://github.com/Pivosgroup/xbmc/blob/...LCodec.cpp
https://github.com/Pivosgroup/xbmc/blob/...mlogic.cpp

hi, davilla. I read your code and I have some questions. In your AMLCodec method, the video stream use hardware decoder and the audio stream use software decoder, right? And without enough media knowledge, I did not understand the way how to synchronize video and audio.

By the way, I follow your code and do some simple test on our own platform. Now I can only play some videos.
find quote
davilla Offline
Retired-Team-XBMC Developer
Posts: 11,508
Joined: Feb 2008
Reputation: 64
Post: #41
see CAMLCodec::Process

get_pts_video returns the pts of the last video frame that was presented by hw decoder

GetPlayerPtsSeconds returns the pts of dvdplayer according to the audio clock

SetVideoPtsSeconds is where we tell the hw video decoder to adjust its presentation pts to match dvdplayer.


MediaInfo : http://mediainfo.sourceforge.net/
Do not e-mail XBMC-Team members directly asking for support. Read/follow the forum rules.
find quote
adolph Offline
Junior Member
Posts: 14
Joined: Oct 2012
Reputation: 0
Post: #42
(2013-01-22 17:35)davilla Wrote:  see CAMLCodec::Process

get_pts_video returns the pts of the last video frame that was presented by hw decoder

GetPlayerPtsSeconds returns the pts of dvdplayer according to the audio clock

SetVideoPtsSeconds is where we tell the hw video decoder to adjust its presentation pts to match dvdplayer.

thank you very much, I will go to see the code you point out.
find quote
sonach Offline
Junior Member
Posts: 35
Joined: Jan 2012
Reputation: 0
Post: #43
(2013-01-22 17:35)davilla Wrote:  see CAMLCodec::Process

get_pts_video returns the pts of the last video frame that was presented by hw decoder

GetPlayerPtsSeconds returns the pts of dvdplayer according to the audio clock

SetVideoPtsSeconds is where we tell the hw video decoder to adjust its presentation pts to match dvdplayer.

so now audio is decoded by CPU? It seems that it is a bit CPU-intensive(total CPU% is 65% when playing one TS file with video-hw-decode) on our hardware.
find quote
davilla Offline
Retired-Team-XBMC Developer
Posts: 11,508
Joined: Feb 2008
Reputation: 64
Post: #44
(2013-01-23 04:42)sonach Wrote:  
(2013-01-22 17:35)davilla Wrote:  see CAMLCodec::Process

get_pts_video returns the pts of the last video frame that was presented by hw decoder

GetPlayerPtsSeconds returns the pts of dvdplayer according to the audio clock

SetVideoPtsSeconds is where we tell the hw video decoder to adjust its presentation pts to match dvdplayer.

so now audio is decoded by CPU? It seems that it is a bit CPU-intensive(total CPU% is 65% when playing one TS file with video-hw-decode) on our hardware.

Patches welcome to enable hw audio decode with dvdplayer Smile Your choice is use amlplayer or dvdplayer, dvdplayer handles much more network streaming formats than amlplayer.


MediaInfo : http://mediainfo.sourceforge.net/
Do not e-mail XBMC-Team members directly asking for support. Read/follow the forum rules.
find quote
sonach Offline
Junior Member
Posts: 35
Joined: Jan 2012
Reputation: 0
Post: #45
(2013-01-23 04:52)davilla Wrote:  Patches welcome to enable hw audio decode with dvdplayer Smile Your choice is use amlplayer or dvdplayer, dvdplayer handles much more network streaming formats than amlplayer.

We are evaluting amlplayer solution and dvdplayer solution at the moment. If dvdplayer solution is better for us, we might try to enable hw audio decode. If we can get it work, we are happy to share the workSmile
I agree with you, dvdplayer solution is more 'generic' than amlplayer solution. Amlplayer solution depends heavily on vendor's SDK.
find quote