Kodi Community Forum
[RELEASE] PseudoTV Addon: Virtual EPG and TV Channel Surfing Script - Printable Version

+- Kodi Community Forum (https://forum.kodi.tv)
+-- Forum: Support (https://forum.kodi.tv/forumdisplay.php?fid=33)
+--- Forum: Add-on Support (https://forum.kodi.tv/forumdisplay.php?fid=27)
+---- Forum: Program Add-ons (https://forum.kodi.tv/forumdisplay.php?fid=151)
+----- Forum: PseudoTV / PseudoTV Live (https://forum.kodi.tv/forumdisplay.php?fid=231)
+----- Thread: [RELEASE] PseudoTV Addon: Virtual EPG and TV Channel Surfing Script (/showthread.php?tid=90738)



- zepfan - 2011-02-10

jtucker1972 Wrote:Same here. Ember really screwed up my movie library. Weird thing is that runtime is set to min for most. Sometime you get 141 min | 160 min (premiere cut). But since I've scrapped my movies with Ember, XBMC no longer updates the streamdetails table which is where PseudoTV gets the durations from. So I get the No duration error when trying to load the channel. I even tried the playlist trick but it would just sit there and not update the durations.

I unchecked Runtime from the Scraper Fields - Global settings. I am re-scraping now to see if that fixes things. We shall see. Fun times.

Image
I'm not going to stop you, because I may be wrong, but I've tried editing out the runtime/duration from some nfo's and it just shows as blank. It could do it differently with ember though.


- jtucker1972 - 2011-02-10

zepfan Wrote:I'm not going to stop you, because I may be wrong, but I've tried editing out the runtime/duration from some nfo's and it just shows as blank. It could do it differently with ember though.

I am deleting all my nfo's and starting over. Since this is a clean import I want to experiment to figure out what the best way is. There doesn't seem to be a definitive answer.

How does XBMC update the streamdetails data? When? During scanning of new content? During playlist viewing? I have more questions than answers at the moment.


- zepfan - 2011-02-10

During scraping I believe.


- jtucker1972 - 2011-02-10

zepfan Wrote:During scraping I believe.

Here's my current test.

1. Deleted all nfo files. (this forces XBMC to scrape the movie information)
2. Deleted MyVideos34.db. (ensure I have a empty video data to start)
3. Scanned for new content
4. Waited for XBMC to scan threw several movies.
5. Verified in log that it had successfully scraped the movies from imdb
6. Used sqlite browser to view the streamdetails table. No records added.

So, based on this test, the streamdetails are not added during the scrape, at least not with imdb scraping. I also tested with using TheMovieDB as the scraping source. Still nothing.

I then opened my utility-all-movies playlist and it populated the streamdetails table. So, what ever XBMC code is behind the playlist window is populating the data we need for PseudoTV. If we could find the code that does that, then it shouldn't be a big deal to update streamdetails on the fly when needed or pull the information directly in the same method the playlist does.


- Sranshaft - 2011-02-10

jtucker1972 Wrote:Here's my current test.

1. Deleted all nfo files. (this forces XBMC to scrape the movie information)
2. Deleted MyVideos34.db. (ensure I have a empty video data to start)
3. Scanned for new content
4. Waited for XBMC to scan threw several movies.
5. Verified in log that it had successfully scraped the movies from imdb
6. Used sqlite browser to view the streamdetails table. No records added.

So, based on this test, the streamdetails are not added during the scrape, at least not with imdb scraping. I also tested with using TheMovieDB as the scraping source. Still nothing.

I then opened my utility-all-movies playlist and it populated the streamdetails table. So, what ever XBMC code is behind the playlist window is populating the data we need for PseudoTV. If we could find the code that does that, then it shouldn't be a big deal to update streamdetails on the fly when needed or pull the information directly in the same method the playlist does.

Have you made sure you've told XBMC to grab that information? Go to Settings > Video > File Lists and check to see if extract thumbnails and video information is checked.

If it isn't enabled, XBMC will only pull that information the first time you play the video. I can't understand why this setting isn't enabled out of the box as it leads to problems exactly like this. Actually, I believe this could be the reason a lot of people are having so many duration-related troubles.


- steppedup - 2011-02-10

Regarding randomization of the playlists - you can ensure that you'll get randomization of playlists by turning on Shuffle....

Play a video, go to Now Playing screen, go to the options page (left-side if confluence), and Shuffle on.


- jtucker1972 - 2011-02-10

Sranshaft Wrote:Have you made sure you've told XBMC to grab that information? Go to Settings > Video > File Lists and check to see if extract thumbnails and video information is checked.

If it isn't enabled, XBMC will only pull that information the first time you play the video. I can't understand why this setting isn't enabled out of the box as it leads to problems exactly like this. Actually, I believe this could be the reason a lot of people are having so many duration-related troubles.

Yes. It is enabled in my XBMC so that is not the issue.

I've traced where in the source code XBMC gets and inserts the streamdetails data. It's in the C++ source file VideoDatabase.cpp. I haven't seen where these could be called via api, but that's where they are. Trying to find more information on the XBMC development API, but looks like the wiki is down.


- jtucker1972 - 2011-02-10

steppedup Wrote:Regarding randomization of the playlists - you can ensure that you'll get randomization of playlists by turning on Shuffle....

Play a video, go to Now Playing screen, go to the options page (left-side if confluence), and Shuffle on.

Wouldn't this just tell the player to pick a random file in the playlist? PseudoTV takes the output of the playlist and builds the m3u file which gets played in the order they appear in the m3u. So, not sure this is the answer. Either way, I would want the ability to program my channel to either randomized the list of video files I pass it or play them in the order I sent.


- Sranshaft - 2011-02-10

jtucker1972 Wrote:Yes. It is enabled in my XBMC so that is not the issue.

I've traced where in the source code XBMC gets and inserts the streamdetails data. It's in the C++ source file VideoDatabase.cpp. I haven't seen where these could be called via api, but that's where they are. Trying to find more information on the XBMC development API, but looks like the wiki is down.

Sorry, I thought you had an issue with XBMC not populating the database with the video stream details. Without that setting enable it won't populate the database until the video is first played.

I'm not sure that PseudoTV really needs the extra overhead having to deal with things that should be / are in the core of XBMC. If you have the setting enabled and it is still not populating the database then that definitely is a bug and should be brought to the attention of the devs.


- jtucker1972 - 2011-02-10

Sranshaft Wrote:Sorry, I thought you had an issue with XBMC not populating the database with the video stream details. Without that setting enable it won't populate the database until the video is first played.

I'm not sure that PseudoTV really needs the extra overhead having to deal with things that should be / are in the core of XBMC. If you have the setting enabled and it is still not populating the database then that definitely is a bug and should be brought to the attention of the devs.

To be clear, the setting is enabled, but the streamdetails data is not being populated when new content is being scanned in. I will do one more test to capture screens, etc. If it still behaves this way, I will report it as a bug then. Do you know where it is documented that enabling the "Extract thumbnails and video information" feature will populate streamdetails table when you scan in new content?

I agree with you that Jason should not have to work around this. I've spent a personal day digging into it for my own edification. Too bad they didn't add an api call for this as well.


- Sranshaft - 2011-02-10

Well, I haven't been able to find any concrete documentation but one would assume that is where it would happen. At least, that is where I would put it if I was programming the scanner (save on read/write to disk, far easier to just fire everything off at once, etc) but I have been known to be wrong. Perhaps it fires off on a separate thread after the scanning is complete.

I have been able to come across a single thread started last year regarding problems with extracting video info for files on network shares.

Some questions to help narrow down some things before classifying it as a bug. Are your files local or on a NAS / server? Which OS are you using? If Windows and using a network, are you using mapped drives or smb? What format are your video files in? ISOs and DVD Video_TS files need to be played first before the information is extracted. The same goes for rar files as well I believe.

I'll let you know if I happen to find anything else.


- jtucker1972 - 2011-02-10

Sranshaft Wrote:Well, I haven't been able to find any concrete documentation but one would assume that is where it would happen. At least, that is where I would put it if I was programming the scanner (save on read/write to disk, far easier to just fire everything off at once, etc) but I have been known to be wrong. Perhaps it fires off on a separate thread after the scanning is complete.

I have been able to come across a single thread started last year regarding problems with extracting video info for files on network shares.

Some questions to help narrow down some things before classifying it as a bug. Are your files local or on a NAS / server? Which OS are you using? If Windows and using a network, are you using mapped drives or smb?

I'll let you know if I happen to find anything else.

They are are on a NAS with mapped drives. I posted a topic in general support forum as well. If not, I know the steps it would take to build a utility to do it. ffmpeg and sqlite library.

1. sqlite to determine which file ids don't have durations in streamdetails
2. loop though each file id and get path to video file
3. call ffmpeg -i <path to video filename>
4. parse out duration and other video information
5. convert to duration to seconds
6. insert into streamdetails
7. move to next file id

Something like that.


- Sranshaft - 2011-02-10

That doesn't sound too hard though I would use something simple like Microsoft.DirectX.AudioVideoPlayback to get the duration if that is all you're after. It's already included in any Windows installation and returns the duration in seconds automatically.

If you're after the codec, resolution and audio properties then I would use MediaInfo over ffmpeg. MediaInfo is what XBMC uses to extract the video information so it's safe to assume the same details would be returned. In fact, I already have a method coded in C# that does this already. I would just need to marry that with a call to SQLite.

Code:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using MediaInfoLib;
using System.IO;

namespace InfoGatherer
{
    class MediaInfoBuilder
    {
        public static StreamInfo GetMediaInfo(string filePath)
        {
            try
            {
                MediaInfo mediaInfo = new MediaInfo();
                mediaInfo.Open(filePath);

                StreamInfo streamInfo = new StreamInfo();
                VideoInfo videoInfo = new VideoInfo();

                videoInfo.VideoAspectDisplayRatio = mediaInfo.Get(StreamKind.Video, 0, "DisplayAspectRatio");
                videoInfo.VideoBitrate = mediaInfo.Get(StreamKind.Video, 0, "BitRate/String");
                videoInfo.VideoCodec = mediaInfo.Get(StreamKind.Video, 0, "Codec");
                videoInfo.VideoCodecID = mediaInfo.Get(StreamKind.Video, 0, "CodecID");
                videoInfo.VideoContainer = Path.GetExtension(filePath);
                videoInfo.VideoDuration = mediaInfo.Get(StreamKind.Video, 0, "PlayTime");
                videoInfo.VideoFormat = mediaInfo.Get(StreamKind.Video, 0, "Format");
                videoInfo.VideoHeight = mediaInfo.Get(StreamKind.Video, 0, "Height");
                videoInfo.VideoScanType = mediaInfo.Get(StreamKind.Video, 0, "ScanType/String");
                videoInfo.VideoWidth = mediaInfo.Get(StreamKind.Video, 0, "Width");

                List<AudioInfo> audioStreamList = new List<AudioInfo>();

                int audioStreamCount = Convert.ToInt32(mediaInfo.Get(StreamKind.Audio, 0, "Count"));


                for (int i = 0; i < audioStreamCount; i++)
                {
                    try
                    {
                        AudioInfo audioInfo = new AudioInfo();

                        audioInfo.Bitrate = mediaInfo.Get(StreamKind.Audio, i, "BitRate/String");
                        audioInfo.Channels = mediaInfo.Get(StreamKind.Audio, i, "Channel(s)");
                        audioInfo.Codec = mediaInfo.Get(StreamKind.Audio, i, "Codec/String");
                        audioInfo.Language = mediaInfo.Get(StreamKind.Audio, i, "Language");
                        
                        if (audioInfo.Bitrate != string.Empty)
                            audioStreamList.Add(audioInfo);
                    }
                    catch
                    {
                        Console.WriteLine("Error while getting audio info!");
                    }
                }

                streamInfo.VideoInfo = videoInfo;
                streamInfo.StreamAudioList = audioStreamList;

                mediaInfo.Close();

                return streamInfo;
            }
            catch (Exception ex)
            {
                Console.WriteLine("Error occured while getting MediaInfo: " + ex.Message);
                return null;
            }
        }
    }

    public partial class StreamInfo
    {
        private VideoInfo videoInfo;
        public VideoInfo VideoInfo { get { return videoInfo; } set { videoInfo = value; } }

        private List<AudioInfo> streamAudioList;
        public List<AudioInfo> StreamAudioList { get { return streamAudioList; } set { streamAudioList = value; } }
    }

    public partial class VideoInfo
    {
        private string videoWidth;
        public string VideoWidth { get { return videoWidth; } set { videoWidth = value; } }

        private string videoHeight;
        public string VideoHeight { get { return videoHeight; } set { videoHeight = value; } }

        private string videoCodec;
        public string VideoCodec { get { return videoCodec; } set { videoCodec = value; } }

        private string videoDuration;
        public string VideoDuration { get { return videoDuration; } set { videoDuration = value; } }

        private string videoFormat;
        public string VideoFormat { get { return videoFormat; } set { videoFormat = value; } }

        private string videoBitrate;
        public string VideoBitrate { get { return videoBitrate; } set { videoBitrate = value; } }

        private string videoContainer;
        public string VideoContainer { get { return videoContainer; } set { videoContainer = value; } }

        private string videoCodecID;
        public string VideoCodecID { get { return videoCodecID; } set { videoCodecID = value; } }

        private string videoScanType;
        public string VideoScanType { get { return videoScanType; } set { videoScanType = value; } }

        private string videoAspectDisplayRatio;
        public string VideoAspectDisplayRatio { get { return videoAspectDisplayRatio; } set { videoAspectDisplayRatio = value; } }
    }

    public partial class AudioInfo
    {
        private string language;
        public string Language { get { return language; } set { language = value; } }

        private string codec;
        public string Codec { get { return codec; } set { codec = value; } }

        private string channels;
        public string Channels { get { return channels; } set { channels = value; } }

        private string bitrate;
        public string Bitrate { get { return bitrate; } set { bitrate = value; } }
    }
}



- jtucker1972 - 2011-02-10

From the log I thought it XBMC used ffmpeg. I guess I am still hoping there is a way to do this from within a python script so it can occur in the background within the XBMC architecture. I know Jason is looking into something like this, but he's been quiet so not sure where he is at in the process. It's just driving me nuts that someone has to go through playlists and wait to have durations calculated.

Code:
18:05:16 T:7568 M:1809375232   DEBUG: CThumbExtractor::DoWork - trying to extract filestream details from video file S:\FreeAgent Drive\Movies [SD] [G]\2001 A Space Odyssey (1968) [SD] [G]\2001 A Space Odyssey (1968) [SD] [G].avi
18:05:16 T:7376 M:1808809984   DEBUG: Thread 7376 terminating
18:05:16 T:7568 M:1808756736   DEBUG: Win32DllLoader::Load(special://xbmcbin/system/players/dvdplayer/avutil-50.dll)
18:05:16 T:7568 M:1808711680   DEBUG: Win32DllLoader::Load(special://xbmcbin/system/players/dvdplayer/avcodec-52.dll)
18:05:16 T:7568 M:1804787712   DEBUG: Win32DllLoader::Load(special://xbmcbin/system/players/dvdplayer/libfaad.dll)
18:05:16 T:7568 M:1804767232   DEBUG: Win32DllLoader::Load(special://xbmcbin/system/players/dvdplayer/avformat-52.dll)
[b]18:05:16 T:7568 M:1800790016   DEBUG: CDVDDemuxFFmpeg::Open - probing detected format [avi]
18:05:28 T:7568 M:1794326528   DEBUG: CDVDDemuxFFmpeg::Open - av_find_stream_info starting
18:05:28 T:7568 M:1793732608   DEBUG: ffmpeg[1D90]: [mpeg4] Invalid and inefficient vfw-avi packed B frames detected
18:05:28 T:7568 M:1794310144   DEBUG: CDVDDemuxFFmpeg::Open - av_find_stream_info finished
18:05:28 T:7568 M:1794310144    INFO: ffmpeg[1D90]: Input #0, avi, from 'S:\FreeAgent Drive\Movies [SD] [G]\2001 A Space Odyssey (1968) [SD] [G]\2001 A Space Odyssey (1968) [SD] [G].avi':
18:05:28 T:7568 M:1794310144    INFO: ffmpeg[1D90]:   Metadata:
18:05:28 T:7568 M:1794310144    INFO: ffmpeg[1D90]:     ISFT            : FairUse Wizard - http://fairusewizard.com
18:05:28 T:7568 M:1794310144    INFO: ffmpeg[1D90]:   Duration: 02:14:51.20, start: 0.000000, bitrate: 1516 kb/s
18:05:28 T:7568 M:1794310144    INFO: ffmpeg[1D90]:     Stream #0.0: Video: mpeg4, yuv420p, 656x304 [PAR 1:1 DAR 41:19], 25 tbr, 25 tbn, 25 tbc
18:05:28 T:7568 M:1794310144    INFO: ffmpeg[1D90]:     Stream #0.1: Audio: ac3, 48000 Hz, 5.1, s16, 384 kb/s
[/b]18:05:28 T:7568 M:1805234176   DEBUG: CThumbExtractor::DoWork - trying to extract filestream details from video file S:\FreeAgent Drive\Movies [SD] [G]\Bambi (1942) [LOW] [APPROVED]\Bambi (1942) [LOW] [APPROVED].avi
18:05:28 T:9968 M:1805230080   DEBUG: SDLKeyboard: scancode: 72, sym: 273, unicode: 0, modifier: 0
18:05:28 T:9968 M:1805230080   DEBUG: CApplication::OnKey: 61478 pressed, action is Up

BTW, sweet C# code. The SQLite adapter is located at http://system.data.sqlite.org/index.html/doc/trunk/www/index.wiki.


- Jason102 - 2011-02-10

Nothing to report. Looking at a couple of different possibilities at the moment. If I have some real progress, don't worry I'll mention it.