EDL frame number data?
#16
Messing with the content of the video file as suggested is unlikely to be supported because it would likely screw with playback, and you'd need to load the content of the file to see if there was anything at the end.

Embedded EDL related content would better be handled similar to how embedded subtitles are managed - in a separate stream. There aren't any standards that I know of that support this though so having a separate file is by far the easiest way to deal with this at present.
Use MythTV for recording TV? Try the integrated MythTV support in XBMC Media Center. Now with commercial skip support built-in and integration with the Movie database!
Reply
#17
The unix tail command is a very efficient way of loading content. It only loads about a page of data by default and the command can be modified to load from 1 byte up. I'd say around 2K should be sufficient.

XBMC is in a position to do ground-breaking work on the next generation of video. Why would we build separate NFO files that require cleanup when we can cleanly implement the content within the target itself?

To write to the file you can use the unix >>
and to retrieve you can use the unix tail

I will do some testing with this to see if it is possible to tag files and play them back correctly once I have finished with mythicalLibrarian. I will create a bash wrapper script for mPlayer in ubuntu which will extract and display data while mPlayer is doing it's thing.
Use mythicalLibrarian to make a library out of your MythTV files. Leave the recording to MythTV and use XBMC as your library.
Installation and Instructions:http://wiki.xbmc.org/index.php?title=MythicalLibrarian
Technical Support:http://forum.xbmc.org/showthread.php?tid=65644
[url=http://forum.xda-developers.com/showthread.php?tid=1081892][/url]
Reply
#18
I have a problem with comskip. This file was just processed. I added it to the TV library and played the video. It started playing right after the first commercial. I do not see any commercial markings on the confluence skin movie timeline bar and I also cannot skip commercials. Am I doing something wrong?

The video file is called
/home/mythtv/Shared/South Park/South Park.S08E14 (Woodland Critter Christmas).mpg

This is my comskip file
/home/mythtv/Shared/South Park/South Park.S08E14 (Woodland Critter Christmas).txt
Code:
FILE PROCESSING COMPLETE
------------------------
0 6807
21966 25283
36852 41520

Nevermind... I just was not being patient enough. However, it would be nice to see some sort of flagging on the timeline bar.
Use mythicalLibrarian to make a library out of your MythTV files. Leave the recording to MythTV and use XBMC as your library.
Installation and Instructions:http://wiki.xbmc.org/index.php?title=MythicalLibrarian
Technical Support:http://forum.xbmc.org/showthread.php?tid=65644
[url=http://forum.xda-developers.com/showthread.php?tid=1081892][/url]
Reply
#19
outleradam Wrote:The unix tail command is a very efficient way of loading content. It only loads about a page of data by default and the command can be modified to load from 1 byte up. I'd say around 2K should be sufficient.

XBMC is in a position to do ground-breaking work on the next generation of video. Why would we build separate NFO files that require cleanup when we can cleanly implement the content within the target itself?

To write to the file you can use the unix >>
and to retrieve you can use the unix tail

I don't think I would call squashing some bytes onto the end of a video file, then reading the bytes at the end of the file and doing magic string comparisons to identify stuff that might or might not be there a clean implementation.

Tail isn't available on all platforms and doesn't work with all transmission protocols, e.g. http, ftp, inside a .rar file etc. It might work fine off a local disk, but that makes the solution pretty restrictive.

Making up what is essentially a new video container format to deal with not having to delete a plain old text file, which most people will understand and be able to do easily, seems a bit over the top.

With the squash it on the end solution you'd probably have to trim the bytes off the end lest the file playback stop prematurely because the player is bufferring 10+ seconds of video and then bails when it gets to content it doesn't comprehend.

From my perspective XBMC is about playing all sorts of video content. It can do that because there are known and defined standards. XBMC isn't (I don't think) about making up new container standards.
Use MythTV for recording TV? Try the integrated MythTV support in XBMC Media Center. Now with commercial skip support built-in and integration with the Movie database!
Reply
#20
I'm trying to come up with a lower maintenance soloution.

I generate about 15 episodes per day on my single tuner card with a maximum possible of about 48 1/2 hour shows. I am getting a HdHomeRun with dual tuner. This will increase my maximum recordings to 144 or 144GB per day. I simply do not have storage for more then 1TB of data spanning 2 hard disks. This means that recordings are constantly deleted and around 15 manual deletions of comskip/nfo data will have to occour per day (currently) and up to 144 manual deletions to keep everything up-to-date.

Ok, so there's a few options,
1.Integrate the data into the main file. This won't work on all platforms
2.Have XBMC look for orphaned nfo files which once had a video file in the library and remove them.

Is there any clean options? Can #2 be an <advancedsettings>?
Use mythicalLibrarian to make a library out of your MythTV files. Leave the recording to MythTV and use XBMC as your library.
Installation and Instructions:http://wiki.xbmc.org/index.php?title=MythicalLibrarian
Technical Support:http://forum.xbmc.org/showthread.php?tid=65644
[url=http://forum.xda-developers.com/showthread.php?tid=1081892][/url]
Reply
#21
outleradam Wrote:Ok, so there's a few options,
1.Integrate the data into the main file. This won't work on all platforms
2.Have XBMC look for orphaned nfo files which once had a video file in the library and remove them.

3. Run a script on a cron job that deletes obviously orphaned EDL related files. There is a direct mapping between the name of the EDL related files and the video file. Just get the list of files ending with *.txt or *.edl (whichever you are using), loop over it and see if the corresponding .mpg file exists. If not delete it.

The problems you are running into is why I baked loading the commercial break information directly into XBMC (just like mythfrontend does).
Use MythTV for recording TV? Try the integrated MythTV support in XBMC Media Center. Now with commercial skip support built-in and integration with the Movie database!
Reply
#22
Yeah, I'm working on building this into mythicalLibrarian currently. How can we load commercial break info into xbmc without scanning every file? MythTV server is waaay too slow for the job. I wouldn't have a problem with xbmc generating comskip because my media center is on all day long anyways.
Use mythicalLibrarian to make a library out of your MythTV files. Leave the recording to MythTV and use XBMC as your library.
Installation and Instructions:http://wiki.xbmc.org/index.php?title=MythicalLibrarian
Technical Support:http://forum.xbmc.org/showthread.php?tid=65644
[url=http://forum.xda-developers.com/showthread.php?tid=1081892][/url]
Reply
#23
outleradam Wrote:Yeah, I'm working on building this into mythicalLibrarian currently. How can we load commercial break info into xbmc without scanning every file? MythTV server is waaay too slow for the job. I wouldn't have a problem with xbmc generating comskip because my media center is on all day long anyways.

I don't understand what you are asking. mythcommflag flags commercials in a background process and stores the flagging information in the mythconverg database.

When a file is played through myth://, XBMC uses the Myth Protocol to ask for the flagged commercial break information from MythTV (which eventually ends up hitting the database). My MythTV server is on as long as there is work for it to do, or it is recording. It automatically turns off and on as necessary.

I have my MythTV server setup to process flagging jobs between 11pm and 6pm (outside the hours we normally watch TV) because it can be quite CPU intensive. The length of time it takes to commercial flag a show depends on the resolution and the flagging mechanism used. I use "Logo Detection" as that seems to work best for the DVB-T TV here in New Zealand and is much, much faster than "All available methods", which is the default.
Use MythTV for recording TV? Try the integrated MythTV support in XBMC Media Center. Now with commercial skip support built-in and integration with the Movie database!
Reply
#24
Ok, basically it goes like this

my script generates a authentic looking comskip file by pulling data from mythconverg database like this:
Code:
#####COMSKIP DATA#####
#Remove old and generate a comskip Start list
    rm $mythicalLibrarian/markupstart.txt
    mysql -u$MySQLuser -p$MySQLpass -e "use '$MySQLMythDb' ; select mark from recordedmarkup where starttime like '$ShowStartTime' and chanid like '$ChanID' and type like "4" ; " |replace "mark" ""|replace " " "">>$mythicalLibrarian/markupstart.txt

#Remove old and generate  comskip Stop list
    rm $mythicalLibrarian/markupstop.txt
    mysql -u$MySQLuser -p$MySQLpass -e "use '$MySQLMythDb' ; select mark from recordedmarkup where starttime like '$ShowStartTime' and chanid like '$ChanID' and type like "5" ; " |replace "mark" ""|replace " " "">>$mythicalLibrarian/markupstop.txt

#Set up counter, remove old markup data and generate new markup file from markupstart and stop
    if [ "$CommercialMarkup" = "Enabled" ]; then    
        mythicalLibrarianCounter=1
        echo "FILE PROCESSING COMPLETE">"$mythicalLibrarian"/markupframes.txt
        echo "------------------------">>"$mythicalLibrarian"/markupframes.txt    
        while read line
            do
            mythicalLibrarianCounter=`expr $mythicalLibrarianCounter + 1`;
            StartData=`sed -n "$mythicalLibrarianCounter"p "$mythicalLibrarian/markupstart.txt"`
            StopData=`sed -n "$mythicalLibrarianCounter"p "$mythicalLibrarian/markupstop.txt"`
            if [ "$StopData" != "" ]; then
                echo "$StartData $StopData">>"$mythicalLibrarian"/markupframes.txt
                CommercialMarkup=Created
                echo "COMMERCIAL DATA START:$StartData STOP:$StopData"
            fi
        done <"$mythicalLibrarian/markupstop.txt"
    fi
fi

The comskip file looks like this:

Code:
FILE PROCESSING COMPLETE
------------------------
0 6807
21966 25283
36852 41520

So that leaves an orphaned file when mythTV removes it's file

So I added a tracker which keeps track of every comskip file generated by my script.
Code:
#####MAINTENANCE#####

#Loop through the list of created comskip files and remove orphans.
while read line
do
    mythicalLibrarianCounter=`expr $mythicalLibrarianCounter + 1`;
    FileToCheck=`sed -n "$mythicalLibrarianCounter"p "$mythicalLibrarian/comskiplog.tracking"`
    FileToCheckOriginalDirName=`dirname "$FileToCheck"`
    FileToCheckFileBaseName=${FileToCheck##*/}
    File
    if [ "$FileToCheck" != "" ]; then
        FileToCheckLS=`ls "$FileToCheckOriginalDirName"|grep "$FileToCheckFileBaseName\."|sed -n 2p`
          if [ "$FileToCheckLS" = "" ]; then
            rm FileToCheck
        else
            echo $FileToCheck>>"$mythicalLibrarian/comskiplog.tracking2"
        fi
    fi
done <"$mythicalLibrarian/comskiplog.tracking"
#Replace old comskip tracking log with new one after maintenance operation.
rm "$mythicalLibrarian/comskiplog.tracking"
mv "$mythicalLibrarian/comskiplog.tracking2" "$mythicalLibrarian/comskiplog.tracking"

Basically, every time the program starts, it's going to validate all comskip files it has generated to see if it has a movie counterpart. I'm now debugging it.

I've noticed that the XBMC comskip has a pretty rough transition. It just cuts to the next word.

Is it possible to do a .05 fade to black, .1 black, then a .05 fade back up? Or maybe at least validate the frame is black and do a correction within .5 seconds?

Comskips are generated by looking for a certain percentage of black frame. When the movie is played through a different player, the timing is different because the framerate is calculated differently. Since we are taking sources from all different places, and calculating our own framerate, it should be double checked for accuracy or give a refined look to the glitch.
Use mythicalLibrarian to make a library out of your MythTV files. Leave the recording to MythTV and use XBMC as your library.
Installation and Instructions:http://wiki.xbmc.org/index.php?title=MythicalLibrarian
Technical Support:http://forum.xbmc.org/showthread.php?tid=65644
[url=http://forum.xda-developers.com/showthread.php?tid=1081892][/url]
Reply
#25
outleradam Wrote:I've noticed that the XBMC comskip has a pretty rough transition. It just cuts to the next word.

Is it possible to do a .05 fade to black, .1 black, then a .05 fade back up? Or maybe at least validate the frame is black and do a correction within .5 seconds?

Comskips are generated by looking for a certain percentage of black frame. When the movie is played through a different player, the timing is different because the framerate is calculated differently. Since we are taking sources from all different places, and calculating our own framerate, it should be double checked for accuracy or give a refined look to the glitch.

Commercial break detection is done via a number of methods. For example, mythcommflag supports "Scene Change Detection", "Blank Scene Detection", "Logo Detection", some voodoo "Experimental" one, or a conglomerate for all of them combined. Other commercially available commercial flagging software may also utilize things like volume level changes.

So, there is little point doing any validation on what sort of frame the detected start or end of a break is because we have no way of knowing how it was detected in the first place. Commercial break detection is a black art and never going to be 100% accurate unless the broadcaster is doing the same thing all the time and it's very easy to detect.

Yes, transitions are hard and they will likely stay that way. Hard cuts are supposed to be direct, and commercial break skipping is really just a convenient way of not having to pickup the remote to skip or fast forward. A fadeout / fade in still won't help if the boundaries of the commercial break aren't in the right place.

For the MythTV commercial break integration I "autowind" into the flagged commercial break so XBMC ends up skiping the flagged commercial break a few seconds late and starts back to the program a few seconds early. I found that very useful to increase our confidence that we hadn't missed any of the show. I can add this "autowind" functionality to the other formats if people are finding that the detected boundaries are a bit too hit and miss or off by just a second or two.

Do you have any concrete evidence that the frame rates being used are different, or is this an assumption based on where you are seeing the commercial break boundaries? I did a number of trials with mythfrontend playback and noted times, compared to the times in the database and then compared with the framerate detected in XBMC and generally they were pretty close.

As an aside, perhaps have a review of some of the advanced settings for commercial break merging. I have built and tweaked this based on experience with how mythcommflag was flagging commercials along with knowledge of how the broadcasters here in New Zealand (and likely other countries) actually group commercials (normal 7 or 8 in a row of 30 seconds each). http://wiki.xbmc.org/?title=Advancedsett...#.3Cedl.3E
Use MythTV for recording TV? Try the integrated MythTV support in XBMC Media Center. Now with commercial skip support built-in and integration with the Movie database!
Reply
#26
Wow, 7 or 8 commercials in a row? Normal here in USA is about 5-6 unless it's a special showing or a very hot show. Of course, I normally record educational shows for the kid and random stuff for the girlfriend and myself. Educational shows normally have only a few commercial breaks.

I will do some hard testing to see if my shows are being flagged the same in xbmc as they are in mythtv streams.

It sounds like you've done some great things. I'd be interested in any improvements that could happen for my recordings. I am using the default mythcommflag and it seems that I always see my show, then some random syllable from a commercial, then the show again on commercial skip. It really seems that an almost unnoticeable fade to black and back up would help immensely and give a refined look to a commercial skip.
Use mythicalLibrarian to make a library out of your MythTV files. Leave the recording to MythTV and use XBMC as your library.
Installation and Instructions:http://wiki.xbmc.org/index.php?title=MythicalLibrarian
Technical Support:http://forum.xbmc.org/showthread.php?tid=65644
[url=http://forum.xda-developers.com/showthread.php?tid=1081892][/url]
Reply

Logout Mark Read Team Forum Stats Members Help
EDL frame number data?0