[adamsutton]

Mouse/Drae,

Just to keep you posted on progress...

I grabbed a raw dump of the data read by the tv_grab script and wrote a quick python script to check I could parse it (just to check my understanding of the protocol was right) looks ok, but I need to tidy it up and make it a little more robust and figure out just how much I can make use of. Then I'll translate to C and look at adding to my tvh branch.

I think I'll try a first pass which will allow "little" configuration, just to make sure I can integrate into TVH ok. I'll look at making it more configurable later.

With regards to your point Mouse there does appear to by MHEG5 support in most of the plugins/scripts I've been using to base the work on. Although my first goal is to get opentv working (since I can actually test that) in time I might be able to write an MHEG5 decoder (it looks fairly straight forward), though I'll need some help testing it as I have no means of doing it myself.

[liquidsky]

Hi Adam,
I tried your version but tvheadend crashes.
The compilation finish with no error:

/ configure --release
make
make install

This is the content of syslog:

Code:

Jun 16 19:04:28 xbmc-mmb tvheadend[2234]: webui: No source path providing HTTP content: "/static". Checked in "./" and in the binary's embedded file system.
Jun 16 19:04:28 xbmc-mmb tvheadend[2234]: webui: No source path providing HTTP content: "/docs". Checked in "./" and in the binary's embedded file system.
Jun 16 19:04:28 xbmc-mmb tvheadend[2234]: webui: No source path providing HTTP content: "/docresources". Checked in "./" and in the binary's embedded file system.
Jun 16 19:04:28 xbmc-mmb tvheadend[2234]: cwc: Attemping to connect to 127.0.0.1:10020
Jun 16 19:04:28 xbmc-mmb tvheadend[2234]: cwc: Attemping to connect to 127.0.0.1:10000
Jun 16 19:04:28 xbmc-mmb tvheadend[2234]: cwc: Connected to 127.0.0.1:10000
Jun 16 19:04:28 xbmc-mmb tvheadend[2234]: cwc: Connected to 127.0.0.1:10020
Jun 16 19:04:29 xbmc-mmb tvheadend[2234]: CRASH: Signal: 6 in PRG: /usr/local/bin/tvheadend () [ffe1a5754aba5b0906556eead3cf4a54034bb476] CWD: /
Jun 16 19:04:29 xbmc-mmb tvheadend[2234]: CRASH: Fault address 0x8ba (N/A)
Jun 16 19:04:29 xbmc-mmb tvheadend[2234]: CRASH: Loaded libraries: /lib/i386-linux-gnu/librt.so.1 /lib/i386-linux-gnu/libdl.so.2 /usr/lib/i386-linux-gnu/libavahi-common.so.3 /usr/lib/i386-linux-gnu/libavahi-client.so.3 /lib/libssl.so.0.9.8 /lib/libcrypto.so.0.9.8 /lib/i386-linux-gnu/libpthread.so.0 /lib/i386-linux-gnu/libm.so.6 /lib/i386-linux-gnu/libc.so.6 /lib/ld-linux.so.2 /lib/i386-linux-gnu/libdbus-1.so.3 /lib/i386-linux-gnu/libz.so.1 /lib/i386-linux-gnu/libnss_compat.so.2 /lib/i386-linux-gnu/libnsl.so.1 /lib/i386-linux-gnu/libnss_nis.so.2 /lib/i386-linux-gnu/libnss_files.so.2
Jun 16 19:04:29 xbmc-mmb tvheadend[2234]: CRASH: Register dump [19]: 00000033 c1500000 ffff007b 0000007b 00718ff4 006f269e bfea0108 bfea00fc 000008ba 00000006 000008ba 00000000 00000000 00000000 00371416 00000073 00000202 bfea00fc 0000007b
Jun 16 19:04:29 xbmc-mmb tvheadend[2234]: CRASH: STACKTRACE
Jun 16 19:04:29 xbmc-mmb tvheadend[2234]: CRASH: /usr/src/tvheadend/src/trap.c:139 0x806a9ea
Jun 16 19:04:29 xbmc-mmb tvheadend[2234]: CRASH: __kernel_rt_sigreturn+0x0  ()
Jun 16 19:04:29 xbmc-mmb tvheadend[2234]: CRASH: abort+0x17e  (/lib/i386-linux-gnu/libc.so.6)
Jun 16 19:04:29 xbmc-mmb tvheadend[2234]: CRASH: __assert_fail+0xf8  (/lib/i386-linux-gnu/libc.so.6)
Jun 16 19:04:29 xbmc-mmb tvheadend[2234]: CRASH: /usr/src/tvheadend/src/epggrab.c:660 0x8056487
Jun 16 19:04:29 xbmc-mmb tvheadend[2234]: CRASH: /usr/src/tvheadend/src/main.c:394 0x804b89a
Jun 16 19:04:29 xbmc-mmb tvheadend[2234]: CRASH: __libc_start_main+0xe7  (/lib/i386-linux-gnu/libc.so.6)
Jun 16 19:04:29 xbmc-mmb tvheadend[2234]: CRASH: ??:0 0x804ae51

Thank you for your work! TVHeadend works well, but the "wrong" EPG is very annoying!

[adamsutton]

Oops,

did you spot the obvious error

I left an assert(0) in the code that I put in while testing the configuration migration, doh!

On a positive note: that gave me a chance to warn you to BACKUP your configuration as my code alters things. However in light of this I've slightly tweaked my code so that it will actually keep all the old files and use new locations for now. So you should in theory be able to run the old and new without them interfering with each other.

Note: because of the assert() your config will be untouched

If you are going to give the code a try, best bet is to pop along to #hts and have a chat. It's still in flux at the moment and although I'd love to have people try it out I definitely can't give any garauntees about performance at the moment.

P.S.
I've fixed the "bug" and it should work now (fingers crossed).

[andyb2000]

Hi Adam,
Sounding good. I was about to upload the stuff I'd done so you could take a look but I think you've beaten me to it! I've been tweaking with the tv_grab_dvb_plus-0.2.1 code, basically what was mentioned here and in the tvh forums, so you had to swap the channel over to the epg background audio channel and do the grab then. I modified the source of the main.cpp so that the application defaulted to the sky epg grabbing, and also forced the devices it was talking to, as defaults didn't work right for me, I also modified it to cut out the commented data that it added to the xml output as that caused xmltv import errors.

However, I thought it was all working...Until today, and found that bits of my EPG are now missing, and it appears the grabbing isn't working right, so I'm going to grab from your github and see if we can progress on that and then submit it up to Andreas.

I'll try and catch you in #hts but I'm around just inbetween family, etc, so tricky!

[adamsutton]

Andy,

what's your IRC nick? just so I can keep an eye out for you?

Yeah same thing here, trying to find as much time as I can but difficult with family etc...

My branch is very different from andoma's I've been speaking to him throughout and I'm hopeful he will accept the changes but it is a complete re-write of the EPG and grabber code (and some other stuff is likely to have to change as a result of this latest stuff).

Also my github branch does NOT contain any of the opentv stuff yet, I've only got that in a local branch as I want to keep it separate from my main work and I'd planned to make it available once I had something that actually worked. But if your keen send me a message and I'll see about putting the branch up on github.

As for the opentv stuff I'll summarise where I am now (slightly moved on from last update):

1. Basic configuration structure is in place, but so far only skyuk config exists and its forced ON. How to actually handle the config is a TBD, but not a particularly tricky task.

2. Most of the integration into TVH with respect to setting up to the receive the data exists, so as long as you tune to the right channel it'll receive the data.

3. Most of the parsing of the OpenTV data is done, however its currently just dumped to stdout.

Things I need to do:

1. Sort out the background tuning.. I'll probably leave this to the end.

2. Modify the TVH DVB tables code, currently only one filter can be added per PID. This is a problem since the standard EIT code installs on 0x11, but opentv code also needs this for some other tables that give it channel info. Shouldn't be too tricky.

3. Process the data into EPG data structures and stick into the EPG.

Given no other obstacles I'd say its a couple of days to get it to a working state. But more likely it'll take a couple of weeks to find enough time.

Adam

[andyb2000]

Hi Adam,

nick on irc is andyb2k5, so will try and catch you on there. I've got your latest build and compile seems fine and stable so far. Interestingly I've been playing about with some other patches that might help with the final bits you mention there:

Background tuning, Rene Herbrich has just submitted some very interesting patches that I think covers this perfectly, what he's done is added in an idle 'monitor' so that when a device goes idle he puts it into an EPG grabber mode based on a schedule you can setup from the GUI. Looks to do that exactly as you need to switch and get the EPG when the tuners are idle. This should fit nicely with the code you're doing, I had quite a few problems getting the code from Rene to patch in automatically using patch, so ended up applying the code manually to the main git source (not your branch). So what I might do is try and apply it to your branch too and see how that goes, at least that should get the background tuning/epg grabbing 'schedule' underway.

Hopefully catch up with you soon.

[Drae]

@adamsutton

Just had a quick look at what you've been doing and must say "nice job!". This has real potential.

[adamsutton]

It's still a work in progress but thanks I'm currently trying to "bend" the opentv data to match my internal data structures and try to cope with some of the issues.

And Andy is busy testing on my behalf and trying to track down bugs

I managed to get a full EPG with series linking and episode equivalence last night. I still have some issues:

- episode equivalence problems to cope with (needs another rewrite of the opentv handler I think).
- multiple services per channel. This results in constant updates to the EPG, since the EID values don't tie up (among other things).
- general performance. I improved it a bit last night, but its still a bit intensive, but that might just be a fact of life. In reality it should only need to run for a few seconds every so often
- on that point I need to determine when the grab is complete AND
- see if I can tie in with standard EIT to detect schedule changes to kick it off to grab full info.

And probably some other niggly stuff too.

[oneadvent]

hi adamsutton, I haven't read all of your google docs, but for me this needed to be added for mine to work with schedulesdirect and usa channels:


https://github.com/oneadvent/tvheadend/c...cc5ddf4c60
https://github.com/oneadvent/tvheadend/c...3a2274a47e
https://github.com/oneadvent/tvheadend/c...394eea1e66

Those feel haphazardly done (because I lack experience doing commits) but if you want to look at the code you can see what I did. Basically I add the sub-categories.

Let me know if you have questions about it.

[the_mouse]

I am willing to test any MHEG5 EPG for NZ you need done just let me know. Looks like ya making progress :-)

[adamsutton]

I believe I've already included sub-category support (though the strings are straight out of the ETSI spec, so they could do with formatting). But its not fully tested and I cannot for the life of me remember whether the raw value gets passed to XBMC (I think it does).

progress is definitely being made. I think MHEG will be a while, main focus is on getting the basic stuff working (EPG changes and grabber framework). OpenTV is a useful test case as I can actually test it myself.

But I will try and take a look at the MHEG5 stuff when I get a chance.

[datty]

Just wanted to post to say thanks for pushing forward with this. My tvheadend is built from your git and seems to have fixed a lot of the EPG based problems I was having. I'm still using tv_grab_dvb_plus piped to tv_grab_file at the minute to get my SKY uk listings through, mainly because I can't tell how/if the opentv stuff is working yet. If theres any way I can help out with the testing of the opentv implementation please let me know, I use this pretty much every day and if theres something I can help out with I'd be happy to.

[oneadvent]

@adamsutton: if I send you the xml I get can you tell me if it will categorize in XBMC from your git?

[oneadvent]

sorry but I am getting this:

Code:

oneadvent@XBMC:/usr/bin$  sudo tvheadend
[INFO]:dvb: Found adapter /dev/dvb/adapter0 (HDHomeRun ATSC) via PCI
[INFO]:dvb: Found adapter /dev/dvb/adapter1 (HDHomeRun ATSC) via PCI
[INFO]:v4l: /dev/video0: uvcvideo Villem usb-0000:00:1a.0-1.5 capabilities: 0x04000001
[INFO]:v4l: /dev/video0: Input #0: Camera 1 (Camera), audio:0x0, tuner:0, standard:0000000000000000,
[INFO]:v4l: /dev/video0: Format #0: YUV 4:2:2 (YUYV) [YUYV]
[WARNING]:v4l: /dev/video0: Device does not have a tuner, device skipped
[ERROR]:webui: No source path providing HTTP content: "/static". Checked in "/usr/bin/" and in the binary's embedded file system.
[ERROR]:webui: No source path providing HTTP content: "/docs". Checked in "/usr/bin/" and in the binary's embedded file system.
[ERROR]:webui: No source path providing HTTP content: "/docresources". Checked in "/usr/bin/" and in the binary's embedded file system.
[NOTICE]:EPG: Injected 3154 event from disk database
[INFO]:dvr: Creating new configuration ''
[INFO]:CSA: Using SSE2 128bit parallel descrambling
[NOTICE]:START: HTS Tvheadend version  started, running as PID:26715 UID:0 GID:0, settings located in '/home/oneadvent/.hts/tvheadend', dataroot: /usr/bin/
[INFO]:xmltv: Grabbing "North America (Data Direct)" using command "/usr/bin/tv_grab_na_dd"
[INFO]:AVAHI: Service 'Tvheadend' successfully established.

and I get no webui at all. Just a blank screen.

-------------------------------------------------------------------------------------------------------------------------------------------
opened a ticket.

[TheGeMan]

I had the same thing, I needed todo a "make install" after make,

I also needed to change the path and daemon location in my /etc/init.d/tvheadend file

# PATH should only include /usr/* if it runs after the mountnfs.sh script
PATH=/usr/sbin:/usr/local/bin:/sbin:/bin
DESC="Tvheadend"
NAME=tvheadend
DAEMON=/usr/local/bin/$NAME
DAEMON_ARGS="-C -f -u hts -g video"
PIDFILE=/var/run/$NAME.pid
SCRIPTNAME=/etc/init.d/$NAME