[Milhouse]

Export your nfs share only to your rpi IP. And you'll know if the data goes through the rpi or not

I can see from the level of network traffic that the Pi isn't streaming anything - 323 bytes/second on TX/RX for a 1080p/DTS movie? That's some heavy-duty compression!

Edit: Also, I can shut down the Pi and the movie/music continues playing on x86 (until the next track, that is, when playback stops with an error on x86).

[Montellese]

I have configured the Pi as a UPnP "server" with the following options enabled:

• Share video and music libraries through UPnP
  
• - Announce library updates via UPnP
  

The last option is crucial in order for UPnP sources to appear on the client - perhaps it could be worded more accurately? I thought it was referring to whether library updates were announced, when in fact this option appears to control the entire visibility of the library (which actually you'd think the first option was supposed to control).

That shouldn't be the case. If so it is a bug.

[Milhouse]

That shouldn't be the case. If so it is a bug.

Yeah sorry I updated the post - it may have been a glitch first time around. Now whenever I disable that second option on the Pi, the UPnP sources remain visible to x86.

[Tolriq]

Export your nfs share only to your rpi IP. And you'll know if the data goes through the rpi or not

I can see from the level of network traffic that the Pi isn't streaming anything - 323 bytes/second on TX/RX for a 1080p/DTS movie? That's some heavy-duty compression!

Edit: Also, I can shut down the Pi and the movie/music continues playing on x86 (until the next track, that is, when playback stops with an error on x86).

So you do realize that this is not normal. As explained earlier, if the NFS is only exported to rpi or using smb with password you will not be able to access the media.
And this is not really wanted.

There's no way to have some magic sources sharing like that.

[Milhouse]

So you do realize that this is not normal.

Hence, my confusion.

As explained earlier, if the NFS is only exported to rpi or using smb with password you will not be able to access the media.

Yet, I can.

Granted, the NFS share is not *only* exported to the Pi (it's visible to the entire subnet) but even when the x86 client knowns nothing about the NAS (in terms of sources.xml) it is still able to access the nfs:// media directly, because it's certainly not pulling this media via the Pi.

And this is not really wanted.

Really, why not? It seems like a perfectly reasonable thing to me, as I certainly don't want data to be streamed from my NAS to a separate UPnP server (which might not have much network bandwidth, CPU grunt, or both) only for the data to then have to stream somewhere else in order to reach the UPnP client for playback. Now *that* is definitely not wanted.

[m.savazzi]

Again let me try to see if I can separate the issue:
1) we need a DB abstraction layer. This will allow to use local file, NAS MySQL, Cloud services as user likes. da-anda explained how to do it in his great post #26 (http://forum.xbmc.org/showthread.php?tid...pid1753147). This will also make the XBMC code better as there will be a unified way of asking for data from libraries.
ODBC abstracts in the correct way all major data formats, they take care of the conversion between TEXT and VARCHAR of the different DBs.
For users this will not be more difficult, at basic level, than server/username/password. For advanced users they could do what they like.

2) is there is an issue with paths the solution is simply to store the UNC (\\servername\pathname\path\filename.ext ) instead of the filepath (v:\movies\movie1.ext) , especially on network shares. Otherwise there should be a path manipulation procedure that "alligns" the path stored on the db to the local machine... but I prefer the UNC

3) for the thumbnails cache it can be local (if is a real cache) build upon the central DB. If we use OOBE in the DB we can store the thumbnails on the central DB and copy them locally for performace considerations

4) network traffic will be IN ANY CASE lower than uPNP as only relevant data will go back and forth the content will go directly form the source to destination

As I wrorte before what I've read of the headless XBMC looks like a custom RDBMS server (this part) so I would not advise it as becomes a mess to manage in time.

Usually I try to tackle those big changes a step at the time.
The first one could be to replace the CDatabase, Video and Music libraries to use ODBC exactly like they are doing today.
Then pictures and other db can be added.

@da-anda: when you write "QOM I'm working with day by day:" means you're a super expert on this?
It has been quite some years since I worked on DB design and code in c++ and I can be a little rustied

[Tolriq]

Well an nfs:// path is generic and can use loaded from any XBMC.

But if the path is only exported to one XBMC then the second one accessing the UPnP libray will not be able to access the media : So useless UPnP sharing.
Now take the example of smb, if you add a password protected share to your main XBMC, then 2 possible things :
- Other XBMC host does not have access to the login password and as such cannot connect to the nas and no access to media.
- Main XBMC does send the login and password in clear form and it's a major security flaw.

Both case should not happens.

And there's lot's of more example.

So I'm sorry this is really not wanted and is not an normal usage of a library sharing feature.

[Milhouse]

Well an nfs:// path is generic and can use loaded from any XBMC.

But if the path is only exported to one XBMC then the second one accessing the UPnP libray will not be able to access the media : So useless UPnP sharing.
Now take the example of smb, if you add a password protected share to your main XBMC, then 2 possible things :
- Other XBMC host does not have access to the login password and as such cannot connect to the nas and no access to media.
- Main XBMC does send the login and password in clear form and it's a major security flaw.

Both case should not happens.

And there's lot's of more example.

So I'm sorry this is really not wanted and is not an normal usage of a library sharing feature.

OK, I can understand that when a client doesn't have access to the data then it can't stream it directly and must stream via the other device, but what I'm after is some clarity on this double-network bandwidth issue - the answer seems to be that it's not actually an issue (doesn't happen) at all whenever the client CAN access the data directly.

Perhaps if I restricted the export to only the Pi, the x86 data would start streaming via the Pi - if so, that's actually understandable. Assuming it works that way of course - does the client fall back to UPnP streaming when it experiences an NFS (or SMB, or whatever) error?

[Tolriq]

OK, I can understand that when a client doesn't have access to the data then it can't stream it directly and must stream via the other device, but what I'm after is some clarity on this double-network bandwidth issue - the answer seems to be that it's not actually an issue (doesn't happen) at all whenever the client CAN access the data directly.

Perhaps if I restricted the export to only the Pi, the x86 data would start streaming via the Pi - if so, that's actually understandable. Assuming it works that way of course - does the client fall back to UPnP streaming when it experiences an NFS (or SMB, or whatever) error?

Well you should test to confirm but I doubt there's fallback.

Anyway 80% + of users does use SMB with login password as NFS is more complicated for basic users, so will go through the main source or fail depending on current implementation.
And the rest as da-anda says earlier will mount locally the share so same result (but well there's no easy solution for those)

[Milhouse]

I restricted the NFS export to only the Pi, and when trying to play media on the x86 I now get an NFS error:

Code:

14:47:10 T:140542976878464   DEBUG: OnPlayMedia upnp://39c25aae-4226-1d78-fed0-2284f5a11214/videodb%3a%2f%2frecentlyaddedmovies%2f796/
14:47:10 T:140542976878464   DEBUG: CAnnouncementManager - Announcement: OnClear from xbmc
14:47:10 T:140542976878464   DEBUG: GOT ANNOUNCEMENT, type: 2, from xbmc, message OnClear
14:47:10 T:140542976878464   DEBUG: GetMovieId (upnp://39c25aae-4226-1d78-fed0-2284f5a11214/videodb%3a%2f%2frecentlyaddedmovies%2f796/), query = select idMovie from movie join files on files.idFile=movie.idFile where files.idPath=-1
14:47:10 T:140542976878464   DEBUG: CAnnouncementManager - Announcement: OnAdd from xbmc
14:47:10 T:140542976878464   DEBUG: GOT ANNOUNCEMENT, type: 2, from xbmc, message OnAdd
14:47:10 T:140542976878464   DEBUG: CUPnPDirectory::GetResource - resource protocol info 'xbmc-get:*:video/x-matroska:DLNA.ORG_OP=01;DLNA.ORG_CI=0'
14:47:10 T:140542976878464   DEBUG: CPlayerCoreFactory::GetPlayers(nfs://192.168.0.3/mnt/share/media/Video/MoviesHD/Transcendence (2014)[BDRip].mkv)
14:47:10 T:140542976878464   DEBUG: CPlayerSelectionRule::GetPlayers: considering rule: system rules
14:47:10 T:140542976878464   DEBUG: CPlayerSelectionRule::GetPlayers: matches rule: system rules
14:47:10 T:140542976878464   DEBUG: CPlayerSelectionRule::GetPlayers: considering rule: rtv
14:47:10 T:140542976878464   DEBUG: CPlayerSelectionRule::GetPlayers: considering rule: hdhomerun/myth/mms/udp
14:47:10 T:140542976878464   DEBUG: CPlayerSelectionRule::GetPlayers: considering rule: lastfm/shout
14:47:10 T:140542976878464   DEBUG: CPlayerSelectionRule::GetPlayers: considering rule: rtmp
14:47:10 T:140542976878464   DEBUG: CPlayerSelectionRule::GetPlayers: considering rule: rtsp
14:47:10 T:140542976878464   DEBUG: CPlayerSelectionRule::GetPlayers: considering rule: streams
14:47:10 T:140542976878464   DEBUG: CPlayerSelectionRule::GetPlayers: considering rule: dvd
14:47:10 T:140542976878464   DEBUG: CPlayerSelectionRule::GetPlayers: considering rule: dvdimage
14:47:10 T:140542976878464   DEBUG: CPlayerSelectionRule::GetPlayers: considering rule: sdp/asf
14:47:10 T:140542976878464   DEBUG: CPlayerSelectionRule::GetPlayers: considering rule: nsv
14:47:10 T:140542976878464   DEBUG: CPlayerSelectionRule::GetPlayers: considering rule: radio
14:47:10 T:140542976878464   DEBUG: CPlayerCoreFactory::GetPlayers: matched 0 rules with players
14:47:10 T:140542976878464   DEBUG: CPlayerCoreFactory::GetPlayers: adding videodefaultplayer (1)
14:47:10 T:140542976878464   DEBUG: CPlayerCoreFactory::GetPlayers: for video=1, audio=0
14:47:10 T:140542976878464   DEBUG: CPlayerCoreFactory::GetPlayers: for video=1, audio=1
14:47:10 T:140542976878464   DEBUG: CPlayerCoreFactory::GetPlayers: adding player: DVDPlayer (1)
14:47:10 T:140542976878464   DEBUG: CPlayerCoreFactory::GetPlayers: added 1 players
14:47:10 T:140542976878464  NOTICE: DVDPlayer: Opening: nfs://192.168.0.3/mnt/share/media/Video/MoviesHD/Transcendence (2014)[BDRip].mkv
14:47:10 T:140542976878464 WARNING: CDVDMessageQueue(player)::Put MSGQ_NOT_INITIALIZED
14:47:10 T:140542976878464   DEBUG: CRenderManager::UpdateDisplayLatency - Latency set to 0 msec
14:47:10 T:140542976878464   DEBUG: LinuxRendererGL: Cleaning up GL resources
14:47:10 T:140542976878464   DEBUG: CLinuxRendererGL::PreInit - precision of luminance 16 is 16
14:47:10 T:140541572400896  NOTICE: Thread DVDPlayer start, auto delete: false
14:47:10 T:140541572400896  NOTICE: Creating InputStream
14:47:10 T:140541572400896   DEBUG: SECTION:LoadDLL(libnfs.so.1)
14:47:10 T:140541572400896   DEBUG: Loading: libnfs.so.1
14:47:10 T:140541572400896   DEBUG: NFS: Context for 192.168.0.3/mnt/share not open - get a new context.
14:47:10 T:140541572400896   ERROR: NFS: Failed to mount nfs share: /mnt/share (mount/mnt call failed with "RPC error: Mount failed with error MNT3ERR_ACCES(13) Permission denied(13)")
14:47:10 T:140541572400896   ERROR: CDVDPlayer::OpenInputStream - error opening [nfs://192.168.0.3/mnt/share/media/Video/MoviesHD/Transcendence (2014)[BDRip].mkv]
14:47:10 T:140541572400896  NOTICE: CDVDPlayer::OnExit()
14:47:10 T:140541572400896  NOTICE: CDVDPlayer::OnExit() deleting input stream
14:47:10 T:140541572400896   DEBUG: OnPlayBackStopped : play state was 1, starting 1
14:47:10 T:140541572400896   DEBUG: Thread DVDPlayer 140541572400896 terminating
14:47:10 T:140542976878464   DEBUG: OnPlayBackStopped : play state was 3, starting 0
14:47:10 T:140542976878464   DEBUG: CAnnouncementManager - Announcement: OnStop from xbmc
14:47:10 T:140542976878464   DEBUG: GOT ANNOUNCEMENT, type: 1, from xbmc, message OnStop
14:47:10 T:140542976878464   ERROR: Playlist Player: skipping unplayable item: 0, path [upnp://39c25aae-4226-1d78-fed0-2284f5a11214/videodb%3a%2f%2frecentlyaddedmovies%2f796/]
14:47:10 T:140542976878464   DEBUG: Playlist Player: one or more items failed to play... aborting playback

Let me test SMB...

[Milhouse]

And SMB behaves the same as with NFS - the x86 client (which knows nothing about the SMB source, or the SMB username/password required to access the source) tries to stream direct from the SMB source, *not* via the UPnP server (the Pi).

Code:

15:02:14 T:140173212690304   DEBUG: OnPlayMedia upnp://39c25aae-4226-1d78-fed0-2284f5a11214/videodb%3a%2f%2frecentlyaddedmovies%2f801/
15:02:14 T:140173212690304   DEBUG: CAnnouncementManager - Announcement: OnClear from xbmc
15:02:14 T:140173212690304   DEBUG: GOT ANNOUNCEMENT, type: 2, from xbmc, message OnClear
15:02:14 T:140173212690304   DEBUG: GetMovieId (upnp://39c25aae-4226-1d78-fed0-2284f5a11214/videodb%3a%2f%2frecentlyaddedmovies%2f801/), query = select idMovie from movie join files on files.idFile=movie.idFile where files.idPath=-1
15:02:14 T:140173212690304   DEBUG: CAnnouncementManager - Announcement: OnAdd from xbmc
15:02:14 T:140173212690304   DEBUG: GOT ANNOUNCEMENT, type: 2, from xbmc, message OnAdd
15:02:14 T:140173212690304   DEBUG: CUPnPDirectory::GetResource - resource protocol info 'xbmc-get:*:video/x-msvideo:DLNA.ORG_PN=AVI;DLNA.ORG_OP=01;DLNA.ORG_CI=0;DLNA.ORG_FLAGS=01500000000000000000000000000000'
15:02:14 T:140173212690304   DEBUG: CPlayerCoreFactory::GetPlayers(smb://NM-WIN7/Torrent/qTorrent/Test/sample.avi)
15:02:14 T:140173212690304   DEBUG: CPlayerSelectionRule::GetPlayers: considering rule: system rules
15:02:14 T:140173212690304   DEBUG: CPlayerSelectionRule::GetPlayers: matches rule: system rules
15:02:14 T:140173212690304   DEBUG: CPlayerSelectionRule::GetPlayers: considering rule: rtv
15:02:14 T:140173212690304   DEBUG: CPlayerSelectionRule::GetPlayers: considering rule: hdhomerun/myth/mms/udp
<snip>
15:02:14 T:140173212690304   DEBUG: CPlayerSelectionRule::GetPlayers: considering rule: radio
15:02:14 T:140173212690304   DEBUG: CPlayerCoreFactory::GetPlayers: matched 0 rules with players
15:02:14 T:140173212690304   DEBUG: CPlayerCoreFactory::GetPlayers: adding videodefaultplayer (1)
15:02:14 T:140173212690304   DEBUG: CPlayerCoreFactory::GetPlayers: for video=1, audio=0
15:02:14 T:140173212690304   DEBUG: CPlayerCoreFactory::GetPlayers: for video=1, audio=1
15:02:14 T:140173212690304   DEBUG: CPlayerCoreFactory::GetPlayers: adding player: DVDPlayer (1)
15:02:14 T:140173212690304   DEBUG: CPlayerCoreFactory::GetPlayers: added 1 players
15:02:14 T:140173212690304  NOTICE: DVDPlayer: Opening: smb://NM-WIN7/Torrent/qTorrent/Test/sample.avi
15:02:14 T:140173212690304 WARNING: CDVDMessageQueue(player)::Put MSGQ_NOT_INITIALIZED
15:02:14 T:140173212690304   DEBUG: CRenderManager::UpdateDisplayLatency - Latency set to 0 msec
15:02:14 T:140173212690304   DEBUG: LinuxRendererGL: Cleaning up GL resources
15:02:14 T:140173212690304   DEBUG: CLinuxRendererGL::PreInit - precision of luminance 16 is 16
15:02:14 T:140171383121664  NOTICE: Thread DVDPlayer start, auto delete: false
15:02:14 T:140171383121664  NOTICE: Creating InputStream
15:02:14 T:140171383121664   DEBUG: CSmbFile::Open - opened Torrent/qTorrent/Test/sample.avi, fd=-1
15:02:14 T:140171383121664    INFO: FileSmb->Open: Unable to open file : 'smb://NM-WIN7/Torrent/qTorrent/Test/sample.avi'
                                            unix_err:'d' error : 'Permission denied'
15:02:14 T:140171383121664   ERROR: CDVDPlayer::OpenInputStream - error opening [smb://NM-WIN7/Torrent/qTorrent/Test/sample.avi]
15:02:14 T:140171383121664  NOTICE: CDVDPlayer::OnExit()
15:02:14 T:140171383121664  NOTICE: CDVDPlayer::OnExit() deleting input stream

Maybe it's a bug...

[m.savazzi]

hem... this is derailing a little on uPNP and similar.

uPNP is not a solution for media library sharing and synchronization. Is a solution to stream data among devices.

@Tolriq: if in the DB you store the correct UNC path

In a network, the Universal Naming Convention (UNC) is a way to identify a shared file in a computer without having to specify (or know) the storage device it is on.

but they are not related with uPNP

M

[Tolriq]

UNC is only tied to SMB, you forgot all the other protocols XBMC support. (And you completely miss the authentication problem that was already discussed in this thread).

I'm going to be a little but do not take it wrong (I add lot's of smiley to help).

But while the initial post was a good idea, your actual knowledge about XBMC internals, how database and abstraction layer, ORM, QOM, DAL and anything debated here are really too minimal to make your follow ups credible.

Your posts just looks like you do some Google on some terms, then post them here like you have found the Graal.

[m.savazzi]

sorry I'm focusing only on:
how to make the DN better.

Then there was an issue on the shared files... UNC is a solution for that and is easy to convert to NFS formats if SMB is not available

In the libraries there are not "other protocols", but maybe I'm wrong.
in any case we need to choose a file format name... and today we save in the database in the path table
idPath strPath strContent strScraper strHash scanRecursive useFolderNames strSettings noUpdate exclude dateAdded
10 smb://NAS-HOMELAN/vDisk1/Film/movie 1/VIDEO_TS/



For the odbc I've found this super interesting:
http://www.m2osw.com/odbcpp

The license of the odbcpp library is the GPL v3.0. This means you are free to use this library in your open source projects as much as you like.

For SQLite: http://www.ch-werner.de/sqliteodbc/
For MySQL: http://www.mysql.com/downloads/

[Tolriq]

Well time to remove notification from this thread

Library support way more than smb and UNC or whatever will not handle login and password unless plain text and it's not possible for obvious security reasons ....

Furthermore Xbmc is gpl v2 and cannot use gpl v3 library without upgrading to v3 and from what I read this is not desired...

So as I said earlier document yourself a little more before pasting Google

As all desired actions needed to go further with the DB was already answered by the team.