NFS issues continue with Gotham beta 2

  Thread Rating:
  • 0 Votes - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Post Reply
Memphiz Offline
Team-XBMC Developer
Posts: 10,675
Joined: Feb 2011
Reputation: 112
Location: germany
Post: #16
Well that stale thingy is nothing special to nfs. If you use smb and kill the server during update it will nuke most of your items too i guess (nothing special in the nfs case here).

The thing with double added entries is interesting indeed (you could check the path on 2 identical entries in the movie info dialog and verify that it is equal ... remember nfs://ip:2049/foo != nfs://ip/foo - so even if those urls are equal based on what they connect too - they are not equal to our library). But also on that one i tend to think its a generic issue and not only related to nfs. *guessing*

AppleTV2/iPhone/iPod: HowTo find debug logs and everything else which the devs like so much: click here
HowTo setup NFS for XBMC: Wiki NFS
HowTo configure avahi (zeroconf): Wiki Avahi
READ THE IOS FAQ!: iOS FAQ
find quote
garretn Offline
Member
Posts: 76
Joined: Oct 2010
Reputation: 3
Post: #17
Did, the file entry isn't duplicated -- only the library entry in the movie table (which itself doesn't have path information). Both entries link to the same idFile in the files table. Setting the content type back to none and selecting to remove all library items in that path does clear them out, and scanning them back in after that gets rid of the duplicates.

-- And I agree, duplicates probably aren't related, however... --

Regarding NFS -- thats what I was implying. I wanted to see if I could reproduce the symptoms with a scenario that didn't really indicate an actual bug with libnfs itself (I wouldn't consider a stale NFS mount a bug with libnfs, it's really more of a hazard of using NFSv3 in general (where its not exactly uncommon)). There are certainly things that could be done to try and mitigate the issue, but that's not quite the same problem.

Discounting the reports from the downgraded libnfs (like mine, and some others), I've only come across two (if anyone has more, I'd appreciate a link!) reports of the original issue -- one of which doesn't actually provide any information, so it's difficult to for-certain associate it with the issue.

At the moment I suspect something in bossanova808's setup is producing a stale-mount-like case, or perhaps it really is just stale-mounts. I'd really like to see if he can reproduce it using the other NFS server software, since I can't reproduce the issue.
(This post was last modified: 2014-03-26 23:17 by garretn.)
find quote
bossanova808 Offline
Donor
Posts: 2,186
Joined: Sep 2009
Reputation: 32
Location: Melbourne, Australia
Post: #18
@garretn

Veyr large library I guess - 800 odd movies, 203 TV shows spanning 10 years.

I first noticed the issue only a bit with Frodo 12.3 - there was some sudden sporadic playback issues. Then I tried a clean library on that macine and lost some of the library. So then I moved to testing on my laptop (wireless), 12.3 and Gotham, and found it nuked my library in entirety - WITH THE EXCEPTION OF SOME ISOs - and consistently. I think the ISOs are not removed because of some sort of 'removed disc' protection more than that they are 'found' or anything. That is - I think the stat fails on the *whole* library.

My server is not very busy at all, an i7 and (before all this) easily capable of sending info to 5+ clients at a time while cleaning xbmc on the master. Indeed I doubt I've ever come even close to hitting the ceiling of it, and i have tested it sending music to 4 clients, video to 5 clients, and doing a variety of post processing tasks at the same time on the server itself without issue. I'm confident there is no general issue with the server or network.

I agree focussing on libnfs 1.6.3+ is the way to go BTW. And it's awesome to have someone else looking at it.

I will try allegro with gotham beta1 and therefore libnfs 1.6.3 tonight with any luck although so far I have just tried that with everytbhing on the one machine, so even that was not a great test. I have to do all this at night after my young kids and wife are in bed, so time is a challenge!!

This being said, I have used Hanewin to server NFS reliably to a whole host of media players for a long long time now, and still do in other non xbmc contexts, without any issues. With xbmc itself I've had nop issues since direct nfs mountining came along until immediately with the libnfs bump they started. I think to reproduce it you might want to try wireless or similar as it was definitely much worse in that case.

My XBMC stuff:
find quote
garretn Offline
Member
Posts: 76
Joined: Oct 2010
Reputation: 3
Post: #19
The sporadic playback issues are very suspect, and lend credence to the theory that this is ultimately just an issue with stale mounts. Is *all* of your testing via wireless connections, or has this ever happened with a wired connection? If thats the case, I would think going back to libnfs 1.6.3 for Gotham now is the way to go, as we're likely looking at simple tweaking of some nature. If its a simple network issue, tweaks would be reasonable to include in 13.1 or later releases etc and soforth.

To let you in on my thought process, I'm thinking (and suspected from the beginning, which is why I was testing with stale mounts earlier) that the newer libnfs may have different cache/timeout/behavior that simply makes it more likely to have issues in the event of a poor network connection. Personally, my entire house is wired (3500 sq. ft.), so if this is the case, it makes sense that I've never ran into the issue -- I rarely use wireless (which is mostly available for embedded devices and the like, where wired isn't an option).

Also -- I'd call that a medium sized library, based on no data whatsoever (Personally, I roll with a 21TB media server / 500 movies / 500 TV shows). Smile
find quote
bossanova808 Offline
Donor
Posts: 2,186
Joined: Sep 2009
Reputation: 32
Location: Melbourne, Australia
Post: #20
It initially happened with Frodo 12.3 on my master, wired HTPC. I did not try it for long on that as it killed my library so pretty low WAF on that!! But it definitely happened wired as well (stable as hell gigabit ethernet).

Since then pretty much all testing was laptop, wireless.

It probably is something like that indeed - bit it affects wired clients as well. If they go back to 1.6.3 they will have users losing libraries. If they stay with current they will have users without artwork. The first is probably worse...anyone without a library backup (I do mine daily via mysqldump)...will be very unhappy.

I don't think it's going to be a simple tweaking scenario....

My XBMC stuff:
find quote
brazen1 Offline
Senior Member
Posts: 122
Joined: Dec 2012
Reputation: 1
Post: #21
Just confirming that artwork is sporadic using nightly from the 25th. There is no rhyme or reason why some titles show artwork and others don't. Originally, Gotham nightlies failed to scan in new items at times as well as wipe out the library after cleaning it and double entries for titles that already existed. I installed the custom build from this post http://forum.xbmc.org/showthread.php?tid=182638 as suggested and all was well. New items scanned in, all artwork showed and the title count was correct after cleaning library multiple times. I installed this latest nightly from the 25th and at least half the artwork is missing. I set the collection to use local information only which lives in each titles folder on the server. That made no difference confirming if the scraped server was down or not. I then set the scrape back to universal movie scraper and It still won't show artwork. I manually refresh each title from the info tab and select choose art using universal scraper and local. Both ways give no choices of anything. Wired or WiFi makes no difference. All the artwork for all the titles has appeared pretty much faithfully until this latest build install.

I don't understand the comment about 1.63 losing libraries and current losing artwork? Unless my prior testing was incomplete, both were working using that custom build.

My server is W7x64/Hanewin NFS3 to a W8.1x64 client with mixed content including large iso's.
(This post was last modified: 2014-03-27 04:05 by brazen1.)
find quote
bossanova808 Offline
Donor
Posts: 2,186
Joined: Sep 2009
Reputation: 32
Location: Melbourne, Australia
Post: #22
Yes the artwork issues do appear to be some new thing on top. At least I didn't notice it with any testing with the libnfs 1.3 back in until Beta2 came along.

My XBMC stuff:
find quote
garretn Offline
Member
Posts: 76
Joined: Oct 2010
Reputation: 3
Post: #23
Sorry brazen1, the artwork comments and some details on this issue were/are split with another thread -- this was my fault.

Basically the issue as it stands is in Beta 1, libnfs was upgraded to version 1.6.1. With this version came the problems of randomly removing items (up to the entire library) on NFS shares, along with double entries when adding back items that were removed as a result of this issue. As a result of these problems, in builds newer then Beta2 libnfs was downgraded (for windows only!) back to the previous version which fixed the removed+duplicate items issues, but introduced the artwork issue.

For artwork, there is actually rhyme and reason to what's missing -- basically, any artwork that XBMC has to pull from an NFS share would be missing, and anything that was either already cached -- remembering that it only pulls this stuff *the first time* from the actual NFS share (with some exceptions that aren't terribly relevant) -- or simply not pulled from the NFS shares (remote images from online scrapers) will show fine.

The issue we're working on is the original issue, with libnfs 1.6.1 and the library problems. The artwork issue is a transient problem caused by a broken fix, so will simply go away once we resolve the original problem. The reports seem to indicate that the 1.6.1 problem is primarily an issue with windows-based NFS servers, however we've only two reported cases of that. I also use the windows XBMC build, but use a linux server on the back-end -- which so far hasn't shown any issues cleaning the library. Likewise, it's simply possible that I just haven't run into the issue yet. Bossanova808 did some testing that seems to indicate its not just Hanewin also, but afaik, still Windows.

I'd *really* like to know if anyone has the problem with a linux server as the backend.
find quote
garretn Offline
Member
Posts: 76
Joined: Oct 2010
Reputation: 3
Post: #24
boassanova808, did you ever get a chance to try 1.6.1 with allegro?
find quote
patseguin Offline
Donor
Posts: 1,363
Joined: Jan 2012
Reputation: 8
Post: #25
It seems like SMB is probably best atm for Windows environments. I am fully sold on OpenElec thanks to a fellow XBMC forum user and NFS works perfectly, in fact a lot better, than SMB. Unless you really need Windows on your htpc, possibly consider OE or XBMC on Ubuntu.

Server: Synology Diskstation 1511+ with 8x 3TB drives and DSM 5.0
Main HTPC: Home built Intel core i3, 8GB RAM, Corsair 128GB SSD, nVidia 630GTX, Rii micro keyboard remote, Pioneer VSX-53, Panasonic VT30 65" 3D TV, OpenElec 4.1.2
Bedroom HTPC: Zotac-ID 41 8GB RAM, 128GB SSD, Rii micro keyboard remote, Samsung HW-E550, Sony Google TV, OpenElec 4.1.2
find quote
brazen1 Offline
Senior Member
Posts: 122
Joined: Dec 2012
Reputation: 1
Post: #26
Switching O/S to accommodate NFS isn't an option for many of us imo. Personally I use Windows on 1 PC to do EVERYTHING and I mean EVERYTHING including being an HTPC and a common everyday computer tasking many functions besides multimedia home theatre entertainment. I have added an AVR and some USB peripherals. There is nothing a NAS, standalone, or Linux/Unix/OpenElec, etc O/S is going to do more efficient, with ease of use, functionality, economically or popularity than my little ol' W7 PC aside from this XBMC NFS Windows issue and I can back that statement up.

SMB was ok for DVD material in the past. Not anymore. Streaming Blu-Ray WiFi with the best of routers and adapters is still impossible. Using AC1900 right now. Some titles work without buffering, some don't. NFS helps a bunch. Powerline adapters are also finicky. Wired there is no problems but many of us cannot add Ethernet cable throughout our homes so we depend on WiFi and that Wifi depends on NFS now and that NFS depends on Windows. Well, Microsoft gave up on NFS. It isn't even included in Windows anymore. So, we go to a 3rd party (Hanewin). I've been using it long enough to know it works better than SMB, it's simple, and I like it.

The point is, switching O/S from Windows to something more easily compatible with NFS is out of the question for users like me. I will use the broken functionality of libnfs and or an XBMC backward version before I would ever change my O/S or add a standalone device. It would be a step backward, not forward. Please pursue this issue and don't give up on it. I don't write code and I am not as techy as others here but I can follow directions and offer help where I can.
find quote
garretn Offline
Member
Posts: 76
Joined: Oct 2010
Reputation: 3
Post: #27
Lets give him the benefit of the doubt, he probably just meant SMB is your best bet *in Gotham* if we can't get this worked out.

I've no intention of giving up on it at the moment, being still early diving into the issue if that makes anyone feel better. However I've still not managed to reproduce the issue in the same way it's been reported, so it might fall on you guys that do have the issue to do some testing. At the moment it'd help with diagnosing the problem if it can be reproduced on either linux, or on windows using the allegro nfs server.

So far the only testing I've seen (unless I misread somewhere) with Allegro was in regards to the Art issue, and not the original problem, with the original problem being reported with Hanewin in both cases. If we can narrow it down, we should have better luck reproducing the problem, so it can actually be fixed.
find quote
gibxxi Offline
Fan
Posts: 377
Joined: Oct 2011
Reputation: 4
Post: #28
garretn, I doubtless am not alone in hoping you get to the bottom of this. I'm a hardware-leaning sort of PC geek, but 99.5% of my knowledge/experience is with Windows (Right back to about '92-'93). I know very little about Linux. I know enough to do basic maintenance of my ReadyNAS devices over SSH, but that's my limit.

To the point... I've been using NFS on Frodo for the past 12 months. I switched over to it at that time, as back them I was using a Wireless adapter with my HTPC and access latency / overall throughput was a bit lacking (Wireless-G back then). I've since dabbled with Powerline, Powerline AV-500, a Wired connection to a Netgear WNCE4004 Wireless Acces Point/Bridge and now a D-Link DAP1522/B Wireless-N (300) Access Point / Bridge. The router is a WNDR3200 (Also Wireless-N 300 AFAIK).

I've not 'seen' issues with library files going AWOL as far as i've noticed, what I have witnessed on Frodo is problems with cdART Manager sometimes not saving files it downloads back to the NAS (CD artwork, folder.jpgs, fanart / etc) evidenced by blank areas when it shows you downloaded art at the end of a run. At the time I put this down to permissions on some of the folders on the NAS not being set properly by some of the apps in use outside of XBMC <cough!>Headphones</cough!> so put that issue down to this. The thing being when I did get a blank image post cdART Manager run, on that system, usually it was 1-2 images, tops.

Likewise in Beta 1 (albeit for the limited amount of time I was playing around with it before Beta 2) I was getting multiple entries for the same album, like has been mentioned in this thread, but again I put that down to some albums having tracks that were extras, or that release I have not having been uploaded to MusicBrainz. Corrected two entries for two CD's I own just this week to correct issues like this.

But Since Beta 2, no artwork over 300*300px (approx) for the Music Library would load. About 80% of the entire library had no artwork. But, some of it was working. Funny thing is, all the backdrops / thumbnails, and poster art for Movies was working fine. But then again, having done a clean install, this may well have been because of Artwork Downloader / tinyMediaManager is/was using remote links for artwork in the nfo files for Movies / TV rather than the local files. I didn't think to check. Now i'm back on SMB since it looked like NFS being a long-term breakage. That's been the sum total of my experiences up till this point.

I will point out that I was using a MySQL Db also running on the NAS through the Eden-Frodo period. But the additions to Gotham make library scraping so terminally slow over MySQL, i've disabled this too now. The convienience of not having to update 3+ installs manually was being severely offset by the length of time required for a single scan to complete (12 hours+ for 25,000 track library or thereabouts). Movies are/were still as swift as they've always been with local nfo files, and not really too shabby (even with MySQL) using remote info.

[Image: all-thin-fanart@2x.jpg]
DT: Intel Core-i7 - 4770K / Nvidia GTX-770 | HTPC: Intel Core-i5 - 4670K / Nvidia GTX-750 Ti
TV: Sony KDL-40EX524 | AVR: Onkyo TX-NR515 | SPK: 2x Pioneer CS-7070 / 3x Tannoy F1 Custom
find quote
gibxxi Offline
Fan
Posts: 377
Joined: Oct 2011
Reputation: 4
Post: #29
One thing I am somewhat confused by In Gotham is the fact that cdART Manager seems to be matching very few of my artists with the MusicBrainz server, even though my collection has recently been pushed through Picard for the first time. So now I'm getting only about 6 artist matches with a better organized library than I was getting on Frodo with a reasonable (but typically not MusicBrainz compliant) library. I'm mentioning this because I'm sure I read a post somewhere about some of the network time-outs people are commenting about, not being limited to NFS alone, but also SMB for Windows users.

I'm going to do another read-through of the cdART Manager thread before I lay the blame for this side-issue at Gotham OR cdART Manager though.

[Image: all-thin-fanart@2x.jpg]
DT: Intel Core-i7 - 4770K / Nvidia GTX-770 | HTPC: Intel Core-i5 - 4670K / Nvidia GTX-750 Ti
TV: Sony KDL-40EX524 | AVR: Onkyo TX-NR515 | SPK: 2x Pioneer CS-7070 / 3x Tannoy F1 Custom
(This post was last modified: 2014-03-28 01:03 by gibxxi.)
find quote
bossanova808 Offline
Donor
Posts: 2,186
Joined: Sep 2009
Reputation: 32
Location: Melbourne, Australia
Post: #30
@gibxxi - have you explicitly used Frodo 12.3 with your NFS server? Frodo 12.3 was where the issues with library dumping started, but your server is not Hanewin.

@garretn - I have not had time to do much with allegro yet but over the weekend should be ok to do some solid testing. But to duplicate you really have to try something fairly similar I'd say - so not everything in a VM, but hanewin in it's own place, over the network, to xbmc clients on other windows machines (ideally both wireless and wired).

My XBMC stuff:
find quote
Post Reply