TMDB scraper not working with Year
#1
I have movie collections stored with the year as part of the folder name (eg 1979 - alien and 1986 - aliens) in XBMC. Each folder contains a single movie. Scanning used to work fine until yesterday when updates to the search function were made by TMDB. Now some movies are not found when searching with the year. A search for 1979 - Alien will give a result, but a search for 1986 - Aliens will not.

I contacted TMDB and this is their response:

XBMC must not use the ?year= parameter then. There is nothing we can do as this was something that had to be done deliberately in order to pave the way for a new, better search.

And just clarification sake, supporting the year at the end of your search string was never something we specifically supported in v3. The only reason XBMC ever had it like this was because they migrated from our old 2.1 API where it was supported. The very first day v3 went live, we changed this in our documentation to only support a proper `?year=' parameter. And the only reason it ever worked at all was because v3 used the same search as 2.1. That was 18 months ago though, so we've given everyone ample time to update to v3 and it was time to put some of those old hacks to bed.

If you contact XBMC, they'll be able to update the scraper (usually very quickly) and get this issue sorted on their end.


Where do I go for help with this?
Reply
#2
Can't follow.
Reply
#3
The scraper already does use the "?year=" parameter, but it requires XBMC (not the scraper) to be able to separate the year from the title - which it can't do if the year comes first as in your example folder names.
Only a "Title year" (or "... (year)", or "... [year]", or a few other variations) will work.

Presumably then your format was causing both year and title to be passed as the title in the search query, and the old search happened to be able to cope with that.

In short, not a fault of the scraper. Consider renaming your folders.
Reply
#4
Ok Thanks for the info
Reply
#5
I too am having this problem, however, my movies and folders are labeled with the date at the end, seperated by hyphen (eg, "Movie -2013", that is the name of the folder and the movie title).
And now when i update my movie library, nothing new is found, unless i go into videos, locate the file, click movie information, manually delete the date from the title, and then, low and behold, it finds my movie (still labeled with the same date i just deleted...derr)
Is this the same problem, or am i experiencing a different issue.
Thanks
Reply
#6
Looks like it's the same general issue, XBMC doesn't recognize the year right at the end of the folder name unless it's wrapped... or for some reason if there's a single non-numeric character after it (seriously, try calling the folder "Movie - 2013X" - it'll work).

The basic forms are:
  • "Movie (year)"
  • "Movie (year) blah blah"
  • "Movie [year]"
  • "Movie [year] blah blah"
  • "Movie.year.blah.blah"
  • "Movie - year blah blah"
  • "Movie, year blah blah"
(Where the blah blah can be anything, and will be ignored.)

There are a few extensions/variations obviously, you can see the regex being run here, if you can decipher it.

Edit: looks like the regex on the wiki isn't right, the one actually being run is:
Code:
(.*[^ _\,\.\(\)\[\]\-])[ _\.\(\)\[\]\-]+(19[0-9][0-9]|20[0-1][0-9])([ _\,\.\(\)\[\]\-]|[^0-9]$)
(Subtly different.)
Reply
#7
Kewl,
So if i go back a version, say to 12.2 the last stable release, or 12.1 previous, will that fix my problem, so i can leave all my 2000 movies and folders as they are. I really dont want to change them all, or put a stupid x on the end.
Or please can i request that someone looks into fixing this, cause really, i think i was fine before. I understand it is being changed for faster indexing, which is great too, but surely, this would just be a simple fix.
Cheers All
and keep up the great work, it is appreciated.
Reply
#8
No, the reason your folder names worked before is because the tmdb api search supported including the year as part of the title parameter. It no longer does.

Rolling back your XBMC version won't change that.

edit:
What you can do is replace the default cleandatetime with your own to deal with your folder names. All you need to do is make an advancedsettings.xml (wiki) in your userdata folder (wiki), containing:
Code:
<advancedsettings>
  <video>
    <cleandatetime>(.*[^ _\,\.\(\)\[\]\-])[ _\.\(\)\[\]\-]+(19[0-9][0-9]|20[0-1][0-9])([ _\,\.\(\)\[\]\-]|[^0-9]?$)</cleandatetime>
  </video>
</advancedsettings>
(If you've already made an advancedsettings.xml, you only need to add the video and cleandatetime lines inside the existing advancedsettings tags.)

Just be wary of something like "Death Race 2000" - be sure to put the real year after it.
Reply
#9
Ok, i am about to try this, so is this the correct fomat for the xml.

<advancedsettings>
<cachemembuffersize>0</cachemembuffersize>
<cleandatetime>(.*[^ _\,\.\(\)\[\]\-])[ _\.\(\)\[\]\-]+(19[0-9][0-9]|20[0-1][0-9])([ _\,\.\(\)\[\]\-]|[^0-9]?$)
</cleandatetime>
</advancedsettings>


ps...edit
Ok, ran with that, but didnt work... did i get something wrong there, (probbably..lol)
Cheers in advance.
Reply
#10
The only possible thing I can see is perhaps having the closing cleandatetime tag on a separate line is throwing off the pattern match (any whitespace/newlines might get interpreted as being part of the regexp).
Reply
#11
Ahhhhhhhh, I get it now, Will modify and test again.... Thanks
Reply
#12
Ok, so i edited the advancedsettings.xml and changed it so the cleandatetime is on one line. thus.

<?xml version="1.0"?>
-<advancedsettings>
<cachemembuffersize>0</cachemembuffersize>
<cleandatetime>(.*[^ _\,\.\(\)\[\]\-])[ _\.\(\)\[\]\-]+(19[0-9][0-9]|20[0-1][0-9])([ _\,\.\(\)\[\]\-]|[^0-9]?$) </cleandatetime>
</advancedsettings>

So, thats what i have, but it still didnt work. is there anything else you can see wong with my advancedsettings.xml file?
once again thanks for your help with his, really appreciated.
Reply
#13
Again, there still looks like a space between the regex and the closing cleandatetime tag which might be interfering.

I'd also remove the loose dash before the advancedsettings tag (just to be safe).

Try:
Code:
<advancedsettings>
  <cachemembuffersize>0</cachemembuffersize>
  <cleandatetime>(.*[^ _\,\.\(\)\[\]\-])[ _\.\(\)\[\]\-]+(19[0-9][0-9]|20[0-1][0-9])([ _\,\.\(\)\[\]\-]|[^0-9]?$)</cleandatetime>
</advancedsettings>
Reply
#14
Sad 
Ok, used copy and paste and copied your text exactly to my machine, saved and ran. But still no go.

edit..

Today I did a clean install of frodo 12.1 on my laptop, forgot about the advancedsettings file and ran. Would you believe it scanned the directory and added all the movies that were in avi format, but nothing in mp4 going to add the advanced settings.xml now and try again, just to see if theres a difference.

edit edit...

Something just occurs. You mentioned something about xbmc/scraper not recognising numbers at the end of the filename. As it is finding avi's and not mp4's could this have something to do with the file extention? .avi .mp4, just a thought, but the only thing i can think of.
Reply
#15
I'm having the same issue with TheMovieDB scraper, but my folders *are* in the "Title (year)" format. If I run a search on the collection folder no new movies get added, if I go and right-click each individual folder and view movie information I have to delete the (year) date from the end of the name and then it will find the movie and add it.

I also tried adding the cleandatetime regex posted above to my advancedsettings.xml file, but it made no difference (didn't really expect it to since supposedly my folders are in a supported format).

Any other thoughts? This is a recent issue, with in the last couple weeks.
Reply

Logout Mark Read Team Forum Stats Members Help
TMDB scraper not working with Year1