[garretn]

I'm having this problem with the October snapshot too. The duplicates enter my library on subsequent scans, but not during the "same" scan. Here's one of the files directly from sqlite to illustrate the problem:

sqlite> select * from episode where c18 = 'nfs://192.168.1.9/home/garret/shares/videos/Anime/One Piece/Season 07/[K-F]_One_Piece_098_[BC713814].avi';

8678|9105|Here Come the Desert Pirates! The Men who Live Free|Nami and Vivi found themselves an easier way to travel in the desert and leave all men behind. While trying to catch up with them, Ace gets lost. Luffy and the rest of the gang find the Desert Pirates, and Vivi meets a figure from her past.||6.000000||2002-01-27|<thumb>http://www.thetvdb.com/banners/episodes/81797/362005.jpg</thumb>||||||7|7||-1|-1|-1|nfs://192.168.1.9/home/garret/shares/videos/Anime/One Piece/Season 07/[K-F]_One_Piece_098_[BC713814].avi|742|||||310

8765|9105|Here Come the Desert Pirates! The Men who Live Free|Nami and Vivi found themselves an easier way to travel in the desert and leave all men behind. While trying to catch up with them, Ace gets lost. Luffy and the rest of the gang find the Desert Pirates, and Vivi meets a figure from her past.||6.000000||2002-01-27|<thumb>http://www.thetvdb.com/banners/episodes/81797/362005.jpg</thumb>||||||7|7||-1|-1|-1|nfs://192.168.1.9/home/garret/shares/videos/Anime/One Piece/Season 07/[K-F]_One_Piece_098_[BC713814].avi|742|||||310

And then the file it's linked to:

9105|742|[K-F]_One_Piece_098_[BC713814].avi|1|2012-10-29 22:39:33|2012-10-25 22:39:35

And the path:

742|nfs://192.168.1.9/home/garret/shares/videos/Anime/One Piece/Season 07/|||||||||

Notice they even link to the same file. What's even more strange is it seems random, sometimes when scanning for new content it only adds actually new content. Other times, it'll duplicate files for an entire folder (such as a TV season) that had no new episodes/hadn't changed in a long time. This last time it added a TON of duplicates from multiple folders and I actually stopped it during scanning since it was duplicating so many.

I'm going to try with the November cycle that I see appears to be available, first with my current database, but afterwards I'll probably remove the database and build it from scratch at this point.

I do use the advanced alternate setting for DateAdded that returns it to the old behavior during content scanning, which I only changed a few weeks ago. The previous post made this occur to me. I have a rather large content library that I update frequently, so I'm not positive about which files entered my library when, but since I updated from stable to snapshots periodically, and eventually realized the DateAdded behavior changed, I went from the "old" scanning behavior to new, and then back to old once I found out it changed and how to change it back. Media added during that time would've had the new behavior "applied" to it. Perhaps this issue is ultimately related to that? Even though the file entries are exactly the same, it seems like it could be related.

This is the thread I'm referring to regarding DateAdded:

http://forum.xbmc.org/showthread.php?tid=127343

[cocjh1]

Ran into this problem this evening!

Added some new content tonight, for the first time in a long while (months probably). All new content appeared twice, but existing content was ok. Thought I'd had this before and remembered the "Clean database" option in settings made no difference (and sure enough this time it didn't help). So, went ahead and deleted my entire "TV Shows" source folder, then added it right back in again, and XBMC updated the library accordingly. Now everything is listed twice! Great

Grabbing my database off the ATV2 and viewing with SQLiteSpy I can see the problem. All episodes have two entries in the "files" table with same strFilename string but different idPath integer. The strPath field in the "path" table for the two entries are different as follows:

75 | nfs://192.168.0.18:2049/volume1/videos/tv_shows/Wallander/
95 | nfs://192.168.0.18/volume1/videos/tv_shows/Wallander/

So, yes, I've got duplicate paths!

[rimster]

Just thought i'd give this thread a bump again as the problem still remains in Frodo Beta 1.

Any of the dev's looking into this perhaps... i really don't want to rebuild me complete tv collection again...

[rimster]

small update on this matter. In the Frodo Beta 1 it seems that if you remove a show and do a rescan it removes the duplicates and retains watched information. Not ideal, but it's a workaround i can live with for the moment. I do hope someone will look into this though.

[Tobisas]

I've had this annoying issue for a while too, has anyone yet found a fix or workaround yet? I'd like to find out what's actually causing it..

Here's the information I've gathered so far (bolded which I think is most important):

• I've had this issue for a few months, on separate (clean) installs of different XBMC versions.
  
• Cleaning library never worked
  
• Dumping the DB and rescanning works.. But takes a long time for me, and the issue will quickly return anyways.
  
• I do not have separate source paths pointing to the same media files.
  
• In the SQLite DB the entries in the episodes table are identical, except for the idEpisode. I've checked this multiple times.
  
• There are no duplicate entries in the paths or files tables (wouldn't expect there to be, since the episodes link to the same idFile)
  
• It only seems to occur for anime episodes downloaded through my sickbeard anime + sabnzbd setup
  
• Oddly enough though, I also download/process 'normal' tv shows through this sickbeard instance and they do not create duplicates
  
• I've tried comparing the metadata which sickbeard creates for one of the problematic anime episodes to a normal show episode, but cannot find anything different. There aren't multiple nfo files, and the content seems the same except for the string data.
  
• It doesn't seem to be caused by XEM, since some anime shows which don't use XEM still generate duplicates while tv shows which do use XEM don't generate duplicates.
  
• The anime and normal tv shows use the same scraper and scraper settings, they're just in different root folders.
  

I'm not sure how pizzachef's odd fix would apply to me since my folders don't share the same name. One thing I'm trying now is turning off the "Update library" option in the notification section of SickBeard. I have no idea whether that will do anything, but who knows, gotta try something

I just don't understand how one folder with tv episodes and nfo files can cause xbmc to generate duplicates, while another similarly structured folder is fine.. But I must be missing some info on how XBMC actually does the scraping or why it would ever generate a new episode entry for a file which already exists..

Copy pasted example of duplicate episode entries from SQLiteSpy:

Code:

idEpisode    idFile    c00    c01    c02    c03    c04    c05    c06    c07    c08    c09    c10    c11    c12    c13    c14    c15    c16    c17    c18    c19    c20    c21    c22    c23    idShow
17554    9748    The Genius Who Descended Into the Darkness    In the Midori Mahjong parlor, Nangou is trying to cancel his major debts by playing some Mahjong but this may be his unlucky day as he is losing badly. Just when the situation gets dire an unknown boy walks in on their game and with a simple word of advice and Nangou is able to survive. This may be the beginning of the legendary Shigeru Akagi who will shake the underworld.        0.000000        2005-10-05                            1    1        -1    -1    -1    smb://TOBINAS/Multimedia/Anime/Akagi/Akagi - 001 - The Genius Who Descended Into the Darkness.avi    853                    288
25265    9748    The Genius Who Descended Into the Darkness    In the Midori Mahjong parlor, Nangou is trying to cancel his major debts by playing some Mahjong but this may be his unlucky day as he is losing badly. Just when the situation gets dire an unknown boy walks in on their game and with a simple word of advice and Nangou is able to survive. This may be the beginning of the legendary Shigeru Akagi who will shake the underworld.        0.000000        2005-10-05                            1    1        -1    -1    -1    smb://TOBINAS/Multimedia/Anime/Akagi/Akagi - 001 - The Genius Who Descended Into the Darkness.avi    853                    288

[Tobisas]

Okay I think the problem was in the episodes using absolute numbering, so the episodes were titled something like

"Akagi - 001 - The Genius Who Descended Into the Darkness.mkv" instead of
"Akagi - 1x01 - The Genius Who Descended Into the Darkness.mkv"

Without an episode nfo metadata file, XBMC wouldn't even recognize that episode, but with the nfo it apparently kept adding it to the library for some reason I am not aware of.

Or at least that's what I think is the problem. I renamed everything so it includes the normale episode numbering, deleted all the episode metadata files, dumped the XBMC video DB and rescanned everything. So far it seemed to have recognized all my episodes and for the past week I haven't encountered any duplicates yet, so hopefully that was the solution.

[nemesis4670]

Just thought I'd share how I fixed this. Although I'm not using MySQL databases I'm just using XBMC and loading my regular movies/tv shows in folders into XBMC's database.

I just simply right clicked on the show in question and went to TV SHOW INFORMATION. Then at the top (using rapier skin) there is a button labeled "REFRESH".

A popup appeared saying "Refresh info for all episodes?" and I clicked YES and then a thing popped up showing the search results for that show. I'm guessing the results are database entries that load the info for the show. I was fixing the show Weeds so it showed both Weeds and some other show called Weeding or something weird. I clicked on the proper name of the show and the episodes updated and all the duplicates were removed.

Hope that helps someone.

[tvont]

What I did and recommend - assuming you have exact duplicates in the path table of your mysql database - is the following:

CREATE TABLE path_copy AS SELECT * FROM path GROUP BY strPath;
REMAME TABLE path TO path_save; #(can be 'DROP TABLE path', this is just to make a copy of the original if sth goes wrong)
RENAME TABLE path_copy TO path;

Afterwards one has to run a clean database in XBMC - dupes are gone

If you have only similar duplicates pointing to the same directory, ie. 'smb://server/h$/TVSeries/Myseries' and 'smb://login:password@server/h$/TVSeries/Myseries'
you have to equalize the paths before (and get rid of a possible existing passwords.xml in userdata):

UPDATE path SET strPath = REPLACE(strPath, 'smb://server', 'smb://user:password@server');

[doug]

I had duplicate entries for all my movies. I think it was because I changed my sources from a hostname to an ip address. So, I had paths like "smb://MYCOMPUTER/path/to/movies" and "smb://192.168.0.100/path/to/movies". And both instances of each movie would both play in xbmc because MYCOMPUTER resolved to 192.168.0.100. I share a database with MediaPortal.

My fix:
-From a command line with admin rights: path/to/MySQL/bin/mysql -u xbmc -p myvideos75 (then enter your database password when prompted)
-From the MySQL prompt: DELETE FROM path WHERE (strPath like '%smb://MYCOMPUTER%');
-start xbmc and clean the video database.

I know almost nothing about SQL, but the previous posts and some googling allowed me to create that command. Be careful with that command as it can delete any record containing "smb://MYCOMPUTER". If you use it and change "smb://MYCOMPUTER" to something simple like "the" you'll delete a lot of records.

[marshalleq]

Well, for my 2c this has only been happening since adding MYSQL server. And it happens with only ONE client connected as I found out today. Has been happening during all the RC's and betas and was not a result of upgrading a database since I have never done that. This sits firmly and squarely in the space of XBMC connecting to mysql and however those two programs interact. If the various reports above are the exact same problem then I understand this is not limited to Mysql so the only common factor is xbmc.

* Not limited to MYSQL
* Not due to duplicate XBMC instances
* Not due to upgrading database
* Not due to multiple paths on a single XBMC install
* Not specific to any particular OS
* Seems to be TV shows only for people with this exact problem (there are many ways movies etc can duplicate)

Bug fix required. Has anyone actually logged a bug?

[paisley]

I had duplicate entries for all my movies. I think it was because I changed my sources from a hostname to an ip address. So, I had paths like "smb://MYCOMPUTER/path/to/movies" and "smb://192.168.0.100/path/to/movies". And both instances of each movie would both play in xbmc because MYCOMPUTER resolved to 192.168.0.100. I share a database with MediaPortal.

My fix:
-From a command line with admin rights: path/to/MySQL/bin/mysql -u xbmc -p myvideos75 (then enter your database password when prompted)
-From the MySQL prompt: DELETE FROM path WHERE (strPath like '%smb://MYCOMPUTER%');
-start xbmc and clean the video database.

Thanks Doug worked a treat for me.

I used my Mac's built in SQLLite3 command to access the XBMC database mounted via SMB on my Windows system...
sqlite3 /Volumes/XBMC/userdata/Database/MyVideos75.db

[dgcruzing]

I had duplicate entries for all my movies. I think it was because I changed my sources from a hostname to an ip address. So, I had paths like "smb://MYCOMPUTER/path/to/movies" and "smb://192.168.0.100/path/to/movies". And both instances of each movie would both play in xbmc because MYCOMPUTER resolved to 192.168.0.100. I share a database with MediaPortal.

My fix:
-From a command line with admin rights: path/to/MySQL/bin/mysql -u xbmc -p myvideos75 (then enter your database password when prompted)
-From the MySQL prompt: DELETE FROM path WHERE (strPath like '%smb://MYCOMPUTER%');
-start xbmc and clean the video database.

Thanks Doug worked a treat for me.

I used my Mac's built in SQLLite3 command to access the XBMC database mounted via SMB on my Windows system...
sqlite3 /Volumes/XBMC/userdata/Database/MyVideos75.db

I also had to go this route but I am able to use PhpMy admin within the Synology NAS
I was able to go in to the MyVideos75 folder and find the "tvshow" entry and delete the secondary path that was getting entered.
I then came out and gave it a clean and foolishly did a libupdate from with in the system menu and it went and re did some of the same path problems again..
this time I have deleted the paths in question and only left the ip address and are added in the shows/seasons one by one and it seems to be holding its own.
Will report back on how it goes..

[looboomoo]

I have PhpMy admin within Synology NAS, but can't seem to get anywhere on it - certainly can't find the MyVideos75 folder, and neither does the my fix work (tried putting it in SQL window and got an error that the syntax was incompatible, and when I put it into the c: on my computer, it did nothing.

As you can tell, I don't really know what I'm doing.... If you could provide a bit more information around the fix, I'd really really appreciate it...

sorry, its ok - I wasn't logged onto phpadmin as root
all good now