How to find duplicate movies?
#1
I have a 20 drive NAS with SD and HD files all over each drive. Now I want to have just HD files, so when I grad an HD file I more than likely already have in SD. How can I find the duplicate SD file so I can delete it?
Reply
#2
Assuming your video files are named appropriately, you could write a script (probably utilising regular expressions) to first: Get a list of all files; then search recursively through that list for similar names.

Alternatively, just turn on XBMC library, Run a info scrape, and it will show doubles of each movie (again, assuming your folders/files are named appropriately).
...DJVege...
HTPC: G840 + GT430 + 2GB RAM + 30GB SSD
Remote: iPad/iPhone + iRule + USB-UIRT // FireFly RF Remote
Software: XBMC Eden + Eventghost + Win7
Reply
#3
I know this is a old thread but I would like to know if there is any way to find duplicate movies in XBMC other than simply brows thru the entire collection, when you have a lot of movies that's not really practical.
I have about 2500 movies and 10000 TV episodes and it would be great if there was some addon or script that could do this automatically just like the clean library function.
Image
Reply
#4
Hello

i use a mysql backend to store the information. I do a simple select statement (select idMovie,count(c00) as moviecount,c00,c22 from xbmc_video.movie group by c00 having moviecount > 1)
This shows me the duplicate movies.
If you use the internal sqlite database this should work also.
Reply
#5
download media companion set the folders for all movies theres a filter you can select that shows duplicates once youve delted all the duplicates clean the xbmc library and you should be fine
Reply
#6
I'm late to the party here, but this maybe useful for others googling the same problem...

I use a MySQL db and the following query shows all duplicate Movie title/year combos

Code:
select idMovie, concat(movie.c00, ' (' , movie.c07, ')') as title_year, movie.c22
from movie
inner join (
                select c00, c07 from movie
                group by concat(movie.c00, ' (' , movie.c07, ')')
                having count(idMovie) > 1) dup on concat(movie.c00, ' (' , movie.c07, ')') = concat(dup.c00, ' (' , dup.c07, ')'
            )
order by title_year
;
Reply
#7
(2014-05-23, 01:43)Dave.Agent Wrote: I'm late to the party here, but this maybe useful for others googling the same problem...

I use a MySQL db and the following query shows all duplicate Movie title/year combos

Code:
select idMovie, concat(movie.c00, ' (' , movie.c07, ')') as title_year, movie.c22
from movie
inner join (
                select c00, c07 from movie
                group by concat(movie.c00, ' (' , movie.c07, ')')
                having count(idMovie) > 1) dup on concat(movie.c00, ' (' , movie.c07, ')') = concat(dup.c00, ' (' , dup.c07, ')'
            )
order by title_year
;

What is the SQL lite equivalent?
Reply
#8
The best way so far is use Ember Media Manager, and filter "Duplicated Movies"
Reply
#9
Or run the script in my signature, "./texturecache.py duplicates" to get a list of duplicate movies based on matching imdbnumber.
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply
#10
(2014-08-18, 05:27)Milhouse Wrote: Or run the script in my signature, "./texturecache.py duplicates" to get a list of duplicate movies based on matching imdbnumber.

Hi

i tried running it but get this error..

FATAL: The task you wish to perform requires that the JSON-RPC server is
enabled and running on the XBMC system you wish to connect.

In addtion, ensure that the following options are ENABLED on the
XBMC client in Settings -> Services -> Remote control:

Allow programs on this system to control XBMC
Allow programs on other systems to control XBMC

A connection cannot be established to the following JSON-RPC server:
localhost:9090

Check settings in properties file texturecache.cfg




I already enable

XBMC client in Settings -> Services -> Remote control:

Allow programs on this system to control XBMC
Allow programs on other systems to control XBMC

I want to learn sql lite

Anyone know what is the equivalent sql lite command?

select idMovie, concat(movie.c00, ' (' , movie.c07, ')') as title_year, movie.c22
from movie
inner join (
select c00, c07 from movie
group by concat(movie.c00, ' (' , movie.c07, ')')
having count(idMovie) > 1) dup on concat(movie.c00, ' (' , movie.c07, ')') = concat(dup.c00, ' (' , dup.c07, ')'
)
order by title_year
;

--------------------------

Also what causes a movie to showing in more than one instance in the search result?
I check the movie, and I only have one copy of it..
but it shows up as multiple copies
Reply
#11
(2014-08-18, 13:24)madmax2 Wrote: I already enable

XBMC client in Settings -> Services -> Remote control:

Allow programs on this system to control XBMC
Allow programs on other systems to control XBMC

Did you restart Kodi? Sometimes the JSON-RPC server doesn't restart just by toggling the options. Also, run the script on the Kodi client, unless you specify the remote Kodi client by adding @xbmc.host=<ipaddress> which then allows you to run it on a central PC and easily manage multiple Kodic clients.
Texture Cache Maintenance Utility: Preload your texture cache for optimal UI performance. Remotely manage media libraries. Purge unused artwork to free up space. Find missing media. Configurable QA check to highlight metadata issues. Aid in diagnosis of library and cache related problems.
Reply
#12
(2014-08-18, 01:44)madmax2 Wrote:
(2014-05-23, 01:43)Dave.Agent Wrote: I'm late to the party here, but this maybe useful for others googling the same problem...

I use a MySQL db and the following query shows all duplicate Movie title/year combos

Code:
select idMovie, concat(movie.c00, ' (' , movie.c07, ')') as title_year, movie.c22
from movie
inner join (
                select c00, c07 from movie
                group by concat(movie.c00, ' (' , movie.c07, ')')
                having count(idMovie) > 1) dup on concat(movie.c00, ' (' , movie.c07, ')') = concat(dup.c00, ' (' , dup.c07, ')'
            )
order by title_year
;

What is the SQL lite equivalent?

Here's what I used with the SQLite Database Browser...

Code:
SELECT c00,
COUNT(c09)
FROM movie
GROUP BY c09
HAVING ( COUNT(c09) > 1 )
Reply
#13
(2014-08-18, 05:08)Soujirou Seta Wrote: The best way so far is use Ember Media Manager, and filter "Duplicated Movies"

...yep...
Scan, sort, delete. Done in about 3 minutes.
Reply
#14
(2014-11-30, 05:38)McButton Wrote:
(2014-08-18, 05:08)Soujirou Seta Wrote: The best way so far is use Ember Media Manager, and filter "Duplicated Movies"

...yep...
Scan, sort, delete. Done in about 3 minutes.

I tried this, it did not find duplicates despite my definitely having some.

Is there any way to filter by duplicate title, rather than imdb number?

Edit:

Never mind, I found a way to export it from Ember Media Manager to an xml, which I then imported into Excel as an xml table and did a Conditional Formatting > Highlight Cell Rules > Duplicate Values on the ListTitle Column.

I then had to go through the list, verifying that they weren't just different remakes of the same movie (which I sometimes keep to compare), and deleted all the ones of lower quality or on differing hard drives.
Reply
#15
Why don't you try Duplicate Files Deleter? It will do a thorough search of your hard disk and find out the two or more duplicate files of the same file which may be stored at different locations.
Reply

Logout Mark Read Team Forum Stats Members Help
How to find duplicate movies?0