Sorting through years is broken? - macromaster - 2020-03-22
Hello!
I recently discovered Kodi and I'm trying to organize around it to create my media center.
I've got around 3000 movies, so efficient sorting is crucial to me.
I've noted weirdnesses in the "Years" and "Genres" categories. Obviously, as you must know, sevreal movies are often sorted in very questionnable genres and some of them are never sorted anywhere. But I'm mostly ok with that since I don't see myself browsing through genres all that often.
"Years" on the other hand is a feature I was excited about but it seems to be broken as well.
Obviously it's hard to double check everything when you have so many movies but one year I use as reference is 1999. The Matrix is missing and so is The Phantom Menace to name a few. I know it is not just a matter of putting the year in the file name since The Mummy hasn't got the date and it's popping in the right year.
Is the category broken? How does it work? How could I fix it?
Thank you
RE: Sorting through years is broken? - Klojum - 2020-03-22
(2020-03-22, 12:40)macromaster Wrote: Obviously it's hard to double check everything when you have so many movies but one year I use as reference is 1999. The Matrix is missing and so is The Phantom Menace to name a few.
When I select Movies / Years / 1999 , I get to see both those movies in the listing. So I'm not quite sure what's up with your setup.
Any chance you are using a different skin than the system default Estuary?
Perhaps enabling debugging, restarting Kodi, display the 1999 year listing again, could maybe show something in Kodi's debug log (wiki).
RE: Sorting through years is broken? - black_eagle - 2020-03-22
Perhaps it's a scraper issue. Might be worth getting up the info dialog for The Matrix and clicking the 'refresh' button to re-scrape the info and see if it picks up the year.
RE: Sorting through years is broken? - macromaster - 2020-03-22
(2020-03-22, 13:54)Klojum Wrote: (2020-03-22, 12:40)macromaster Wrote: Obviously it's hard to double check everything when you have so many movies but one year I use as reference is 1999. The Matrix is missing and so is The Phantom Menace to name a few.
When I select Movies / Years / 1999 , I get to see both those movies in the listing. So I'm not quite sure what's up with your setup.
Any chance you are using a different skin than the system default Estuary?
Perhaps enabling debugging, restarting Kodi, display the 1999 year listing again, could maybe show something in Kodi's debug log (wiki). All default, skins and such. I will try the debug thing. I often rescrap everything to get everything right in one go, correcting along the way my filenames (sequels not displaying properly if the year isn't included and such).
I did a refresh on Episode I and it finally entered 1999. Which is strange because it was in my titles list and had the correct info already. Could it be Kodi messes up when it's batch scrapping too many movies?
As for Matrix, my filename was "The Matrix 1999" and it was flat out unlisted. Double checked if the movie was indeed called "The Matrix" and not "Matrix" but it is indeed "The Matrix" from 1999. Refreshing it also got me the proper info and put it in the correct year.
Although it is possible I added The Matrix recently and I didn't exable refresh library on start. Honestly can't tell.
Thank you for your advice. I'll do a wide refresh and come back with some debug log. But I definitely think there's something fishy here and there.
RE: Sorting through years is broken? - black_eagle - 2020-03-22
(2020-03-22, 14:29)macromaster Wrote: Could it be Kodi messes up when it's batch scrapping too many movies?
Although that's possible I think it's more likely that whichever site you are scraping from simply becomes 'overwhelmed' by the amount of data being requested. I know for a fact that when scraping music for instance, that the musicbrainz website is limited to one request per second from an individual IP and if you try and go quicker, it just doesn't return any info for those queries. I'd guess that a similar thing is in play here and that if you scrape your stuff in smaller batches you'll get a better success rate. I could of course be totally wrong, but a debug log (wiki) should show up that issue quite clearly.
RE: Sorting through years is broken? - Klojum - 2020-03-22
Scraping is done per video, movie or tv episodes. I don't think the size of a media collection should matter.
RE: Sorting through years is broken? - macromaster - 2020-03-22
(2020-03-22, 15:09)black_eagle Wrote: (2020-03-22, 14:29)macromaster Wrote: Could it be Kodi messes up when it's batch scrapping too many movies?
Although that's possible I think it's more likely that whichever site you are scraping from simply becomes 'overwhelmed' by the amount of data being requested. I know for a fact that when scraping music for instance, that the musicbrainz website is limited to one request per second from an individual IP and if you try and go quicker, it just doesn't return any info for those queries. I'd guess that a similar thing is in play here and that if you scrape your stuff in smaller batches you'll get a better success rate. I could of course be totally wrong, but a debug log (wiki) should show up that issue quite clearly.
If that is the case. Could we slow down the rate on which Kodi scrapps? Or is there an Add-on to ask confirmation for each scrapped movie?
I just scrapped the whole library again. This time Matrix still didn't show up in 1999. Simply scrolling in my files and hovering on Matrix, the cover appeared blank but popped up a couple of seconds later. Came back in 1999 and it appeared. I'm puzzled.
Again, Matrix is just one example, but my library is full of diverse movies I don't know so well. The point is to be able to browse through the ones I don't know so much, by actors, or years or directors so I really want this to work well.
Here's the log for when it scrapped Matrix I think
Quote:2020-03-22 14:38:05.031 T:2784 DEBUG: VideoInfoScanner: Adding new item to movies:E:\The Matrix 1999.mp4
2020-03-22 14:38:05.161 T:5152 DEBUG: CCurlFile::GetMimeType - http://image.tmdb.org/t/p/original/sFsEesIS6VtPiQtV4VCj5c1DA49.jpg -> image/jpeg
2020-03-22 14:38:05.161 T:5152 DEBUG: CurlFile::Open(0x1f7051687e0) http://image.tmdb.org/t/p/original/sFsEesIS6VtPiQtV4VCj5c1DA49.jpg
2020-03-22 14:38:05.263 T:2784 DEBUG: CAddonSettings[metadata.themoviedb.org]: loading setting definitions
2020-03-22 14:38:05.263 T:2784 DEBUG: CAddonSettings[metadata.themoviedb.org]: trying to load setting definitions from old format...
2020-03-22 14:38:05.263 T:2784 DEBUG: CAddonSettings[metadata.themoviedb.org]: loading setting values
Then it skipped to The Matrix Revolutions (don't have Reloaded)
This is when I hovered the film I think
Quote:2020-03-22 14:46:43.967 T:12600 INFO: ffmpeg[3138]: Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'E:\The Matrix 1999.mp4':
2020-03-22 14:46:43.967 T:12600 INFO: ffmpeg[3138]: Metadata:
2020-03-22 14:46:43.967 T:12600 INFO: ffmpeg[3138]: major_brand : isom
2020-03-22 14:46:43.968 T:12600 INFO: ffmpeg[3138]: minor_version : 1
2020-03-22 14:46:43.968 T:12600 INFO: ffmpeg[3138]: compatible_brands: isom
2020-03-22 14:46:43.968 T:12600 INFO: ffmpeg[3138]: creation_time : 2013-01-30T19:36:47.000000Z
2020-03-22 14:46:43.968 T:12600 INFO: ffmpeg[3138]: Duration: 02:16:17.96, start: 0.000000, bitrate: 2334 kb/s
2020-03-22 14:46:43.968 T:12600 INFO: ffmpeg[3138]: Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 1920x796 [SAR 1:1 DAR 480:199], 1827 kb/s, 23.98 fps, 23.98 tbr, 24k tbn, 47.95 tbc (default)
2020-03-22 14:46:43.968 T:12600 INFO: ffmpeg[3138]: Metadata:
2020-03-22 14:46:43.968 T:12600 INFO: ffmpeg[3138]: creation_time : 2013-01-30T18:17:03.000000Z
2020-03-22 14:46:43.968 T:12600 INFO: ffmpeg[3138]: handler_name : GPAC ISO Video Handler
2020-03-22 14:46:43.968 T:12600 INFO: ffmpeg[3138]: Stream #0:1(eng): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, 5.1, fltp, 377 kb/s (default)
2020-03-22 14:46:43.968 T:12600 INFO: ffmpeg[3138]: Metadata:
2020-03-22 14:46:43.968 T:12600 INFO: ffmpeg[3138]: creation_time : 2013-01-30T13:58:43.000000Z
2020-03-22 14:46:43.968 T:12600 INFO: ffmpeg[3138]: handler_name : Sound Media Handler
2020-03-22 14:46:43.968 T:12600 INFO: ffmpeg[3138]: Stream #0:2(eng): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 124 kb/s (default)
2020-03-22 14:46:43.968 T:12600 INFO: ffmpeg[3138]: Metadata:
2020-03-22 14:46:43.968 T:12600 INFO: ffmpeg[3138]: creation_time : 2013-01-30T14:58:29.000000Z
2020-03-22 14:46:43.968 T:12600 INFO: ffmpeg[3138]: handler_name : Sound Media Handler
2020-03-22 14:46:43.968 T:12600 DEBUG: CDVDDemuxFFmpeg::AddStream ID: 0
2020-03-22 14:46:43.968 T:12600 DEBUG: CDVDDemuxFFmpeg::AddStream ID: 1
2020-03-22 14:46:43.968 T:12600 DEBUG: CDVDDemuxFFmpeg::AddStream ID: 2
2020-03-22 14:46:43.968 T:12600 NOTICE: Opening stream: 0 source: 256
2020-03-22 14:46:43.968 T:12600 NOTICE: Creating video codec with codec id: 27
2020-03-22 14:46:43.969 T:12600 NOTICE: CDVDVideoCodecFFmpeg::Open() Using codec: H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10
2020-03-22 14:46:43.969 T:12600 DEBUG: CDVDVideoCodecFFmpeg - Updated codec: ff-h264
2020-03-22 14:46:43.969 T:12600 DEBUG: CVideoPlayerVideo::OpenStream - open stream with codec id: 27
2020-03-22 14:46:43.969 T:12600 NOTICE: Creating video thread
2020-03-22 14:46:43.969 T:12600 DEBUG: CEdl::ReadEditDecisionLists - Checking for edit decision lists (EDL) on local drive or remote share for: E:\The Matrix 1999.mp4
2020-03-22 14:46:43.969 T:16884 DEBUG: Thread VideoPlayerVideo start, auto delete: false
2020-03-22 14:46:43.969 T:16884 NOTICE: running thread: video_thread
2020-03-22 14:46:43.969 T:12600 NOTICE: Opening stream: 1 source: 256
2020-03-22 14:46:43.969 T:16884 DEBUG: CVideoPlayerVideo - CDVDMsg::GENERAL_PAUSE: 0
2020-03-22 14:46:43.969 T:12600 NOTICE: Finding audio codec for: 86018
2020-03-22 14:46:43.969 T:12600 NOTICE: CDVDAudioCodecFFmpeg::Open() Successful opened audio decoder aac
2020-03-22 14:46:43.970 T:12600 NOTICE: Creating audio thread
2020-03-22 14:46:43.970 T:12600 NOTICE: Opening stream: 0 source: 1024
2020-03-22 14:46:43.970 T:16876 DEBUG: Thread VideoPlayerAudio start, auto delete: false
2020-03-22 14:46:43.970 T:16876 NOTICE: running thread: CVideoPlayerAudio:rocess()
2020-03-22 14:46:43.970 T:16876 DEBUG: CVideoPlayerAudio - CDVDMsg::GENERAL_PAUSE: 0
Then found it mentionned here
Quote:2020-03-22 14:46:45.475 T:1596 INFO: Deleting settings information for files E:\The Matrix 1999.mp4
2020-03-22 14:46:45.476 T:10844 DEBUG: CSaveFileState:oWork - Saving file state for video item E:\The Matrix 1999.mp4
2020-03-22 14:46:45.476 T:3236 DEBUG: Radio UECP (RDS) Processor - delete CDVDRadioRDSData::~CDVDRadioRDSData
2020-03-22 14:46:45.476 T:14228 DEBUG: CAESinkDirectSound::Initialize: secondary buffer created
2020-03-22 14:46:45.477 T:14228 DEBUG: CAESinkDirectSound::Initialize: Initializing DirectSound with the following parameters:
2020-03-22 14:46:45.477 T:14228 DEBUG: Audio Device : EP-HDMI-RX (NVIDIA High Definition Audio)
2020-03-22 14:46:45.477 T:14228 DEBUG: Sample Rate : 44100
2020-03-22 14:46:45.477 T:14228 DEBUG: Sample Format : AE_FMT_FLOAT
2020-03-22 14:46:45.477 T:14228 DEBUG: Bits Per Sample : 32
2020-03-22 14:46:45.477 T:14228 DEBUG: Valid Bits/Samp : 32
2020-03-22 14:46:45.477 T:14228 DEBUG: Channel Count : 2
2020-03-22 14:46:45.477 T:14228 DEBUG: Block Align : 8
2020-03-22 14:46:45.477 T:14228 DEBUG: Avg. Bytes Sec : 352800
2020-03-22 14:46:45.478 T:14228 DEBUG: Samples/Block : 32
2020-03-22 14:46:45.478 T:14228 DEBUG: Format cBSize : 22
2020-03-22 14:46:45.478 T:14228 DEBUG: Channel Layout : FL, FR
2020-03-22 14:46:45.478 T:14228 DEBUG: Channel Mask : 3
2020-03-22 14:46:45.478 T:14228 DEBUG: Frames : 661
2020-03-22 14:46:45.478 T:14228 DEBUG: Frame Size : 8
2020-03-22 14:46:45.478 T:14228 DEBUG: CActiveAESink::OpenSink - DIRECTSOUND Initialized:
2020-03-22 14:46:45.478 T:14228 DEBUG: Output Device : default
2020-03-22 14:46:45.478 T:14228 DEBUG: Sample Rate : 44100
2020-03-22 14:46:45.478 T:14228 DEBUG: Sample Format : AE_FMT_FLOAT
2020-03-22 14:46:45.478 T:14228 DEBUG: Channel Count : 2
2020-03-22 14:46:45.478 T:14228 DEBUG: Channel Layout: FL, FR
2020-03-22 14:46:45.478 T:14228 DEBUG: Frames : 661
2020-03-22 14:46:45.478 T:14228 DEBUG: Frame Size : 8
2020-03-22 14:46:45.478 T:19276 DEBUG: CActiveAE::ClearDiscardedBuffers - buffer pool deleted
2020-03-22 14:46:45.489 T:1596 DEBUG: Previous line repeats 1 times.
2020-03-22 14:46:45.489 T:1596 DEBUG: CApplication::OnPlayBackStopped: CApplication::OnPlayBackStopped
2020-03-22 14:46:45.498 T:3236 DEBUG: CGUIWindowManager:reviousWindow: Deactivate
2020-03-22 14:46:45.498 T:3236 DEBUG: ------ Window Deinit (VideoOSD.xml) ------
2020-03-22 14:46:45.498 T:3236 DEBUG: ------ Window Deinit (VideoFullScreen.xml) ------
2020-03-22 14:46:45.500 T:3236 DEBUG: CGUIWindowManager:reviousWindow: Activate new
2020-03-22 14:46:45.500 T:14228 DEBUG: CAESinkDirectSound::CheckPlayStatus: Resuming Playback
2020-03-22 14:46:45.502 T:3236 DEBUG: ------ Window Init (MyVideoNav.xml) ------
2020-03-22 14:46:45.503 T:3236 DEBUG: CGUIMediaWindow::GetDirectory (E:\)
2020-03-22 14:46:45.503 T:3236 DEBUG: ParentPath = [E:\]
2020-03-22 14:46:45.574 T:3236 DEBUG: CAddonSettings[metadata.themoviedb.org]: loading setting definitions
2020-03-22 14:46:45.574 T:3236 DEBUG: CAddonSettings[metadata.themoviedb.org]: trying to load setting definitions from old format...
2020-03-22 14:46:45.574 T:3236 DEBUG: CAddonSettings[metadata.themoviedb.org]: loading setting values
2020-03-22 14:46:45.575 T:3236 DEBUG: CAddonSettings[metadata.themoviedb.org]: loading setting definitions
2020-03-22 14:46:45.575 T:3236 DEBUG: CAddonSettings[metadata.themoviedb.org]: trying to load setting definitions from old format...
2020-03-22 14:46:45.576 T:3236 DEBUG: CAddonSettings[metadata.themoviedb.org]: loading setting values
2020-03-22 14:46:45.750 T:500 DEBUG: Thread BackgroundLoader start, auto delete: false
RE: Sorting through years is broken? - black_eagle - 2020-03-22
(2020-03-22, 15:13)Klojum Wrote: Scraping is done per video, movie or tv episodes. I don't think the size of a media collection should matter.
It's not the size, but the rate you are hitting the info site which is also getting hit by multiple other people at the same time. Lets say you scrape 1000 videos and get 900 returned successfully. You've got 100 videos with no or partial info. If you scrape 100 videos and get the same failure rate then you only have 10 to redo. So it makes sense to do smaller blocks no?
RE: Sorting through years is broken? - macromaster - 2020-03-22
or maybe it's been scrapped here and skipped to Police Academy for some reason... And came back... What the...
* log removed by moderator *
Link: https://paste.kodi.tv/akupuxemip.kodi
RE: Sorting through years is broken? - Klojum - 2020-03-22
(2020-03-22, 16:20)macromaster Wrote: or maybe it's been scrapped here
Ehh... Which part of "Do not post logs in the forum. Use https: //paste.kodi.tv/ instead." is unclear to you? We prefer to keep our forum server lean.
RE: Sorting through years is broken? - macromaster - 2020-03-22
(2020-03-22, 17:28)Klojum Wrote: (2020-03-22, 16:20)macromaster Wrote: or maybe it's been scrapped here
Ehh... Which part of "Do not post logs in the forum. Use https: //paste.kodi.tv/ instead." is unclear to you? We prefer to keep our forum server lean. My bad, sorry.
|