2021-02-15, 03:32
Was wondering if it's normal for TMM to have a high cpu usage. I've seen it as high as 90% and 25-30% when sitting idle. Using Windows 10. Thanks.
2021-02-15, 07:55
completely depends on the circumstances. If your free memory is running low, Java tries to free more memory which results in endless garbage collector runs. I've also seen the DB to compacting many times to shrink its size (where I think there is an issue in the MVStore, but I could not reproduce it to open an issue over there).
So in general: it is not normal that tmm eats up this amount of CPU in idle, BUT it can happen (most of the time the next restart of tmm it did not happen any more)
So in general: it is not normal that tmm eats up this amount of CPU in idle, BUT it can happen (most of the time the next restart of tmm it did not happen any more)
2021-02-15, 08:24
I just found an old issue at H2 (https://github.com/h2database/h2database/issues/471) where this behaviour is described. Looks like there are other projects having this issue to -> I've just added our information there
2021-02-15, 13:28
In my case tmm rarely uses up more than 10% of CPU when idle, but I can see the db size constantly changing while tmm is running.
Speaking of the db size, I noticed the movie db size has recently grown significantly - more than 50% larger compared to 4.0.7. I tried re-creating the library but the db size was almost the same.
Can you tell me what has changed?
Speaking of the db size, I noticed the movie db size has recently grown significantly - more than 50% larger compared to 4.0.7. I tried re-creating the library but the db size was almost the same.
Can you tell me what has changed?
2021-02-15, 14:04
nothing changed in this regard.. In my observations today the db size of my default testbed was between 1.4 MB and 3.2 MB with exact the same data.. This has to be something MVStore internal
In the next build I turned of the MVStore internal compacting, but trigger a manual compact every time the DB has been changed (10 sec interval). This resulted in the same db size almost all the time (2.8 MB)
In the next build I turned of the MVStore internal compacting, but trigger a manual compact every time the DB has been changed (10 sec interval). This resulted in the same db size almost all the time (2.8 MB)
2021-02-15, 14:50
What about movie set module?
The new feature "display missing movies for movie sets" comes to mind since it added lots of new (missing) movies and they must reside in the movie db.
But there still has to be more. Do the newly added columns affect the db size for the module?
The new feature "display missing movies for movie sets" comes to mind since it added lots of new (missing) movies and they must reside in the movie db.
But there still has to be more. Do the newly added columns affect the db size for the module?
2021-02-15, 15:13
ah ok - missing movies for movie sets could be the difference, but also this should not be a huge difference
2021-02-15, 16:26
So, in regards to resource usage, it's a java problem and not TMM? Basically something one has to live with.
2021-02-15, 17:13
not Java directly - the database engine we use internally. I've included a mitigation of this problem in v4.1, but I needed to find a compromise between database size and CPU usage (which can be high when constantly shrinking the database)
2021-02-15, 17:25
I just tested the new build. Now it uses almost zero resources when idle and the db size remains the same if there's no change.
But it looks like a different level of compacting is going on when you make a change and when you exit.
My initial db size was 38 MB and if I make changes it fluctuates between 50 MB and 55 MB. Then when I exit, it becomes 34 MB.
It keeps showing this sort of pattern with the final db size fluctuating, which has been always the case even before this build.
Then should the manual compacting while running be really necessary as it auto-compacts on exit anyway?
But it looks like a different level of compacting is going on when you make a change and when you exit.
My initial db size was 38 MB and if I make changes it fluctuates between 50 MB and 55 MB. Then when I exit, it becomes 34 MB.
It keeps showing this sort of pattern with the final db size fluctuating, which has been always the case even before this build.
Then should the manual compacting while running be really necessary as it auto-compacts on exit anyway?
2021-02-15, 18:29
If it matters, from my point of view, I could care less about the db size. Disc thrashing and memory usage is more important to me. Just my $.02.
Just loaded the 2-15 nightly. TMM sitting idle. CPU usage ~70%. That normal?
Disregard. I loaded the program again and it seemed ok. sorry.
Just loaded the 2-15 nightly. TMM sitting idle. CPU usage ~70%. That normal?
Disregard. I loaded the program again and it seemed ok. sorry.
2021-02-15, 21:34
@tars I've had a look at the code of MVStore and there is some extra code when closing the database - there is a more "intensive" way to compact the database which results in rewriting the whole file. I did not add this in the scheduled task to avoid unnecessary disk I/O. If I would not implement the auto compacting, every time a dataset is written to the database (even an update to an existing dataset) will create a new "chunk" of the database which results in a constant growth until tmm is closed. Unfortunately this is the way the MVStore works - so a way of "constantly" compacting is needed to avoid massive growth of the database.
I think the actual solution is a good compromise between CPU usage and disk usage
I think the actual solution is a good compromise between CPU usage and disk usage
2021-02-16, 09:53
Thanks for the detailed explanation! Now I can see why it has to work that way.
So it's an improvement in that it's no longer constantly compacting without any changes and even when it's compacting it uses a "lighter" way using less resources.
So it's an improvement in that it's no longer constantly compacting without any changes and even when it's compacting it uses a "lighter" way using less resources.
2021-02-17, 06:07
Probably related to this change. Now running every command keeps logging the following error though it seems working fine:
And upon running "Download missing artwork for movie set" command, it logs the following error for every movie set, but again it seems to do the job properly:
And occasionally this error pops up:
https://pastebin.com/SuuJdupN
Code:
ERROR [tmmpool-rename-T1] o.tinymediamanager.core.movie.MovieList:468 - failed to persist movie: [some movie title] - Reading from file nio:data/movies.db failed at 19636578 (length -1), read 0, remaining 768 [1.4.200/1]And upon running "Download missing artwork for movie set" command, it logs the following error for every movie set, but again it seems to do the job properly:
Code:
ERROR [tmmpool-download-task-T1-G846] o.tinymediamanager.core.movie.MovieList:488 - failed to persist movie set: [some movie set name]And occasionally this error pops up:
https://pastebin.com/SuuJdupN
