Kodi Community Forum - Reliably obtain free disk space on shares

Pages: 1 2

Hi,

For my addon XBMC File Cleaner I need to test whether the free disk space is below a certain threshold. If so, the addon will clean up watched videos on that location. This works great with local paths as well as UNC paths. However, when using smb or nfs shares from within a Linux-based XBMC installation, my current code will throw an OSError. I have done some research and it appears that it is not possible to stat an absolute smb path such as 'smb://path/to/videos' using os.statvfs(), both while it is mounted and while it is not. Apparently this has to do with the way Linux handles mounted drives. This has been an issue with my addon for a while now, and before releasing a Gotham-ready version I would really like to test the disk space reliably, but I could not find any such method in the xbmc addon documentation. Could anyone give me some tips how I am supposed to do this?

Thanks in advance,
Anthirian

(2014-04-21, 21:21)Anthirian Wrote: [ -> ]I have done some research and it appears that it is not possible to stat an absolute smb path such as 'smb://path/to/videos' using os.statvfs(), both while it is mounted and while it is not. Apparently this has to do with the way Linux handles mounted drives.

You can, when mounted. You obviously can't stat random urls though.
Check if pysmb support it.

Anything useful in this?
http://mirrors.xbmc.org/docs/python-docs...mcvfs.html

(2014-04-22, 21:59)takoi Wrote: [ -> ]
(2014-04-21, 21:21)Anthirian Wrote: [ -> ]I have done some research and it appears that it is not possible to stat an absolute smb path such as 'smb://path/to/videos' using os.statvfs(), both while it is mounted and while it is not. Apparently this has to do with the way Linux handles mounted drives.
You can, when mounted. You obviously can't stat random urls though.
Check if pysmb support it.

Unfortunately this is not that obvious to me, being a bit of a beginner with Linux. Maybe you could help me understand?

Below is a snippet of the logs produced by the current code, while the 'E' share is mounted via SMB both in Linux itself and in XBMC by adding a network location.

Code:
44:10 T:140102252599040  NOTICE: XBMC File Cleaner: XBMC File Cleaner version 3.2.1 loaded.

44:10 T:140102252599040  NOTICE: XBMC File Cleaner: Starting cleaning routine.

44:10 T:140102252599040  NOTICE: XBMC File Cleaner: Checking for disk space on path: 'smb://GEERT/E/'

44:10 T:140102252599040  NOTICE: XBMC File Cleaner: We are checking disk space from a non-Windows file system

44:10 T:140102252599040   ERROR: XBMC File Cleaner: Could not check disk space. Access denied.

44:10 T:140102252599040  NOTICE: XBMC File Cleaner: Error accessing 'smb://GEERT/E/': OSError(2, 'No such file or directory')

44:10 T:140102252599040  NOTICE: XBMC File Cleaner: Free space: 100.00%

44:10 T:140102252599040  NOTICE: XBMC File Cleaner: No cleaning required

44:11 T:140103384143744  NOTICE: Samba is idle. Closing the remaining connections

I found out here and here that in order to access the share over in the terminal I have to use Linux' internal mount path (see below for the one Linux Mint uses) instead of the path XBMC uses for the file's location.

Code:
/run/user/1000/gvfs/smb-share:server=geert,share=e

When I use this internal mount path, I get the following log.

Code:
17:28 T:140102974043904  NOTICE: XBMC File Cleaner: XBMC File Cleaner version 3.2.1 loaded.

17:28 T:140102974043904  NOTICE: XBMC File Cleaner: Starting cleaning routine.

17:28 T:140102974043904  NOTICE: XBMC File Cleaner: Checking for disk space on path: '/run/user/1000/gvfs/smb-share:server=geert,share=e'

17:28 T:140102974043904  NOTICE: XBMC File Cleaner: We are checking disk space from a non-Windows file system

17:28 T:140102974043904  NOTICE: XBMC File Cleaner: Hard disk check results:

17:28 T:140102974043904  NOTICE: XBMC File Cleaner: Bytes free: 37,350,150

17:28 T:140102974043904  NOTICE: XBMC File Cleaner: Bytes total: 244,189,695

17:28 T:140102974043904  NOTICE: XBMC File Cleaner: Free space: 15.30%

17:28 T:140102974043904  NOTICE: XBMC File Cleaner: No cleaning required

19:03 T:140103384143744  NOTICE: Samba is idle. Closing the remaining connections

As you can see, disk space checks work correctly. The only thing that is a bit strange is the fact that my 1 TB drive is recognized to be only 244 GB in size. The percentage of free space however is in fact correct.

Code:
29:45 T:5232  NOTICE: XBMC File Cleaner: Starting cleaning routine.

29:45 T:5232  NOTICE: XBMC File Cleaner: Checking for disk space on path: 'E:\\'

29:45 T:5232  NOTICE: XBMC File Cleaner: We are checking disk space from a Windows file system

29:45 T:5232  NOTICE: XBMC File Cleaner: The path to check is 'E:\\'

29:45 T:5232  NOTICE: XBMC File Cleaner: We are dealing with local paths

29:45 T:5232  NOTICE: XBMC File Cleaner: Converting path to unicode for disk space checks

29:45 T:5232  NOTICE: XBMC File Cleaner: New path: u'E:\\'

29:45 T:5232  NOTICE: XBMC File Cleaner: Hard disk check results:

29:45 T:5232  NOTICE: XBMC File Cleaner: Bytes free: 152,961,269,760

29:45 T:5232  NOTICE: XBMC File Cleaner: Bytes total: 1000,200,990,720

29:45 T:5232  NOTICE: XBMC File Cleaner: Free space: 15.29%

When I run the checks from my local Windows machine containing the media files, checks run just fine. The free disk space percentage matches the one determined via SMB.

Thanks for the tip about pysmb though, I will look into that. Does this support NFS too? These are the two main networking protocols my addon needs to support.

(2014-04-22, 22:02)Martijn Wrote: [ -> ]Anything useful in this?
http://mirrors.xbmc.org/docs/python-docs...mcvfs.html

I'm afraid not. I assumed the Stat class would help me with this, but it lacks a method to check the amount of space currently used/free. It only has a method for determining the size of a folder, which is only half the solution.

Anyone?

Did you not get your answer?

There's no difference between windows and linux here. If the path is a file system path (starts with /) and the file system supports it, statvfs should work. Regardless of the underlying file system being samba or not. (I don't know why you get incorrect result though. Have always worked fine for me.) "Paths" like smb://foo/bar are not file system paths and won't work on either linux nor windows.

You probably need to either implement your own samba/nfs clients in python to check the disk space, or implement statvfs or similar method in the xbmcvfs module. The latter might even be an easy copy paste job if xbmc already implements this method in its internal vfs.

(2014-04-23, 20:37)Anthirian Wrote: [ -> ]I'm afraid not. I assumed the Stat class would help me with this, but it lacks a method to check the amount of space currently used/free. It only has a method for determining the size of a folder, which is only half the solution.

I don't think it means what you think it does though. st_size is just the size of the actual folder (4096 bytes), not the content

(2014-04-28, 17:05)takoi Wrote: [ -> ]Did you not get your answer?

There's no difference between windows and linux here. If the path is a file system path (starts with /) and the file system supports it, statvfs should work. Regardless of the underlying file system being samba or not. (I don't know why you get incorrect result though. Have always worked fine for me.) "Paths" like smb://foo/bar are not file system paths and won't work on either linux nor windows.

I didn't understand the answer well enough. For example, I didn't know you can never perform statvrs on non-system paths. Searches I did elsewhere did not explain this. Thanks again for your reply though.

(2014-04-28, 17:05)takoi Wrote: [ -> ]You probably need to either implement your own samba/nfs clients in python to check the disk space, or implement statvfs or similar method in the xbmcvfs module. The latter might even be an easy copy paste job if xbmc already implements this method in its internal vfs.

For the second option, what you're saying is that I should build my own statvfs method for use with XBMC's internal file system? How would I do that? I assume it would require writing C(++) but I know little of that.
As for the internal file system, is there not an option to universally determine the mount point of a share in the XBMC internal file system once it's mounted, seeing as the path appears to vary per Linux distro? Also, where do I find documentation on the internal file system and its (public) methods, if it's available?

(2014-04-28, 17:05)takoi Wrote: [ -> ]
(2014-04-23, 20:37)Anthirian Wrote: [ -> ]I'm afraid not. I assumed the Stat class would help me with this, but it lacks a method to check the amount of space currently used/free. It only has a method for determining the size of a folder, which is only half the solution.
I don't think it means what you think it does though. st_size is just the size of the actual folder (4096 bytes), not the content

It appears I don't. The Stat class will not be useful at all then.

I apologize for all these questions, I have little knowledge of stat and the details.

Code:
def get_free_space():

   if os.path.exists("/storage"): #rPi/openelec

     path = "/storage"

   elif os.path.exists("/root"): #pivos

     path = "/root"

   else:

     path = "/" # all other *nix

   st = os.statvfs(path)

   free = st.f_bavail * st.f_frsize

   percent = (math.ceil(float(100) / float(st.f_frsize * st.f_blocks) * free))

   total = st.f_blocks * st.f_frsize

   return total,free, percent

Thanks amet, but how would I locate the mount point of the shares to use with your code? I understand no Linux version uses the same internal mount point?

there is no way that I know of to retrieve all sources you have setup... you could ask users to set path in addon setting

That's what I'm currently doing. I've created a setting with which users can select the path to the share they want to check for disk space, but when I retrieve that path it will give me "smb://share" instead of the system path. This is why I started the thread, because I don't know how to go from a smb:// path to a system path.

I'll try to explain it better: When users browse samba shares (etc.) though xbmc, there is no mount path or system path to get, because it's not mounted anywhere. Xbmc implements a client which acts as a virtual file system. That's what I meant by xbmc's internal vfs, and that's how the methods in the xbmcvfs python module supports smb:// type urls. Most of these functions are just pass-through to other functions already implemented in c++. So what you should do, if you want to go this route, is look at the source code if there is anything like statvfs already implemented and add a python interface to it.

Alternatively implement it from scratch in pysmb in the add-on or with c++ directly in xbmc.

@amet That's not what he's asking about. Also, you can get all sources. GetSources in json-rpc api

ok, so get all sources set and check available space. above snip needs to be modified to use xbmcvfs.Stat() - > http://mirrors.xbmc.org/docs/python-docs...mcvfs.html

EDIT: seems like Stat needs to be extended a bit to allow for that, my bad

stat and statvfs is not the same thing

(2014-05-01, 22:15)takoi Wrote: [ -> ]stat and statvfs is not the same thing

sure but xbmcvfs could be extended for it

Pages: 1 2