[Release] XBMC Backup

  Thread Rating:
  • 5 Votes - 5 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Post Reply
robweber Offline
Posting Freak
Posts: 901
Joined: Sep 2009
Reputation: 27
Post: #736
I'll need a debug log of the restore you're trying to do. Can't really do much without one.
find quote
D.Z.G Offline
Junior Member
Posts: 2
Joined: Oct 2014
Reputation: 0
Post: #737
How do I create one?
find quote
robweber Offline
Posting Freak
Posts: 901
Joined: Sep 2009
Reputation: 27
Post: #738
(2014-10-11 17:34)D.Z.G Wrote:  How do I create one?

http://wiki.xbmc.org/?title=Log_file - several methods to choose from



Google Drive Functionality

For anyone interested in testing this I've attempted to add functionality for Google Drive. Until this is all in the official repo you'll have to jump through a few hoops but might be worth it. The more testing done the faster this can be rolled out.

Step 1 - You'll have to manually install the googleapi module. There were a lot of dependencies and stuff for this so I rolled them in to their own addon: https://github.com/robweber/script.module.googleapi . Just download the zip and install into XBMC.

Step 2 - Download the "google_drive" branch of the addon. You might want to uninstall any current version of your system as the addon ids are the same. https://github.com/robweber/xbmcbackup/t...ogle_drive

There are instructions in the README file but I'll cut/paste them here. Very similiar to Dropbox in that you have to have a Google API account setup and then oauth your account via a URL.

Something I've noticed in testing is that the Drive API will sometimes block requests as there is a limit for "free" accounts on how many per second. It didn't happen often but it did happen. The API is also kind of slow, probably another limitation of the non-paid account. Using the "compress backups" option might be the best choice since you are minimizing the number of files that actually need to get sent to the cloud.

Code:
Using the Google Drive target is very similar to the Dropbox one. You must create a Google API project and authenticate your account via the id and secret. Instructions for creating the Google API for Google Drive can be found here. You'll need the client id and client secret generated for the addon settings (https://developers.google.com/drive/web/quickstart/quickstart-python). You only need to follow Step 1.

Once you have the client ID and Secret add them to the addon settings and run a backup. You'll get a notification that you need to enter your authorization code. Check your XBMC log file for a line from "script.xbmcbackup" containing the authorization URL. Cut/paste this into a browser and click Allow. Once this is done put the code from your browser into the pop-up dialog. The addon will cache these credentials so it should be a one-time authenication. From there the backup should start to run.
find quote
schumi2004 Offline
Posting Freak
Posts: 1,425
Joined: Aug 2011
Reputation: 14
Post: #739
For some reason this addon only backups first level (from root) folders and files, second level folders/files won't be uploaded to Dropbox
When you open a folder it's just empty
find quote
robweber Offline
Posting Freak
Posts: 901
Joined: Sep 2009
Reputation: 27
Post: #740
(2014-10-16 19:00)schumi2004 Wrote:  For some reason this addon only backups first level (from root) folders and files, second level folders/files won't be uploaded to Dropbox
When you open a folder it's just empty

DEBUG LOG!

I'm not trying to pick on you here but getting error reports is very frustrating with no information to do with it. What exactly do you expect me to respond with? "Yes - you're right I coded this to not do anything on purpose" or "I magically figured out it is because you didn't setup your SMB server right, or your internet connection went out" How am I supposed to know anything without a log of what actually happened.

I'm willing to help anyone with a problem but please go through the proper steps, otherwise this is just a thread full of complaints.
find quote
schumi2004 Offline
Posting Freak
Posts: 1,425
Joined: Aug 2011
Reputation: 14
Post: #741
Debug log was on my todo list, maybe I should have mentioned that in post Wink

/edit:
Nothing usefull in Debug log. No errors or such.
http://pastebin.com/4xTdCGWv

System running OpenELEC Generic x86_64 Helix beta 5
(This post was last modified: 2014-10-17 08:32 by schumi2004.)
find quote
robweber Offline
Posting Freak
Posts: 901
Joined: Sep 2009
Reputation: 27
Post: #742
Thanks for the log.

Glad you had a sense of humor about the message. Just seems there are a lot of "its broken" with no follow-up messages lately. I guess yours was the stray that broke the camel's back.

Looking at what you posted I guess I'm a little confused. Is this the full run of the addon? I don't see the start where it normally lists the full path and then goes through it's file add process. It seems to be a snippet from somewhere in the middle. I do see your custom directory. Is that the one that isn't working or is the whole thing only putting up the first level folders?

One thing to note about the Dropbox stuff is a slight caveat to their upload process. Here is the code from the vfs.py file:

Code:
def put(self,source,dest,retry=True):
        dest = self._fix_slashes(dest)
        
        if(self.client != None):
            f = open(source,'rb')
            try:
                response = self.client.put_file(dest,f,True)
                return True
            except:
                #if we have an exception retry
                if(retry):
                    return self.put(source,dest,False)
                else:
                    #tried once already, just quit
                    return False
        else:
            return False

You'll see that it tries to put the file, and if that fails it tries again. If that fails we just give up and return "False" for the result. I think it's an error on the Dropbox side because you can continue to put the file indefinitely and it will never make it in some cases. Perhaps these "false" returns should be logged. I'll make a note to add that to the debug logging.
find quote
schumi2004 Offline
Posting Freak
Posts: 1,425
Joined: Aug 2011
Reputation: 14
Post: #743
It's not uploading for all directory's
I'll do another test with and without dropbox and see what happens. It could be a snippet from half way the process, there was so much garbage from other service addon that i only selected the information I thought was relevant.
find quote
schumi2004 Offline
Posting Freak
Posts: 1,425
Joined: Aug 2011
Reputation: 14
Post: #744
Same behavior on Windows7 test machine.

Local storage: http://xbmclogs.com/show.php?id=321093
Strange thing is that it returns a error unable to create D:/ although it writes on that disc (first level only)

Remote storage (DropBox): http://xbmclogs.com/show.php?id=321095
(This post was last modified: 2014-10-17 16:11 by schumi2004.)
find quote
robweber Offline
Posting Freak
Posts: 901
Joined: Sep 2009
Reputation: 27
Post: #745
That log file error about "error creating d:/" is really weird. It shouldn't be trying to create the root directory.

If you could, please download the latest source and try that. Just uninstall the addon version you have now and re-install from this zip file: https://github.com/robweber/xbmcbackup/a...master.zip
find quote
schumi2004 Offline
Posting Freak
Posts: 1,425
Joined: Aug 2011
Reputation: 14
Post: #746
(2014-10-17 22:04)robweber Wrote:  That log file error about "error creating d:/" is really weird. It shouldn't be trying to create the root directory.

If you could, please download the latest source and try that. Just uninstall the addon version you have now and re-install from this zip file: https://github.com/robweber/xbmcbackup/a...master.zip

It seems to do even less. (issue still there)
Note: All options enabled except for custom folder!
When i enabled zip archive http://pastebin.com/drjAPzTb
Local storage http://xbmclogs.com/show.php?id=321494
DropBox gives me the same result

TreeSize report from source and backup destination
(This post was last modified: 2014-10-18 00:11 by schumi2004.)
find quote
robweber Offline
Posting Freak
Posts: 901
Joined: Sep 2009
Reputation: 27
Post: #747
Thanks for pointing out the zip archive issue - I'll look in to that.

Is your "D" drive seriously mapped as a lowercase "d"? Seems like a weird thing to point out I'm sure but although Windows doesn't care about case - it seems XBMC does. Change the remote directory to "d:/private/" and see what happens. I'm thinking that what it is trying to do is when it calls the "mkdir" command to create the directory we need for the addons it's having some issues resolving the path. It might even be trying to recursively create the directories which is where the "error creating D:\" text comes from.
find quote
schumi2004 Offline
Posting Freak
Posts: 1,425
Joined: Aug 2011
Reputation: 14
Post: #748
Doesn't make a difference.
The uppercase D comes from browsing to the path, with type path the error persist.

http://pastebin.com/u7BGquAS
find quote
robweber Offline
Posting Freak
Posts: 901
Joined: Sep 2009
Reputation: 27
Post: #749
Well there is one idea down.

It looks like there is no problem creating directories - just files. The XFILE errors are from the underlying xbmc filesystem trying to write the file. Are you absolutely sure XBMC has permission for this directory? Have you tried others just to make sure (like just on a folder on the desktop or something)?
find quote
Post Reply