xbmc.org website recovery

  Thread Rating:
  • 0 Votes - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Post Reply
aptalca Offline
The Dude
Posts: 916
Joined: Sep 2009
Reputation: 25
Post: #1
Is there any chance of recovering last month's messages through an index such as google cache?

I know it's a stretch, but maybe? the forum was being indexed by google several times a day, and everything was text based. . .
find quote
mr.b Offline
Senior Member
Posts: 165
Joined: May 2007
Reputation: 0
Post: #2
For those that are looking to recover data in the lost threads, most of it should still be cached on google/bing/yahoo if you act fast. There's even a few tools out there that are made for recovering data via this method (see warrick). Is anyone from the xbmc team already working this angle? Perhaps people could just highlight specific threads they are after and we can work together to get them from cache before they disappear. Just trying to be helpful....
find quote
bradvido88 Offline
Donor
Posts: 796
Joined: Nov 2008
Reputation: 9
Location: MN
Post: #3
I've posted and subscribed to quite a few threads over the last month that i'd like back.
Is team-XBMC looking at this, or just letting the data disappear?

These are just a few to start out with. I've got a whole list of now-broken links in my browser history.
http://forum.xbmc.org/showthread.php?p=526213
http://forum.xbmc.org/showthread.php?p=522197
http://forum.xbmc.org/showthread.php?p=526213

I'm guessing you could probably find the ID# where it cut off and recover anything greater than that from the cache?

XBMC.MyLibrary (add anything to the library)
ForTheLibrary (Argus TV & XBMC Library PVR Integration)
SageTV & XBMC PVR Integration
(This post was last modified: 2010-03-17 21:38 by bradvido88.)
find quote
theuni Offline
Team-XBMC Communication Manager
Posts: 1,105
Joined: Oct 2007
Reputation: 2
Location: Atlanta, Ga, USA
Post: #4
You can't just cram text into the db. There may be extreme ways of salvaging posts from google cache, but nothing I'm willing to do. Sorry guys, 2 sleepless nights is enough for me.

The info will be back in no time anyway...

TheUni
find quote
sebak Offline
Junior Member
Posts: 33
Joined: Mar 2010
Reputation: 0
Location: Belgium
Post: #5
It's not that easy as just scraping off googles cache.
First you should make a script that does just that, but then there are still a lot of problems.

You can't just put the data in the database right away. If you post a reply or thread, a lot of things change (post count, cache, ...) and to maintain integrity of the system, you need to use the API.
However, for example posts posted by users that are now also gone (users registererd within the last month) will give problems, because guests are not allowed to post. And there might be a lot of these kind of problems.

Edit: theuni was faster
(This post was last modified: 2010-03-17 23:12 by sebak.)
find quote
exobuzz Online
Fan
Posts: 595
Joined: Aug 2004
Reputation: 12
Post: #6
I would say not much can realistically be done now about lost data, but I am willing to help if xbmc want to set up offsite database replication. It is a good way of not only handling backups (backing up from slave database rather than primary, and having no "delay" whilst non innodb tables are locked, and also having a copy which is always at most a few seconds/minutes behind the main database.
find quote
Jezz_X Offline
Team-XBMC Skinner
Posts: 5,289
Joined: Jun 2006
Reputation: 57
Location: Earth
Post: #7
I just think everyone should be grateful we didn't go with option 1 which was start the forum from scratch and decided to go with option 2 (use the backup before the site move)
find quote
bradvido88 Offline
Donor
Posts: 796
Joined: Nov 2008
Reputation: 9
Location: MN
Post: #8
theuni Wrote:...Sorry guys, 2 sleepless nights is enough for me.

The info will be back in no time anyway...
Thank you for your time and help! I was just curious if XMBC had any plans. Like you said, it is not a clean and easy thing to do.

Jezz_X Wrote:I just think everyone should be grateful we didn't go with option 1 which was start the forum from scratch and decided to go with option 2 (use the backup before the site move)
Very grateful! Starting from scratch would have lost us a lot of knowledge, information and time.

XBMC.MyLibrary (add anything to the library)
ForTheLibrary (Argus TV & XBMC Library PVR Integration)
SageTV & XBMC PVR Integration
find quote
mr.b Offline
Senior Member
Posts: 165
Joined: May 2007
Reputation: 0
Post: #9
I appreciate sleepless nights as well (I've been there). Also, i wasn't advocating reimporting into the database, rather, an html capture and attachment to a new thread.
find quote
hippojay Offline
Fan
Posts: 300
Joined: Mar 2008
Reputation: 14
Location: Sheffield, UK
Post: #10
For the few odd posts (such as guides which has post 1 updated) then google cache will probably help track down the post contents, such as this linux mavemkv one:

Luckily, you can use teh cache: keywork in a google search to bring back the cached page only:

cache:http://forum.xbmc.org/showthread.php?tid=67420

At least this allows some threads to get the first post back on track - but not for wholesale db restoration...
find quote
Gaudi Offline
Member
Posts: 57
Joined: Dec 2009
Reputation: 0
Post: #11
I don't think that is that useful. If you find some important post of you is missing, you can grab the cache info and repost.
That will do and will not put into trouble the administrators.
find quote
gibsonlp Offline
Junior Member
Posts: 37
Joined: Apr 2009
Reputation: 0
Post: #12
Hi,
I administrate a similarly sized forum (less visitors but over half a million posts) and I run full backup every day and keep a few days worth of binary log (hourly rotated to avoid corruption). Very simple to setup and very effective, the daily snapshot takes a couple of minutes and nobody notices anything (the forum continues to work as is without any issues, the database guarantees data integrity for the exact moment where the dump has started)
With these backups and almost a week's worth of binary logs I can rollback to any point in time in the past week and can easily recover after a major crash.
Additionally - daily dumps are compressed and copied to an off site storage server so even if my server burns and turns to dust, in my worst case scenario I have lost a day's worth of data (I don't replicate the binary logs).

If you need help setting up a similar mechanism - just drop me a note.
find quote
osirisjem Offline
Senior Member
Posts: 175
Joined: Oct 2008
Reputation: 1
Post: #13
gibsonlp Wrote:Hi,
I administrate a similarly sized forum (less visitors but over half a million posts) and I run full backup every day and keep a few days worth of binary log (hourly rotated to avoid corruption). Very simple to setup and very effective, the daily snapshot takes a couple of minutes and nobody notices anything (the forum continues to work as is without any issues, the database guarantees data integrity for the exact moment where the dump has started)
With these backups and almost a week's worth of binary logs I can rollback to any point in time in the past week and can easily recover after a major crash.
Additionally - daily dumps are compressed and copied to an off site storage server so even if my server burns and turns to dust, in my worst case scenario I have lost a day's worth of data (I don't replicate the binary logs).

If you need help setting up a similar mechanism - just drop me a note.

Great Idea !
I assumed xbmc.org did a backup a few times per day, at least !
It's hard to imagine how much data was lost in 1 month.

ASrock Ion 330HT - XBMCFreak 10.1 Lucid LiveCD. Everything works but System sounds over HDMI.
find quote
zag Offline
Team-Kodi Member
Posts: 1,735
Joined: Oct 2007
Reputation: 20
Location: UK
Post: #14
I would be very interested in how to backup mysql regularly. I have read about scripting mysqldump but not really implemented it outside a test site.

I administer a popular forum and backup once every 2 weeks manually at the moment using phpmyadmin.

I've also seen phpmybackup, but not used it myself yet.
(This post was last modified: 2010-03-29 12:29 by zag.)
find quote
dfelix Offline
Junior Member
Posts: 4
Joined: Apr 2010
Reputation: 0
Post: #15
Now I understand why i could not log in... and why needed to register again!
Sad
find quote
Post Reply