Improved allmusic.com scraper (plus a few questions)

  Thread Rating:
  • 0 Votes - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Post Reply
ronie Offline
Team-XBMC Member
Posts: 9,097
Joined: Jan 2009
Reputation: 157
Post: #76
sho Wrote:Is there a ticket for this issue?

there's one now:
http://trac.xbmc.org/ticket/7510

vdrfan Wrote:svn revision?

i'm using svn 24150 on linux,
kiboy6 is on 24107-GL if i'm not mistaken.

Always read the XBMC online-manual, FAQ and search the forum before posting.
Do not PM or e-mail Team-XBMC members directly asking for support. Read/follow the forum rules.
For troubleshooting and bug reporting please make sure you read this first.
find quote
ronie Offline
Team-XBMC Member
Posts: 9,097
Joined: Jan 2009
Reputation: 157
Post: #77
the fix in svn 24170 works fine for me.
thanx!

Always read the XBMC online-manual, FAQ and search the forum before posting.
Do not PM or e-mail Team-XBMC members directly asking for support. Read/follow the forum rules.
For troubleshooting and bug reporting please make sure you read this first.
find quote
kiboy6 Offline
Fan
Posts: 626
Joined: Mar 2009
Reputation: 0
Post: #78
Me too....thanks!

Just to confirm, we still need to overwrite the music / scrapers folder with the one talisto posted a few pages back to see the improvements, right?
find quote
spiff Offline
Retired Developer
Posts: 12,386
Joined: Nov 2003
Post: #79
no.
find quote
talisto Offline
Junior Member
Posts: 40
Joined: Jun 2005
Reputation: 0
Post: #80
kiboy6 Wrote:Just to confirm, we still need to overwrite the music / scrapers folder with the one talisto posted a few pages back to see the improvements, right?

Nope, my changes were added to the SVN as of r24107. Hooray! Thanks spiff! Smile
find quote
kiboy6 Offline
Fan
Posts: 626
Joined: Mar 2009
Reputation: 0
Post: #81
Awesome...thanks spiff, and thanks again to talisto Smile


edit:
....one bizarre thing...for some reason I'm getting no biography for "tenor saw", depite there being a description at allmusic.com. All other artists seem fine.....any idea?
(This post was last modified: 2009-11-03 06:34 by kiboy6.)
find quote
steve1977 Online
Posting Freak
Posts: 1,808
Joined: Dec 2006
Reputation: 0
Post: #82
Using Camelot Beta-1 and the preference of allmusic scraper doesnt include discogs anymore. Is this intentional?

Server: Asus Sabertooth Z77 | Intel Core i5 3.4 GHz | 12 GB DDR3 | 256 GB SSD, 40 TB (1 x 4 TB, 6 x 3TB, 9 x 2 TB)
HTPC 1: Apple Mac Mini (mid-2011) | Intel Core i5 2.5 GHz | 4GB DDR3 | 500 GB
HTPC 2: Apple Mac Mini (mid-2007) | Intel Core 2 Duo 2.0 GHz | 4 GB DDR2 | 16 GB SSD
find quote
steve1977 Online
Posting Freak
Posts: 1,808
Joined: Dec 2006
Reputation: 0
Post: #83
steve1977 Wrote:Using Camelot Beta-1 and the preference of allmusic scraper doesnt include discogs anymore. Is this intentional?

Any thoughts?

Server: Asus Sabertooth Z77 | Intel Core i5 3.4 GHz | 12 GB DDR3 | 256 GB SSD, 40 TB (1 x 4 TB, 6 x 3TB, 9 x 2 TB)
HTPC 1: Apple Mac Mini (mid-2011) | Intel Core i5 2.5 GHz | 4GB DDR3 | 500 GB
HTPC 2: Apple Mac Mini (mid-2007) | Intel Core 2 Duo 2.0 GHz | 4 GB DDR2 | 16 GB SSD
find quote
spiff Offline
Retired Developer
Posts: 12,386
Joined: Nov 2003
Post: #84
discogs blocked us - so we removed it.
find quote
infinite2 Offline
Junior Member
Posts: 2
Joined: Dec 2009
Reputation: 0
Post: #85
Dear all,

After I found out that Discogs was blocking XBMC I took the liberty to ask the guys @ Discogs about this issue.

Here is the complete answer from Teo @ Discogs help Center

"Yes, we're blocking XBMC. It was basically acting like a DDOS and impacting site performance. Contact me directly and tell me more about it. Maybe we can find a way to allow it."

I think that XBMC developers should get in touch with discogs and find a solution ti this problem. Discogs is probably the best music database so its a shame to remove it from XBMC.

I hope that something good can come out of this.

Looking forward for your comments

Kind Regards

G
find quote
theuni Offline
Team-XBMC Communication Manager
Posts: 1,105
Joined: Oct 2007
Reputation: 2
Location: Atlanta, Ga, USA
Post: #86
Thanks for the heads up.. Teo at Discogs got in touch with me via email yesterday, I'm trying to work out some solution.

TheUni
find quote
infinite2 Offline
Junior Member
Posts: 2
Joined: Dec 2009
Reputation: 0
Post: #87
theuni Wrote:Thanks for the heads up.. Teo at Discogs got in touch with me via email yesterday, I'm trying to work out some solution.

TheUni

Well that's great news mate!

I really hope both sides to find a way to make this work.

Keep us posted!

Thnx

G
(This post was last modified: 2009-12-02 18:51 by infinite2.)
find quote
Belgrath Offline
Member
Posts: 50
Joined: May 2010
Reputation: 0
Post: #88
Is there a way to force a re-scrape of the entire music library again?

I found that I needed to nuke the entire music database in the user profile directory, and then I needed to delete all of the thumbs and artist fan art from the user profile directory as well.

Also for testing purposes I exported my whole collection, nuked the db file, but didn't clear out the various fanart and thumbnail folders 0,1,2,3,4,5 etc. I noticed that after the import some of the pics didn't exactly match what my collection folders were showing, meaning that somehow the library 'remembers' even though the db file is gone.

When I tried the exact same test AND removed the thumbnail and fanart files from the userdata directories then the newly created db was insync.

Did I stumb onto something here or is this already known?

I guess really that this is more of an import issue, however, if I do a full scrape and then I do another full scrape it seems as if the query blows by artisst that it previously determined no items were available. The only way that I found to do a full scrape was to nuke the db, nuke the artist thumbs and fanart then reimport everything. Is there any way to force a re-scrape in this scenario

Thanks ahead of time,
Belgrath
(This post was last modified: 2010-07-13 00:13 by Belgrath.)
find quote
Belgrath Offline
Member
Posts: 50
Joined: May 2010
Reputation: 0
Post: #89
Belgrath Wrote:Is there a way to force a re-scrape of the entire music library again?

I found that I needed to nuke the entire music database in the user profile directory, and then I needed to delete all of the thumbs and artist fan art from the user profile directory as well.

Also for testing purposes I exported my whole collection, nuked the db file, but didn't clear out the various fanart and thumbnail folders 0,1,2,3,4,5 etc. I noticed that after the import some of the pics didn't exactly match what my collection folders were showing, meaning that somehow the library 'remembers' even though the db file is gone.

When I tried the exact same test AND removed the thumbnail and fanart files from the userdata directories then the newly created db was insync.

Did I stumb onto something here or is this already known?

I guess really that this is more of an import issue, however, if I do a full scrape and then I do another full scrape it seems as if the query blows by artisst that it previously determined no items were available. The only way that I found to do a full scrape was to nuke the db, nuke the artist thumbs and fanart then reimport everything. Is there any way to force a re-scrape in this scenario

Thanks ahead of time,
Belgrath

Thoughts?
find quote
johnny utah Offline
Member
Posts: 73
Joined: Jul 2008
Reputation: 0
Post: #90
Hi, sorry to barge in, but I think my issue may be related to the scraper.

I'm currently trying to scrape all artist info/discogs using allmusic scraper.

When I run it it only seems to scrape the artist I happen to be focusted on when I run the scraper instead of all the artists.

When I select another artist I have to select the scraper again to get the data but again only works for one at a time not the whole library.

I'm using a nightly build from 3 days ago and have updated all my scrapers. Using reFocus skin for XBMC OSX.

Can't figure out what's wrong =(
Any suggestions?
find quote
Post Reply