2013-04-07, 05:44
Hello,
I am writing a movie scraper for a Japanese website which sets a cookie if you are from a country not on its approved list and then doesn't allow you access based on the cookie. If using a web browser to access the site, it's a pretty trivial task to just block the cookies from that domain and use the site as normal.
When I am testing my scraper in either a 3rd party scraper editor (I am using ScraperEdit) or Ember Media Manager, it scrapes successfully because I don't think these programs are sophisticated enough to set cookies. However, when I use the scraper in XBMC directly, it seems to try to download this cookie and prevent things from scraping successfully. Is there some way to bypass downloading these cookies while scraping in XBMC, either in my scraper code, advancedsettings.xml, or something else? I already tried using a Japanese proxy, but I don't think that helped as I seem to be getting the same behavior.
Thank you.
I am writing a movie scraper for a Japanese website which sets a cookie if you are from a country not on its approved list and then doesn't allow you access based on the cookie. If using a web browser to access the site, it's a pretty trivial task to just block the cookies from that domain and use the site as normal.
When I am testing my scraper in either a 3rd party scraper editor (I am using ScraperEdit) or Ember Media Manager, it scrapes successfully because I don't think these programs are sophisticated enough to set cookies. However, when I use the scraper in XBMC directly, it seems to try to download this cookie and prevent things from scraping successfully. Is there some way to bypass downloading these cookies while scraping in XBMC, either in my scraper code, advancedsettings.xml, or something else? I already tried using a Japanese proxy, but I don't think that helped as I seem to be getting the same behavior.
Thank you.
