+- XBMC Community Forum (http://forum.xbmc.org)
+-- Forum: Help and Support (/forumdisplay.php?fid=33)
+--- Forum: Add-ons Help and Support (/forumdisplay.php?fid=27)
+---- Forum: Weather Add-ons (/forumdisplay.php?fid=155)
+---- Thread: [RELEASE] OzWeather - Australian Weather Addon using BOM data (/showthread.php?tid=116905)
- scotia - 2012-02-27 05:48
Hmm I looked under "downloads", not "source"
My bad.
Thanks for the add-on!
Scott
- bossanova808 - 2012-02-27 05:49
No probs.
script modification to handle www.weatherzone.com.au - scotia - 2012-02-27 06:56
Hi Bossa,
I was having the same trouble as the chap before with a script error when searching a postcode.
Bearing in mind in know very little about python...
The returned url from weather zone has a www. at the front which is missing from the script.
Also, unfortunately there are TWO HTML lists with class typ2, which breaks the script. It only checks one of the two lists, which is the one containing news. Because the list you want is in the "structure_middle" div I included a step before extracting the list to limit the html to just that div.
The third thing is that the length of skimmed is 1, even though the search returns many results. So I changed it to the length of the locations array.
To that end, I've created a patch to fix things. It has some log() calls which really shouldn't be there, feel free to remove them. I'm sure there are ten better ways to accomplish the same thing (like checking for (www\.)? at the start of the URL.
I hope you don't mind me fiddling with your code.
Regards
Scott
Code:
--- default.py.dist 2012-02-27 15:49:06.000000000 +1100
+++ default.py 2012-02-27 15:44:40.000000000 +1100
@@ -521,7 +521,7 @@
text = keyboard.getText()
#need to submit the postcode to the weatherzone search
- searchURL = 'http://weatherzone.com.au/search/'
+ searchURL = 'http://www.weatherzone.com.au/search/'
user_agent = 'Mozilla/4.0 (compatible; MSIE 5.5; Windows NT)'
host = 'www.weatherzone.com.au'
headers = { 'User-Agent' : user_agent, 'Host' : host }
@@ -530,9 +530,11 @@
req = urllib2.Request(searchURL, data, headers)
response = urllib2.urlopen(req)
resultPage = str(response.read())
+# log("OzWeather: got response", resultPage);
#was there only one match? If so it returns the page for that match so we need to check the URL
responseurl = response.geturl()
- if responseurl != 'http://weatherzone.com.au/search/':
+# log("OzWeather: got responseurl", responseurl);
+ if responseurl != 'http://www.weatherzone.com.au/search/':
#we were redirected to an actual result page
locationName = common.parseDOM(resultPage, "h1", attrs = { "class": "unenclosed" })
locationName = str.split(locationName[0], ' Weather')
@@ -540,17 +542,24 @@
locationids = [responseurl]
else:
#we got back a page to choose a more specific location
- skimmed = common.parseDOM(resultPage, "ul", attrs = { "class": "typ2" })
+ middle = common.parseDOM(resultPage, "div", attrs = { "id": "structure_middle" })
+# log("OzWeather: got middle", middle);
+ skimmed = common.parseDOM(middle, "ul", attrs = { "class": "typ2" })
+# skimmed = common.parseDOM(resultPage, "ul", attrs = { "class": "typ2" })
+# log("OzWeather: got skimmed", skimmed);
#ok now get two lists - one of the friendly names
#and a matchin one of the URLs to store
locations = common.parseDOM(skimmed[0], "a")
+# log("OzWeather: got locations", locations);
+# log("OzWeather: got locations length", len(locations));
templocs = common.parseDOM(skimmed[0], "a", ret="href")
+# log("OzWeather: got templocs", templocs);
#build the full urls
locationids = []
for count, loc in enumerate(templocs):
locationids.append(WeatherZoneURL + loc)
#if we did not get enough data back there are no locations with this postcode
- if len(skimmed)<=1:
+ if len(locations)<=1:
locations = []
locationids = []- bossanova808 - 2012-02-27 07:18
Not at all, patches welcome! But what postcode was giving you errors - I check about 7 by default and I haven't hit the issue.
Um....also, I am stupid with git - not immediately sure how to apply a patch
can you just post the changed function as you have it now? I'll also try and learn...
- scotia - 2012-02-27 07:19
Hi Bossa,
and I have a feature request.
The UV value is the forecast from the BOM. Is there any chance of displaying the current. I couldn't find any source on the BOM site, but there is a real time capital city reading in XML here:
http://www.arpansa.gov.au/uvindex/realtime/xml/uvvalues.xml
Is there any chance you could read that data in and display it alongside the forecast?
Cheers,
Scott
- bossanova808 - 2012-02-27 07:25
Hi Scott - the problem with that is tying it to the user's location - it's no good to me if it is just capital cities, I really need a Weatherzone source for it and I couldn't find one. Otherwise how do I deal with Yackandanda's UV, for example?
Basically, I need a BOM ID of some sort, or a weahterzone one, or it gets pretty hard....
Also, please supply postcodes with issues as I want to look at that a bit more and can't find any...
New code - scotia - 2012-02-27 07:29
Bossa,
I was searching 3122. 3000 was also failing however. The search page in my browser had (http://www.) at the front, which was not matching your script.
Here is the new function. Changed lines are commented out with '#' with the new one below. I also added a line.
I'd rather the responseurl check for both a url WITH www. and with out it. I'm too lazy to google python regex!
Regards
Scott
Code:
##############################################
### NOW ACTUALLTY RUN THIS PUPPY - this is main() in the old language...
socket.setdefaulttimeout(100)
#the being called from the settings section where the user enters their postcodes
if sys.argv[1].startswith('Location'):
keyboard = xbmc.Keyboard('', 'Enter your 4 digit postcode e.g. 3000', False)
keyboard.doModal()
if (keyboard.isConfirmed() and keyboard.getText() != ''):
text = keyboard.getText()
#need to submit the postcode to the weatherzone search
# searchURL = 'http://weatherzone.com.au/search/'
searchURL = 'http://www.weatherzone.com.au/search/'
user_agent = 'Mozilla/4.0 (compatible; MSIE 5.5; Windows NT)'
host = 'www.weatherzone.com.au'
headers = { 'User-Agent' : user_agent, 'Host' : host }
values = {'q' : text, 't' : '3' }
data = urllib.urlencode(values)
req = urllib2.Request(searchURL, data, headers)
response = urllib2.urlopen(req)
resultPage = str(response.read())
#was there only one match? If so it returns the page for that match so we need to check the URL
responseurl = response.geturl()
# if responseurl != 'http://weatherzone.com.au/search/':
if responseurl != 'http://www.weatherzone.com.au/search/':
#we were redirected to an actual result page
locationName = common.parseDOM(resultPage, "h1", attrs = { "class": "unenclosed" })
locationName = str.split(locationName[0], ' Weather')
locations = [locationName[0] + ', ' + text]
locationids = [responseurl]
else:
#we got back a page to choose a more specific location
# next line added
middle = common.parseDOM(resultPage, "div", attrs = { "id": "structure_middle" })
# skimmed = common.parseDOM(resultPage, "ul", attrs = { "class": "typ2" })
skimmed = common.parseDOM(middle, "ul", attrs = { "class": "typ2" })
#ok now get two lists - one of the friendly names
#and a matchin one of the URLs to store
locations = common.parseDOM(skimmed[0], "a")
templocs = common.parseDOM(skimmed[0], "a", ret="href")
#build the full urls
locationids = []
for count, loc in enumerate(templocs):
locationids.append(WeatherZoneURL + loc)
#if we did not get enough data back there are no locations with this postcode
# if len(skimmed)<=1:
if len(locations)<=1:
locations = []
locationids = []
### no changes past this point- bossanova808 - 2012-02-27 07:36
What was in your browser isn't really relevant to be honest, it's what the result of the Python urlopen and it was coming back without the www hence the code.
But more importantly, both those codes work here on my Windows box, and 3000 at least definitely works on my linux box, so I can't see why they are failing for you...
What's your platform details?
- bossanova808 - 2012-02-27 07:43
(that said I am happy to make the changes as they don't appear to break anything, it's just odd and hard for me to test as they all work fine here anyway!)
- scotia - 2012-02-27 07:49
Ok, when I was debugging the script I got:
Code:
NOTICE: Oz Weather-0.3.5: Exception: OzWeather: got responseurl [http://www.weatherzone.com.au/search/]But my browser returns the www. as well.
What the site returns is really up to it. I am using a squid proxy may alter the request.
Also of some use:
Code:
[root]# curl -D /tmp/curl-headers "http://weatherzone.com.au/search"
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>301 Moved Permanently</title>
</head><body>
<h1>Moved Permanently</h1>
<p>The document has moved <a href="http://www.weatherzone.com.au/search">here</a>.</p>
</body></html>