Error when using 'urllib2': GetSocketForIndex()
#1
Hi,

I keep getting this error when firing alot of html/request/open commands within a short space of time..It seems to stuff the connection between the xbox and the reset of world..

All other html posts work without any problem but my script 'Torrent-X' when using the Azureus module runs into a problem in one area..It is when attempting to change priorites on all files for the item selected. For some reason azureus does not support a bulk feature so a seperate html/request/open command has to be sent for each individual file which in some cases could be 30+ files. In these cases the connection seems to get stuffed with the errors below being reported..

Any help would be appreciated..

This is my caller to core.Download which is the urllib2 request/open thread.

Code:
def UrlThread(self,url,post='',login=''):
        try :
            print '--UrlThread'
            self.WEBDATA    =   ''
            Caller          =   core.Download_URL(self,url,post,login)
            Caller.start()
            Caller.join(self.TIMEOUT)
            print '--WEBDATA--: '+ str(len(self.WEBDATA))
            if Caller.isAlive() :
                Caller.terminate()
                raise
            else                :
                return self.WEBDATA
        except :
            traceback.print_exc()
            self.win.FAILURE = True
            return None

Has anybody had similar errors reported like this before..

Code:
16:54:54 M: 15138816    INFO: --CheckUrl
16:54:54 M: 15138816    INFO:
16:54:54 M: 15138816   ERROR: GetSocketForIndex() invalid index:0
16:54:54 M: 15138816   ERROR: GetSocketForIndex() invalid index:0
16:54:54 M: 15138816   ERROR: ReleaseSocket() invalid index:0
16:54:54 M: 15138816    INFO: Traceback (most recent call last):
16:54:54 M: 15138816    INFO:   File "Q:\scripts\Torrent-X\libs\core.py", line 40, in run
16:54:54 M: 15138816    INFO: Stating file Q:\scripts\Torrent-X\libs\core.py
16:54:54 M: 15138816    INFO:     html                = urllib2.urlopen(reqst)
16:54:54 M: 15138816    INFO:   File "Q:\system\python\python24.zlib\urllib2.py", line 130, in urlopen
16:54:54 M: 15138816    INFO: Stating file Q:\system\python\python24.zlib\urllib2.py
16:54:54 M: 15138816    INFO: Stating file q:\scripts\torrent-x\urllib2.py
16:54:54 M: 15138816    INFO: Stating file Q:\system\python\python24.zlib\urllib2.py
16:54:54 M: 15138816    INFO: Stating file Q:\system\python\DLLs\urllib2.py
16:54:54 M: 15138816    INFO: Stating file Q:\system\python\Lib\urllib2.py
16:54:54 M: 15138816    INFO: Stating file Q:\system\python\spyce\urllib2.py
16:54:54 M: 15138816    INFO: Stating file q:\scripts\torrent-x\libs\urllib2.py
16:54:54 M: 15138816    INFO: Stating file q:\scripts\torrent-x\controllers\urllib2.py
Server: FreeNas 9 NAS: 6*3TB - Kodi Sql Database
Kodi Systems: Nvidia Shield Pro, G-Box Q (OpenElec), RikoMagic MK802 IV[
Skin: reFocus
Reply
#2
does this happen to be done in parallel? ie multiple threads at the same time?
Always read the XBMC online-manual, FAQ and search the forum before posting.
Do not e-mail XBMC-Team members directly asking for support. Read/follow the forum rules.
For troubleshooting and bug reporting please make sure you read this first.


Image
Reply
#3
Hi,

No these are all single threads that wait with a join...but there can be atleast 3-4 calls within a second..
Server: FreeNas 9 NAS: 6*3TB - Kodi Sql Database
Kodi Systems: Nvidia Shield Pro, G-Box Q (OpenElec), RikoMagic MK802 IV[
Skin: reFocus
Reply
#4
Ok heres what I have found out...

I am actually getting two errors being reported...
## URLError: <urlopen error (10038, 'Socket operation on non-socket')>
## URLError: <urlopen error (10055, 'No buffer space available')>

Now the Socket error comes first then the No buffer space comes afterwards. The No buffer space according to what I found on the net is basically saying out of resources. There is no memory issue as far as XBMC is concerned as there is still 18-20MB free according to loglevel2 reports.

I have found a hacky way around by trapping for the error and if found delete the urllib and urllib2 modules and then reload them and recall my url module. This does work successfully with no real noticeable delay.

If someone knows of a way to flush or cleanup whatever the socket/urlopen is using I would love to hear it..

What a pain!!
Server: FreeNas 9 NAS: 6*3TB - Kodi Sql Database
Kodi Systems: Nvidia Shield Pro, G-Box Q (OpenElec), RikoMagic MK802 IV[
Skin: reFocus
Reply
#5
well first of all, our socket code emulation in dllloader is totally non thread safe, so what i meant is. are there at any time two threads running that are using sockets at the same time?

secondly, we currently only allow 50 open sockets at one time, so if you exeed that, it will start to fail on socket creation.
Always read the XBMC online-manual, FAQ and search the forum before posting.
Do not e-mail XBMC-Team members directly asking for support. Read/follow the forum rules.
For troubleshooting and bug reporting please make sure you read this first.


Image
Reply
#6
I put some code to list the threads running in the url-thread and it only ever reports the thread currently being executed so the threads are finishing ok.

my Url code perform and Request/urlopen read data then close so they aren't doing any from the normal. The fix does work reliably but maybe I'll have to look for a way to view open sockets and flush them as and when required..
Server: FreeNas 9 NAS: 6*3TB - Kodi Sql Database
Kodi Systems: Nvidia Shield Pro, G-Box Q (OpenElec), RikoMagic MK802 IV[
Skin: reFocus
Reply
#7
Only thing I can think of is that it's running out of sockets. ie not cleaning up them completly. (it could be an error in the url lib). this then get's cleaned up when dll is closed.

I'm cleaning up this code as we speak, and i'll also increase the maximum number of sockets allowed. can't see a reason for the limit we have.
Always read the XBMC online-manual, FAQ and search the forum before posting.
Do not e-mail XBMC-Team members directly asking for support. Read/follow the forum rules.
For troubleshooting and bug reporting please make sure you read this first.


Image
Reply
#8
Thanks for looking into this..

The deleting of the the module using "del sys.modules['urllib2']" does the trick 'del urllib2' doesnt actually unload the module and the two errors reported are the only ones that seem to highlight this problem.

searching on the net I have found noticed posts relating to problems like this which are apparently caused by a failure in socket.close and they've recommended using socket.shutdown to clear this..I dont really know much about sockets/packets and all that anyway....

Cheers for that anyway..Im not sure that if the sockets are increased it may caused more problems down the line with a huge number of open sockets lying around...There nothing worth than a load of sockets and pants all over the floor...:-)
Server: FreeNas 9 NAS: 6*3TB - Kodi Sql Database
Kodi Systems: Nvidia Shield Pro, G-Box Q (OpenElec), RikoMagic MK802 IV[
Skin: reFocus
Reply
#9
could be.. i have no idea how python works in that regard. anyway, i've rewritten the socket code in dllloader to be much simpler, and almost thread safe Smile. it also removes the limitation on 50 sockets. give it a go, when you can.
Always read the XBMC online-manual, FAQ and search the forum before posting.
Do not e-mail XBMC-Team members directly asking for support. Read/follow the forum rules.
For troubleshooting and bug reporting please make sure you read this first.


Image
Reply
#10
Cheers I'll check it out on the next T3ch release...and report back..
Server: FreeNas 9 NAS: 6*3TB - Kodi Sql Database
Kodi Systems: Nvidia Shield Pro, G-Box Q (OpenElec), RikoMagic MK802 IV[
Skin: reFocus
Reply

Logout Mark Read Team Forum Stats Members Help
Error when using 'urllib2': GetSocketForIndex()0