v18 [bug] cURL unable to connect to any url starting with https
#1
Kodi version is 18.0-BETA3 Git:20180923-1a67bd7, OS is Ubuntu 16.04.5 LTS, kernel: Linux x86 32-bit version 4.4.0-135-generic

lsb_release -a
No LSB modules are available.
Distributor ID:    Ubuntu
Description:    Ubuntu 16.04.5 LTS
Release:    16.04
Codename:    xenial

Curl is apparently unable to connect to any secure sites, just giving this instead.
Code:
INFO: easy_acquire - Created session to https://assets.planetradio.co.uk
11:25:22.805 T:2813324096   DEBUG: Curl::Debug - TEXT:   Trying 52.222.253.57...
11:25:22.854 T:2813324096   DEBUG: Curl::Debug - TEXT: Connected to assets.planetradio.co.uk (52.222.253.57) port 443 (#0)
11:25:22.855 T:2813324096   DEBUG: Curl::Debug - TEXT: ALPN, offering http/1.1
11:25:22.855 T:2813324096   DEBUG: Curl::Debug - TEXT: Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
11:25:22.884 T:2813324096   DEBUG: Curl::Debug - TEXT: successfully set certificate verify locations:
11:25:22.884 T:2813324096   DEBUG: Curl::Debug - TEXT:   CAfile: /etc/ssl/certs/ca-certificates.crt
11:25:22.884 T:2813324096   DEBUG: Curl::Debug - TEXT:   CApath: /etc/ssl/certs
11:25:22.885 T:2813324096   DEBUG: Curl::Debug - TEXT: Unknown SSL protocol error in connection to assets.planetradio.co.uk:443
11:25:22.885 T:2813324096   DEBUG: Curl::Debug - TEXT: Closing connection 0
11:25:22.889 T:2813324096   ERROR: CCurlFile::Stat - Failed: SSL connect error(35) for https://assets.planetradio.co.uk/img/ConfigWebListenBarLogoImageUrl/49.png?ver=1463741190
11:25:22.889 T:2813324096   DEBUG: GetImageHash - unable to stat url https://assets.planetradio.co.uk/img/ConfigWebListenBarLogoImageUrl/49.png?ver=1463741190

Using curl in a terminal on the same machine works.
Code:
curl  https://assets.planetradio.co.uk/img/ConfigWebListenBarLogoImageUrl/49.png?ver=1463741190 > logo.png
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  4327  100  4327    0     0   8504      0 --:--:-- --:--:-- --:--:--  8517

I also have a 64 bit version built from source which does not exhibit this, so I'm wondering if it's peculiar to the 32 bit version ??

Debug log : http://paste.ubuntu.com/p/XBQqMPTnxZ/   Apologies for the amount of chatter in there from curl grabbing art from http addresses !!

As far as I know, the certificate store on the machine is correct.  I'm guessing it must be because curl in a terminal can connect fine and it's using the same ones.

I'd attempt a build from source, but I reckon it would take that machine around three hours vs 30 seconds to install the nightly.  I doubt many people are using a 32bit version these days, so wondered if something has sneaked in 'under the radar'.
Learning Linux the hard way !!
Reply
#2
(2018-09-23, 12:59)black_eagle Wrote: I also have a 64 bit version built from source which does not exhibit this, so I'm wondering if it's peculiar to the 32 bit version ??
I assume you tried that on a different machine? Is it also running 16.04?
Where are you getting the nightlies from? Are you using the official PPA?
Reply
#3
Yeah, the other machine is running 4.15.0-33-generic #36~16.04.1-Ubuntu SMP Wed Aug 15 17:21:05 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

lsb_release -a
LSB Version:    core-9.20160110ubuntu0.2-amd64:core-9.20160110ubuntu0.2-noarch:printing-9.20160110ubuntu0.2-amd64:printing-9.20160110ubuntu0.2-noarchConfusedecurity-9.20160110ubuntu0.2-amd64Confusedecurity-9.20160110ubuntu0.2-noarch
Distributor ID:    Ubuntu
Description:    Ubuntu 16.04.5 LTS
Release:    16.04
Codename:    xenial

Dunno if it might make a difference but the 32bit machine is an Intel Processor and the 64bit is an AMD.  The nightly is from the official PPA.
Code:
apt-cache policy kodi kodi-x11
kodi:
  Installed: 2:18.0+git20180923.0202-1a67bd7-0xenial
  Candidate: 2:18.0+git20180924.0201-3238e00-0xenial
  Version table:
     2:18.0+git20180924.0201-3238e00-0xenial 500
        500 http://ppa.launchpad.net/team-xbmc/xbmc-nightly/ubuntu xenial/main i386 Packages
 *** 2:18.0+git20180923.0202-1a67bd7-0xenial 100
        100 /var/lib/dpkg/status
     2:18.0+git20171104.0201-a5436a4-0xenial 500
        500 http://ppa.launchpad.net/team-xbmc/xbmc-nightly/ubuntu xenial/main i386 Packages
     15.2+dfsg1-3ubuntu1.1 500
        500 http://gb.archive.ubuntu.com/ubuntu xenial-updates/universe i386 Packages
        500 http://security.ubuntu.com/ubuntu xenial-security/universe i386 Packages
     15.2+dfsg1-3ubuntu1 500
        500 http://gb.archive.ubuntu.com/ubuntu xenial/universe i386 Packages
kodi-x11:
  Installed: 2:18.0+git20180923.0202-1a67bd7-0xenial
  Candidate: 2:18.0+git20180924.0201-3238e00-0xenial
  Version table:
     2:18.0+git20180924.0201-3238e00-0xenial 500
        500 http://ppa.launchpad.net/team-xbmc/xbmc-nightly/ubuntu xenial/main i386 Packages
 *** 2:18.0+git20180923.0202-1a67bd7-0xenial 100
        100 /var/lib/dpkg/status 

Because cURL can't connect to https addresses means I can't connect to the official repository to install any addons, the weather service doesn't work and most of the streaming radio that I listen to fails as well.  I worked around the addon issue by downloading the ones I wanted and installing via zip, without even realising that there was an issue.  It was only when trying to open streaming radio that it became apparent what was (not) happening.

*** EDIT ***

Tried downgrading to beta2 from the unstable repo, with the same results. Don't _really_ want to downgrade to Krypton as that will break the upstart script and in any case, I really want all my machines running the same version. Having to download addons outside of Kodi is a pain when there are dependencies involved.
Learning Linux the hard way !!
Reply
#4
This looks like a really strange error... For clarification: On your 64-bit machine, did you try only your self-compiled version are also the official nightly for comparison?
Reply
#5
Also can you please post the output of the following commands:

ldd $(which curl)
ldd /usr/lib/i386-linux-gnu/kodi/kodi-x11
cat /proc/$(pidof kodi-x11)/maps
cat /proc/$(pidof kodi-x11)/environ
Reply
#6
(2018-09-25, 12:53)yol Wrote: This looks like a really strange error... For clarification: On your 64-bit machine, did you try only your self-compiled version are also the official nightly for comparison?
 The 64-bit machine has only had the self-compiled version on it.
Code:
Kodi compiled Sep 13 2018 by GCC 5.4.0 for Linux x86 64-bit version 4.4.140 (263308)

By comparison, the beta2 build currently on the 32-bit machine is
Code:
Starting Kodi (18.0-BETA2 Git:20180910-7868a27). Platform: Linux x86 32-bit
Using Release Kodi x32 build
Kodi compiled Nov  4 2012 by GCC 5.4.0 for Linux x86 32-bit version 4.4.140 (263308)
Running on Ubuntu 16.04.5 LTS, kernel: Linux x86 32-bit version 4.4.0-135-generic

So that is actually a version 3 days earlier than the self-compiled version.  I can't really mess with the 64-bit version as it is running in my living room and Kodi is the only method of supplying any A/V media - can't really tell my wife to go watch her soaps in the kitchen !!

I will clone the source on the 32-bit and build it to see if that makes a difference.

EDIT - The output of those is probably a little large for here.  I'll grab it into a file and pastebin it.
Learning Linux the hard way !!
Reply
#7
One more request, trying to get to the bottom of this... Please test in a fresh Kodi installation (just temporarily rename the.kodi folder in your home to something else and rename it back after testing). Maybe it's connected to some setting.
Reply
#8
(2018-09-25, 13:40)yol Wrote: One more request, trying to get to the bottom of this... Please test in a fresh Kodi installation (just temporarily rename the.kodi folder in your home to something else and rename it back after testing). Maybe it's connected to some setting.
 Bingo !!!  Renaming the directory fixed the issues immediately. (Probably this is something I should have thought of doing myself Blush )  I tested by installing a couple of addons that I knew had dependencies and a weather addon and everything worked fine.  I have re-added my advancedsettings.xml and the path subs and SQL databases are working correctly.

I am curious though as to what setting could possibly have caused it.  The only network related settings that I have altered from the defaults are to reduce smb to v1 and allow legacy security.  Quite clearly there was 'something' interfering with it but at the moment I am at a loss as to know what.  With the actual problem resolved, I now want to track down the cause of it.
Learning Linux the hard way !!
Reply
#9
OK, I spoke too soon because the error soon reared it's head again.  I started over again and took the following steps.

1) Install Yahoo! weather - check for forecast data - restart Kodi
2) Install Rad.io - check for forecast data - restart
3) Copy advancedsettings.xml to userdata directory - restart
4) Check for forecast data
5) Install Youtube - check for forecast data - restart
6) Forecast data missing on restart !!

Looked in the addons directory and found 'script.module.certifi' which is a youtube dependency.  Removing the addon in Kodi leaves the dependencies in place, so I deleted them all.  Restarted and weather forecast back! 

I then installed iPlayerWWW because it has similar dependencies, but didn't run it.  Restarted Kodi and still got weather info.  Ran BBC iPlayer and watched a channel for 10 seconds or so, then tried to open a link to a local streaming radio station (https://) - failed immediately.  Uninstalled iPlayer and restarted.  Weather info back again and streaming link works.

At this point I removed all the dependency directories and re-installed youtube.  I immediately got the error back, but started deleting the dependency directories one by one (with a restart each time) to see which (if any) resolved the issue. Removing both certifi and idna made no difference however removing 'requests' immediately resolved things at the next restart.

Further testing - Installed iPlayer, removed directory 'script.module.requests', restarted and verified iPlayer still works.  Installed Youtube, removed requests directory again, restarted and verified Youtube works and weather still works.  I presume this is because 'requests' is already installed as a module in python on this machine so Kodi is using the system installed version, rather than it's own.  Quite why the packaged module breaks SSL I haven't looked into yet, but as I can repeat the problem by installing it and fix it by removing it, I'm 100% sure it's the cause.

Problems it causes (in case anyone else suffers something similar !)

Unable to install any addons from official repository.
Unable to obtain weather info from Yahoo!.
Unable to open secure streaming links (https://).
Unable to cache art from sites that use secure connections.
Debug log full of SSL(35) errors when trying to establish a secure connection.
Learning Linux the hard way !!
Reply
#10
Can you still post the output of the commands I asked for?
NB: You need to run them in the non-working state.
Reply
#11
Sure.  http://paste.ubuntu.com/p/hYbBymVt5J/

Last bit was truncated but should have read

Code:
MAIL=/var/mail/xbmcUSER=xbmcLANGUAGE=en_GB:enUPSTART_INSTANCE=SHLVL=1HOME=/home/xbmcXDG_SESSION_COOKIE=fd0424922f65692458b50855539029f3-1537953352.926204-710526374UPSTART_JOB=xbmcKODI_HOME=/usr/share/kodiLOGNAME=xbmc_=/usr/bin/xinitMALLOC_MMAP_THRESHOLD_=131072XDG_SESSION_ID=c26TERM=linuxWINDOWPATH=7PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/gamesXDG_RUNTIME_DIR=/run/user/1000DISPLAY=:0LANG=en_GB.UTF-8LC_MESSAGES=C__GL_YIELD=USLEEPSHELL=/bin/bashLC_ALL=en_US.UTF-8PWD=/
Learning Linux the hard way !!
Reply
#12
Nothing stands out in particular Undecided Your CLI curl uses gnutls instead of openssl, but that shouldn't really matter.
Can you confirm that it breaks when just installing script.module.requests from the repo in a fresh installation?
Reply
#13
@yol - Ah well, just installing it won't break it, it has to be called once I think.

Evidence - requests is a dependency of iPlayerWWW.  I can install this and everything still works as expected, until I watch something with it.  After that, SSL is borked.  If I install Youtube (pretty much the same dependencies), it starts a service after installation and again, SSL is borked.  None of the dependency modules show up in Kodi's addon browser (because they shouldn't) so the only way to install it is either to get it pulled in as a dependency, or to copy the directory over.  I can easily do the latter, but I don't think that will make a difference unless something actually calls the library.  That's when it breaks.

I can confirm that if it is installed as a dependency, then after it has been called once, SSL no longer works correctly and that removing the entire directory (and only that directory) restores correct functionality. 

I restarted with a fresh installation, installed Youtube, backed out of the addon browser (to give time for YT to start up), went back in, tried to install Yahoo! weather and it failed.

Full debug log of session with cURL logging enabled -> https://paste.ubuntu.com/p/V3F4PkPKHG/

The failure occurs at line 4736 but the curl stuff just above looks correct to me.  Same path to the certificate store, same cert.

As an aside, my 10 year old son has a Dell netbook (Intel Atom based) that had Kodi v16.1 on.  Because I updated my versions, he decided to look in the wiki and do his (it's running on Lubuntu) last night.  This morning at breakfast, he told me that some of the radio shortcuts no longer work - That's the first symptom I noticed.  Removing the requests module on the netbook restored correct behaviour.  FWIW his machine reports this for uname -a and lsb_release -a.

Code:
Linux Inspiron-1011 4.4.0-34-generic #53-Ubuntu SMP Wed Jul 27 16:06:28 UTC 2016 i686 i686 i686 GNU/Linux
Distributor ID:    Ubuntu
Description:    Ubuntu 16.04.1 LTS
Release:    16.04
Codename:    xenial

His Kodi version is V17.6 from the official repository.

Common ground - both Intel chips, both running 32bit versions of Ubuntu 16.04 - I don't know if this is relevant or not, but thought I'd mention it as the same issue on two machines cannot just be a coincidence.
Learning Linux the hard way !!
Reply
#14
Does removing '<extension point="xbmc.service" library="resources/lib/startup.py" start="login"/>' from plugin.video.youtube/addon.xml (https://github.com/xbmc/repo-plugins/blo...on.xml#L12) also fix your issue.

Can you also try to remove script.module.certifi, script.module.chardet, script.module.idna and script.module.urllib3 one by one (with original youtube addon.xml and script.module.requests present). Those are all dependencies of script.module.requests.
Reply
#15
(2018-09-27, 13:41)Rechi Wrote: Does removing '<extension point="xbmc.service" library="resources/lib/startup.py" start="login"/>' from plugin.video.youtube/addon.xml (https://github.com/xbmc/repo-plugins/blo...on.xml#L12) also fix your issue.

Can you also try to remove script.module.certifi, script.module.chardet, script.module.idna and script.module.urllib3 one by one (with original youtube addon.xml and script.module.requests present). Those are all dependencies of script.module.requests.
 Well, it fixes it in that everything works as expected until you try and use Youtube.  I can search ok and it returns results but is then unable to play them as SSL returns the same error as in the other logs.  Same with BBC iPlayer which installs much the same dependencies.  It's fine as long as you don't try and use it.

I removed each module in turn the other day when testing to try and find out which was causing it.  Removing certifi, idna and chardet (with restarts in between) makes no difference at all.  Removing requests restores functionality.  Removing urllib3 breaks both youtube and iplayer to a non-functioning state and therefore there is nothing to call the requests library to test against.  However, I'm sure I can add urllib3 into my python installation and then I'll try again.

I have a logfile of the run with yt not starting at login if you need it.

*** EDIT ***

Installed urllib3 in python.  Restarted Kodi, opened streaming radio https link which worked.  Started youtube addon (only script.module.requests left of it's dependencies) and it searched ok but again was unable to actually play a video.  I have also got a saved log of this should it be required.
Learning Linux the hard way !!
Reply

Logout Mark Read Team Forum Stats Members Help
[bug] cURL unable to connect to any url starting with https0