Python performance gap between OSX/Windows

  Thread Rating:
  • 0 Votes - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Post Reply
elupus Offline
Team-XBMC Developer
Posts: 3,796
Joined: Mar 2004
Reputation: 3
Post: #16
You are not running a debug build of xbmc are you? They are horribly much slower.

Always read the XBMC online-manual, FAQ and search the forum before posting.
Do not e-mail XBMC-Team members directly asking for support. Read/follow the forum rules.
For troubleshooting and bug reporting please make sure you read this first.


[Image: badge.gif]
find quote
A600 Offline
Fan
Posts: 461
Joined: Oct 2003
Reputation: 7
Post: #17
elupus Wrote:You are not running a debug build of xbmc are you? They are horribly much slower.

No.
find quote
OriginNG Offline
Junior Member
Posts: 25
Joined: Jan 2010
Reputation: 0
Post: #18
elupus Wrote:Yea, it's a known issue (to me anyway). From what I can tell it's due to slow filesystem access on windows.

Windows xbmc is horribly slow at loading thumbnails and such from disk aswell.
Using procmon, from sysinternals.com, shows why it is so slow.
For example it scan throught sets.py 1 byte a time resulting in tens of thousands of file access. They can be buffered but is slow anyway.


Regards,

Joop
find quote
OriginNG Offline
Junior Member
Posts: 25
Joined: Jan 2010
Reputation: 0
Post: #19
A600 Wrote:Is urllib2.urlopen slow as hell on Windows 7? The same plugin running on XP and Xbox needs less than one second to open 16 webpages stored on a localhost but with Windows 7 it needs more than 30 seconds.

What is the main difference between XP and Windows 7?

Yep, you're not allowed anymore to write to C:\Program Files\XBMC and thus to system\python\Lib. That is if you don't run as Admin all the time and have stripped Win7 of all its default security settings.
I noticed that my Win7 install is indeed quite slow while my test install of XBMC under XP is faster. Reason, probably, being the fact that no *.pyo file can be written under Win7, which is NOT an error, but slows down python in general.
The quick and dirty way is to edit security on Lib to allow standard users to write to the folder. I have the feeling that is actually quicker right now.

Maybe there is a way to test this more in a quantitative way.

Regards,

Joop
find quote
spiff Offline
Retired Developer
Posts: 12,386
Joined: Nov 2003
Post: #20
ah, lol. that is quite the silly issue. wonder if we can have python dump its pyo in a fixed folder..
find quote
elupus Offline
Team-XBMC Developer
Posts: 3,796
Joined: Mar 2004
Reputation: 3
Post: #21
Well windows7 should redirect those writes to another folder normally.. Or so i though.. Maybe that isn't getting enabled on windows.

Always read the XBMC online-manual, FAQ and search the forum before posting.
Do not e-mail XBMC-Team members directly asking for support. Read/follow the forum rules.
For troubleshooting and bug reporting please make sure you read this first.


[Image: badge.gif]
find quote
CrashX Offline
Posting Freak
Posts: 1,164
Joined: Jan 2009
Reputation: 2
Post: #22
OriginNG Wrote:What is the main difference between XP and Windows 7?

Yep, you're not allowed anymore to write to C:\Program Files\XBMC and thus to system\python\Lib. That is if you don't run as Admin all the time and have stripped Win7 of all its default security settings.
I noticed that my Win7 install is indeed quite slow while my test install of XBMC under XP is faster. Reason, probably, being the fact that no *.pyo file can be written under Win7, which is NOT an error, but slows down python in general.
The quick and dirty way is to edit security on Lib to allow standard users to write to the folder. I have the feeling that is actually quicker right now.

Maybe there is a way to test this more in a quantitative way.

Regards,

Joop

Why are your python scripts located in that folder ? Shouldn't it be in %appdata% folder where regular users are allowed to write ...

The normal XBMC log IS NOT a debug log, to enable debug logging you must toggle it on under XBMC Settings - System or in advancedsettings.xml. Use XBMC Debug Log Addon to retrieve it.
find quote
galvanash Offline
Senior Member
Posts: 102
Joined: Jan 2010
Reputation: 0
Location: Mostly my couch
Post: #23
Just to confirm this. My plugin on Dharma Beta 2 under Windows was also slower than expected. If you give the "Users" group modify rights on "C:\Program Files (x86)\XBMC\system\python\lib", the problem completely goes away.

@ChrashX

It doesn't matter where your plugin is installed. The problem is that XBMC cannot cache its own compiled .pyos of anything (your .pys get cached fine) - meaning that every .py in the central python library has to be re-compiled on every run of a script or plugin. The more system .pys you use the slower it gets...

Changing the permissions fixes it.

note: as far as I can tell this is not related to the fix that was referenced by topfs2 (r32230). This is happening in Dharma Beta 2, which is much later than that build. Also, the explanation of that bug doesn't jive with this problem. I'm not sure exactly what SHOULD be happening (i.e. where the .pyo files should be written so that they can be read back by the current user - I suspect they should be written to %profile%/system) but XBMC is definitely trying to write them to the installation folder...
(This post was last modified: 2010-09-29 21:44 by galvanash.)
find quote
galvanash Offline
Senior Member
Posts: 102
Joined: Jan 2010
Reputation: 0
Location: Mostly my couch
Post: #24
elupus Wrote:Well windows7 should redirect those writes to another folder normally.. Or so i though.. Maybe that isn't getting enabled on windows.

If UAC redirection were happening, the writes would get redirected to "Users/%user%/AppData/Local/VirtualStore/Program Files (x86)/XBMC", but that definitely isn't happening with Dharma Beta 2.

I only do script stuff and have not done more than glance at the XBMC source now and again, is XBMC for Windows built with an app manifest and if so does it have a "requestedExecutionLevel" directive? If it has a manifest, and that directive is in it - that effectively tells Windows 7 to disable UAC redirection (which might be why this isn't working)... That directive is essentially telling Windows that you promise not to write to the appplication installation folder - it should not be included for apps that actually do perform writes to it.

If there is no manifest, or if that directive is omitted, redirection SHOULD be happening automatically - I don't know of any reason why it wouldn't. But regardless it isn't working...
find quote
galvanash Offline
Senior Member
Posts: 102
Joined: Jan 2010
Reputation: 0
Location: Mostly my couch
Post: #25
I decided to dive in and get my head around the XBMC source for a different reason and I remembered this thread...

I am very new to this so I don't want to make any assumptions - but from the previous discussion in this thread it appears that the XBMC developers want UAC redirection to function (i.e. they want writes to the installation folder redirected automatically).

I just did my first svn checkout against trunk and started digging. The VC2008 project is configured like this:

Generate Manifest=Yes
Manifest File=$(IntDir)\$(TargetFileName).intermediate.manifest
Allow Isolation=Yes
Enable User Account Control (UAC)=Yes
UAC Execution Level=asInvoker
UAC Bypass UI Protection=No

You can verify by going to Properties on the XBMC project and navigating to Configuration Properties->Linker->Manifest Files.

This setup as is will disable UAC redirection, i.e. writes to the installation folder will simply fail. If you want them redirected, the last 2 items have to be removed completely from the manifest, but I don't know if that can be done with the automatic manifest generation business - I think you have to create your own manifest and link it in. Either that or just don't generate a manifest at all.

I did a quick test - I generated a release build with no modifications other than disabling manifest generation. I then copied the resulting exe to my xbmc installation folder and ran it - UAC redirection kicked in and .pyo files started getting written to the virtualstore folder as expected - and they definitely appeared to be used (my plugin started up normally, no lag and no waiting throbber).

I'm not saying that disabling the manifest entirely is a good idea - I just did it to confirm my findings. I suspect you'll want to manually generate one - just without the requestedExecutionLevel tag. Either that or handle the redirection in code to - maybe to profile/system or something.

See here for further info: http://msdn.microsoft.com/en-us/library/bb756929.aspx
(This post was last modified: 2010-10-01 08:29 by galvanash.)
find quote
galvanash Offline
Senior Member
Posts: 102
Joined: Jan 2010
Reputation: 0
Location: Mostly my couch
Post: #26
I submitted a ticket for this: Ticket #10392
find quote
Post Reply