[RELEASE] Pneumatic

  Thread Rating:
  • 9 Votes - 4.56 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Post Reply
pleo Offline
Junior Member
Posts: 2
Joined: Nov 2006
Reputation: 0
Post: #1681
(2014-08-15 11:14)Popeye Wrote:  
(2014-07-03 21:46)timgt Wrote:  It works fine in Gotham, even Helix which is what I am running. Just make a small change in the addon.xml file so that the python version says python version 2.1.0 and then it installs fine.

Popeye, is there any way to have Pneumatic look in the par2 file before anything else? This file is what shows what order the .rar files are in when they use the format like .10, .11, .12, etc. which are out of order.
These are the files that Pneumatic fails to find a playable file in. I'm sure it would require a lot of work but is it possible that this is the solution for the encrypted/hashed files that fail to play??

As far as I remember I looked into parsing the par2 file for getting the rar file order. However I couldn't find a clear table with nameX->nameY matches. The renaming is done once some checksum is calculated and for that the whole file needs to be present (AFAIK). Thus to get the correct order, all files has to be home and then the purpose of pneumatic is null...

This actually is possible to do without downloading every file. The trick is to use the par2 files to check the MD5 hash of the first 16KB of each file included in the parset. So, typically you need to grab the first segment of every file, rather than the whole post. I put together a proof of concept that takes an NZB, remaps the mangled names and creates a new NZB with the correct rar file names (serial and threaded versions). It is pretty quick, with the threaded version taking about a minute or two to process a 90+ rar file post using 20 nntp connections.

Some guy also wrote an NZB verification tool, which very quickly tests for the existence of each segment. These two tools, used together, can save a lot of time on downloads that otherwise would be a waste of time due to incompleteness or name mangling. Now if only SAB had these abilities integrated...
find quote
Popeye Offline
Posting Freak
Posts: 922
Joined: Aug 2009
Reputation: 26
Location: Sweden
Post: #1682
(2014-08-21 20:06)pleo Wrote:  
(2014-08-15 11:14)Popeye Wrote:  
(2014-07-03 21:46)timgt Wrote:  It works fine in Gotham, even Helix which is what I am running. Just make a small change in the addon.xml file so that the python version says python version 2.1.0 and then it installs fine.

Popeye, is there any way to have Pneumatic look in the par2 file before anything else? This file is what shows what order the .rar files are in when they use the format like .10, .11, .12, etc. which are out of order.
These are the files that Pneumatic fails to find a playable file in. I'm sure it would require a lot of work but is it possible that this is the solution for the encrypted/hashed files that fail to play??

As far as I remember I looked into parsing the par2 file for getting the rar file order. However I couldn't find a clear table with nameX->nameY matches. The renaming is done once some checksum is calculated and for that the whole file needs to be present (AFAIK). Thus to get the correct order, all files has to be home and then the purpose of pneumatic is null...

This actually is possible to do without downloading every file. The trick is to use the par2 files to check the MD5 hash of the first 16KB of each file included in the parset. So, typically you need to grab the first segment of every file, rather than the whole post. I put together a proof of concept that takes an NZB, remaps the mangled names and creates a new NZB with the correct rar file names (serial and threaded versions). It is pretty quick, with the threaded version taking about a minute or two to process a 90+ rar file post using 20 nntp connections.

Some guy also wrote an NZB verification tool, which very quickly tests for the existence of each segment. These two tools, used together, can save a lot of time on downloads that otherwise would be a waste of time due to incompleteness or name mangling. Now if only SAB had these abilities integrated...

This is something for Sabnzbd. Pneumatic have a very limited control over Sabnzbd and the API does not provide any option to only fetch a certain amount of bytes...

sverigesradio | Pneumatic | SABnzbd | XBMC that just works - openelec
find quote
hasamba Offline
Member
Posts: 60
Joined: Sep 2012
Reputation: 0
Post: #1683
did someone was able to bypass the "malformed error"?
popeye, can you please fix it? if not, can you please answer and tell us why not?

thanks
find quote
hasamba Offline
Member
Posts: 60
Joined: Sep 2012
Reputation: 0
Post: #1684
(2014-09-13 14:22)hasamba Wrote:  did someone was able to bypass the "malformed error"?
popeye, can you please fix it? if not, can you please answer and tell us why not?

thanks

popeye, can we get any comment please?
find quote
hasamba Offline
Member
Posts: 60
Joined: Sep 2012
Reputation: 0
Post: #1685
bump...

nothing? just ignore?
find quote
VladTepz Offline
Junior Member
Posts: 18
Joined: Aug 2012
Reputation: 0
Post: #1686
(2014-08-19 15:42)Popeye Wrote:  
(2014-08-19 12:48)xbmosx Wrote:  thank you popeye. very much appreciate your work!

I just did a quick google and found that XBMC apparently supports encrypted rars. The URI is rar://passwor@file://path/to/file.rar/content.mp4 so my suggested fix might work...

I was able to get this to work, although in a rather hacky, proof-of-concept way, by manually adding the Rar Password to line 516 & 520 of the default.py

Code:
515:            for arch_rar, movie_file in zip(play_list[0::2], play_list[1::2]):
516:                        raruri = "rar://RARPASSWORD@" + utils.rarpath_fixer(folder, arch_rar) + "/" + movie_file
517:                        rar.append(raruri)
518:                        raruri = 'stack://' + ' , '.join(rar)
519:            else:
520:                raruri = "rar://RARPASSWORD@" + utils.rarpath_fixer(folder, play_list[0]) + "/" + play_list[1]
521:            uri = raruri

Still looking for a more elegant solution. hint hint... Wink

-Vlad
find quote
wolverine30008 Offline
Junior Member
Posts: 30
Joined: May 2014
Reputation: 0
Post: #1687
Popeye

I have been beating my head against the wall trying to get this to work.I downloaded all the needed components like newznab and puenamic if that was spelled correctly also I just signed up for a uns account with high speed downloads and high retention but anyway I followed your tutorial to the T with site urls and keys but I just cant seem to get this working on either frodo or gotham Finally my questions are-is this add on still supported?Why does it keep saying newzab addon script errror?It all works fine straight from the site and via sick beard not to mention I can stream with puenamic just fine with sabnzbd while its incomplete.Any help or advice would be appreciated.

Thanls

Gotham 13.2
Frodo 12.3
Mac OSX 10.9

Here is the best I could do for a debug log.Noob-cant figure out how to post my xbmc log file to this post.
23:18:07 T:2954457088 DEBUG: Process - Entering source directory /Users/wolverine30008/Library/Application Support/XBMC/addons/plugin.video.nzbtv
23:18:07 T:2954457088 DEBUG: Instantiating addon using automatically obtained id of "plugin.video.nzbtv" dependent on version 2.1.0 of the xbmc.python api
23:18:07 T:2706284968 DEBUG: ------ Window Init (DialogBusy.xml) ------
23:18:08 T:2953924608 DEBUG: WaitOnScriptResult- plugin returned successfully
23:18:08 T:2954457088 INFO: Scriptresult: Success
23:18:08 T:2959323136 NOTICE: Thread Background Loader start, auto delete: false
23:18:08 T:2959323136 DEBUG: Thread Background Loader 2959323136 terminating
23:18:08 T:2959323136 NOTICE: Thread Background Loader start, auto delete: false
23:18:08 T:2959323136 DEBUG: Thread Background Loader 2959323136 terminating
23:18:08 T:2954457088 INFO: Python script stopped
23:18:08 T:2954457088 DEBUG: Thread XBPyThread 2954457088 terminating
23:18:08 T:2706284968 DEBUG: waiting for python thread 15 (/Users/wolverine30008/Library/Application Support/XBMC/addons/plugin.video.nzbtv/default.py) to stop
23:18:08 T:2706284968 DEBUG: python thread 15 (/Users/wolverine30008/Library/Application Support/XBMC/addons/plugin.video.nzbtv/default.py) destructed
23:18:08 T:2706284968 DEBUG: ------ Window Deinit (DialogBusy.xml) ------
23:18:12 T:2706284968 DEBUG: ------ Window Init (Pointer.xml) ------
23:18:12 T:2706284968 DEBUG: ------ Window Deinit (DialogKaiToast.xml) ------
23:18:16 T:2706284968 DEBUG: ------ Window Deinit (Pointer.xml) ------
23:18:18 T:2706284968 DEBUG: CAnnouncementManager - Announcement: OnQuit from xbmc
find quote
wolverine30008 Offline
Junior Member
Posts: 30
Joined: May 2014
Reputation: 0
Post: #1688
Last post #1687 was frodp 12.3

Here is the best I could make out with gotham 13.2
23:28:33 T:2706284968 DEBUG: ParentPath = [addons://sources/video/]
23:28:33 T:2961985536 DEBUG: StartScript - calling plugin Newznab('plugin://plugin.video.newznab/','1','')
23:28:33 T:2961985536 DEBUG: WaitOnScriptResult - waiting on the Newznab (id=5) plugin...
23:28:33 T:2953318400 NOTICE: Thread LanguageInvoker start, auto delete: false
23:28:33 T:2953318400 INFO: initializing python engine.
23:28:33 T:2953318400 DEBUG: CPythonInvoker(5, /Users/wolverine30008/Library/Application Support/XBMC/addons/plugin.video.newznab/default.py): start processing
23:28:33 T:2953318400 NOTICE: -->Python Interpreter Initialized<--
23:28:33 T:2953318400 DEBUG: CPythonInvoker(5, /Users/wolverine30008/Library/Application Support/XBMC/addons/plugin.video.newznab/default.py): the source file to load is /Users/wolverine30008/Library/Application Support/XBMC/addons/plugin.video.newznab/default.py
23:28:33 T:2953318400 DEBUG: CPythonInvoker(5, /Users/wolverine30008/Library/Application Support/XBMC/addons/plugin.video.newznab/default.py): setting the Python path to /Users/wolverine30008/Library/Application Support/XBMC/addons/plugin.video.newznab:/Users/wolverine30008/Library/Application Support/XBMC/addons/script.module.six/lib:/Users/wolverine30008/Library/Application Support/XBMC/addons/script.module.simplejson/lib:/Users/wolverine30008/Library/Application Support/XBMC/addons/plugin.program.sabnzbd/lib:/Users/wolverine30008/Library/Application Support/XBMC/addons/script.module.beautifulsoup/lib:/Applications/XBMC.app/Contents/Resources/XBMC/addons/script.module.pil/lib:/Users/wolverine30008/Library/Application Support/XBMC/addons/script.common.plugin.cache/lib:/Users/wolverine30008/Library/Application Support/XBMC/addons/script.module.simple.downloader/lib:/Applications/XBMC.app/Contents/Resources/XBMC/addons/xbmc.debug/:/Users/wolverine30008/Library/Application Support/XBMC/addons/script.module.parsedom/lib:/Users/wolverine30008/Library/Application Support/XBMC/addons/script.module.xbmcswift2/lib:/Applications/XBMC.app/Contents/Frameworks:/Applications/XBMC.app/Contents/Frameworks/lib/python26.zip:/Applications/XBMC.app/Contents/Frameworks/lib/python2.6:/Applications/XBMC.app/Contents/Frameworks/lib/python2.6/plat-darwin:/Applications/XBMC.app/Contents/Frameworks/lib/python2.6/plat-mac:/Applications/XBMC.app/Contents/Frameworks/lib/python2.6/plat-mac/lib-scriptpackages:/Applications/XBMC.app/Contents/Frameworks/lib/python2.6/lib-tk:/Applications/XBMC.app/Contents/Frameworks/lib/python2.6/lib-old:/Applications/XBMC.app/Contents/Frameworks/lib/python2.6/lib-dynload:/Applications/XBMC.app/Contents/Frameworks/lib/python2.6/site-packages:/Applications/XBMC.app/Contents/Frameworks/lib/python2.6/site-packages/PIL
23:28:33 T:2953318400 DEBUG: CPythonInvoker(5, /Users/wolverine30008/Library/Application Support/XBMC/addons/plugin.video.newznab/default.py): entering source directory /Users/wolverine30008/Library/Application Support/XBMC/addons/plugin.video.newznab
23:28:33 T:2953318400 DEBUG: CPythonInvoker(5, /Users/wolverine30008/Library/Application Support/XBMC/addons/plugin.video.newznab/default.py): instantiating addon using automatically obtained id of "plugin.video.newznab" dependent on version 2.1.0 of the xbmc.python api
23:28:33 T:2706284968 DEBUG: ------ Window Init (DialogBusy.xml) ------
23:28:33 T:2706284968 DEBUG: ------ Window Init (DialogAddonSettings.xml) ------
23:28:33 T:2706284968 INFO: Loading skin file: DialogAddonSettings.xml, load type: KEEP_IN_MEMORY
23:28:33 T:2706284968 DEBUG: LocalizeStrings: no strings.po file exist at /Users/wolverine30008/Library/Application Support/XBMC/addons/plugin.video.newznab/resources/language/English, fallback to strings.xml
23:28:34 T:270628496
find quote
wolverine30008 Offline
Junior Member
Posts: 30
Joined: May 2014
Reputation: 0
Post: #1689
FYI Newznab v 0.2.5,NZBtv v 0.2.7 and Pneumatic v 0.3.1 Regarding post 1687-1688.
I use usenet crawler and nmatrix as my nzb indexers.Now one thing Im curious on though-How do these addons know you have server access?I use usenetserver.Is there somewhere I am supposed to be inputing that?
Finally,this should go without saying for all of us xbmc users Thank-You so much for your time and effort on all these add ons.Hope to hear from you soon.
(This post was last modified: 2014-09-26 19:34 by wolverine30008.)
find quote
wolverine30008 Offline
Junior Member
Posts: 30
Joined: May 2014
Reputation: 0
Post: #1690
OK have it all working besides browse local NZBs how do I set this is settings.I have the folder where I store my nzb files set it the NZB chache settings but every time I choose browse local NZBs its empty.What am II missing??how do i set it in settings to "browse local NZBs" is it in SABnzbd or is it a blackhole kind of setup.I cant figure it out-Can anyone help please??
find quote
Post Reply