Kodi Community Forum
Newznab - Anyone using it? - Printable Version

+- Kodi Community Forum (https://forum.kodi.tv)
+-- Forum: Support (https://forum.kodi.tv/forumdisplay.php?fid=33)
+--- Forum: Supplementary Tools for Kodi (https://forum.kodi.tv/forumdisplay.php?fid=116)
+--- Thread: Newznab - Anyone using it? (/showthread.php?tid=82119)

Pages: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17


RE: Newznab - Anyone using it? - Tigggger - 2012-12-18

(2012-12-11, 14:05)fungi Wrote: Also when scraping a certain group the update binaries script gets stuck on this for long periods of time:

Code:
Processing alt.binaries.hdtv.x264
Attempting to repair 1 parts...
-repairing 0 to 0

Not too sure what's going on there...

Ended up here thanks to google as I had the same problem, having had no luck finding the solution I managed to figure it out.

It appears to be stuck on repair and in the mysql database is a table called 'partrepair', mine had 1 entry in it which I deleted and normal service was resumed.




RE: Newznab - Anyone using it? - jatilq - 2012-12-18

I did a re-install and adding groups 2 at a time after a cycle, but getting this error.

Notice: Undefined property: stdClass::$curlproxytype in C:\xampp\htdocs\nnplus\w
ww\lib\util.php on line 151

Notice: Undefined property: stdClass::$curlproxyuserpwd in C:\xampp\htdocs\nnplu
s\www\lib\util.php on line 152
TVRage : Didnt find Bad Girls locally, checking web

Notice: Undefined property: stdClass::$curlproxyaddress in C:\xampp\htdocs\nnplu
s\www\lib\util.php on line 150

Notice: Undefined property: stdClass::$curlproxytype in C:\xampp\htdocs\nnplus\w
ww\lib\util.php on line 151

Notice: Undefined property: stdClass::$curlproxyuserpwd in C:\xampp\htdocs\nnplu
s\www\lib\util.php on line 152



RE: Newznab - Anyone using it? - lurkingsheep - 2012-12-18

(2012-12-18, 09:49)eluSive_NZ Wrote:
(2012-12-17, 20:21)lurkingsheep Wrote: How often should I should run update_binaries.php & update_releases.php ?

Do I need to set this up in CRON or is this being done automatically?

Thanks

Was in the same boat trying to figure this out.
I tried with Byobu and its currently running great! Just running the newznab_screen.sh within this

Thanks. After lurking on the newsnab IRC for a while, I understand that the "newznab_screen.sh" script is designed to run in a loop using update_binaries.php & update_releases.php automatically so there is no need to run "update_binaries.php & update_releases.php" in a CRON.

The only thing I have yet to figure out is how to start "newznab_screen.sh" as a deamon upon startup without having to manually use the screen.




RE: Newznab - Anyone using it? - cflynt - 2012-12-18

Just some questions, I have this running on an old P4 with 1G ram, runs fine, slow of course but seems it is slow regardless on what it runs on heh. I have seen in the earlier posts on this thread about changing the setup from 20,000 headsers to 1Million or 1 Day. And resetting. Is resetting necessary, and will it not get all of them properly after a few runs anyways? Just wanted to make sure I am setting up right. Already have quite a database built, would hate to purge it heh.

Next, just due to ignorance of not understanding, being as this is just a P4 with 1G of ram, can and should I still run the threaded scripts instead of the regular? Will it do ok if I don't have dual cores etc?

I am currently running on Astra, but would like to switch to a different provider, I have heard in IRC that you need to do a few steps to switch providers. I know where to change it in the config.php but didn't know what the proper steps otherwise were to switch things around.

And lastly for now at least heh, due to the mass hit on TVRage, it seems that lots of things aren't getting ID's so SickBeard isn't seeing everything when it searches. Is there a way to make it redo all the TVRage ID's so that everything gets populated correctly after it has already done the releases?

Thanks for any help, and thanks for the community for making such great stuff!


RE: Newznab - Anyone using it? - tododo - 2012-12-18

If you find an answer to make it start on boot and automatically poll I would be interested also

Ive got mine currenty set to 100days back fill and 200000 max messaged - not sure if thats any good though




RE: Newznab - Anyone using it? - wade32505 - 2012-12-18

(2012-12-18, 16:24)lurkingsheep Wrote:
(2012-12-18, 09:49)eluSive_NZ Wrote:
(2012-12-17, 20:21)lurkingsheep Wrote: How often should I should run update_binaries.php & update_releases.php ?

Do I need to set this up in CRON or is this being done automatically?

Thanks

Was in the same boat trying to figure this out.
I tried with Byobu and its currently running great! Just running the newznab_screen.sh within this

Thanks. After lurking on the newsnab IRC for a while, I understand that the "newznab_screen.sh" script is designed to run in a loop using update_binaries.php & update_releases.php automatically so there is no need to run "update_binaries.php & update_releases.php" in a CRON.

The only thing I have yet to figure out is how to start "newznab_screen.sh" as a deamon upon startup without having to manually use the screen.

Below are the notes I copied when I setup the screen auto update:
In the nix_scripts directory there is a useful script called newznab_screen.sh that runs update_binaries.php and update_releases.php, in addition to a few other scripts, continuously and automatically. First, we need to modify it however, so lets change dir and make a copy:

cd /var/www/newznab/misc/update_scripts/nix_scripts
cp newznab_screen.sh newznab_screen_local.sh

Now open newznab_screen_local.sh in a text editor and modify NEWZNAB_PATH near the top to point to our installation path:

nano newznab_screen_local.sh

Set NEWZNAB_PATH:

export NEWZNAB_PATH="/var/www/newznab/misc/update_scripts"

Now we can run the script via screen:

screen sh newznab_screen_local.sh

You should see the script download headers for the groups that you have enabled and then run various stages that will attempt to group and catalogue the headers. For now, just leave the script running and detach from screen by typing cntl a d.
You can reattach the screen by first entering 'screen -ls'. This gives your the process number. To reattach you enter 'screen -r [process number].

My newznab is setup on a netbook with 1gig of ram and 160gig of harddrive. It's closed up sitting on a shelf and I access it via ssh.


RE: Newznab - Anyone using it? - eluSive_NZ - 2012-12-18

(2012-12-18, 20:06)wade32505 Wrote: Below are the notes I copied when I setup the screen auto update:
In the nix_scripts directory there is a useful script called newznab_screen.sh that runs update_binaries.php and update_releases.php, in addition to a few other scripts, continuously and automatically. First, we need to modify it however, so lets change dir and make a copy:

cd /var/www/newznab/misc/update_scripts/nix_scripts
cp newznab_screen.sh newznab_screen_local.sh

Now open newznab_screen_local.sh in a text editor and modify NEWZNAB_PATH near the top to point to our installation path:

nano newznab_screen_local.sh

Set NEWZNAB_PATH:

export NEWZNAB_PATH="/var/www/newznab/misc/update_scripts"

Now we can run the script via screen:

screen sh newznab_screen_local.sh

You should see the script download headers for the groups that you have enabled and then run various stages that will attempt to group and catalogue the headers. For now, just leave the script running and detach from screen by typing cntl a d.
You can reattach the screen by first entering 'screen -ls'. This gives your the process number. To reattach you enter 'screen -r [process number].

My newznab is setup on a netbook with 1gig of ram and 160gig of harddrive. It's closed up sitting on a shelf and I access it via ssh.

This is what I did, but used Byobu instead of screen
Byobu is quite handy as you can add stats to the screen e.g. network (u/d), disk io, system load etc etc
When the script seems stuck, you can check the stats quickly to see if it is still "busy"



RE: Newznab - Anyone using it? - Wozman - 2012-12-18

(2012-12-18, 00:58)jatilq Wrote: Now though, when I run the update releases script, I am having problems with mediainfo. When the script uses the mediainfo app, it opens a mediainfo window and wont do anything until I close the window manually.

I had the same problem, which was that I was using the windows gui version that I had installed already. Go get the CLI version from here and use the path to the new command line exe fle, that should get rid of the mediainfo window popping up for every video release.


RE: Newznab - Anyone using it? - Aenima99x - 2012-12-19

(2012-12-18, 21:05)eluSive_NZ Wrote:
(2012-12-18, 20:06)wade32505 Wrote: Below are the notes I copied when I setup the screen auto update:
In the nix_scripts directory there is a useful script called newznab_screen.sh that runs update_binaries.php and update_releases.php, in addition to a few other scripts, continuously and automatically. First, we need to modify it however, so lets change dir and make a copy:

cd /var/www/newznab/misc/update_scripts/nix_scripts
cp newznab_screen.sh newznab_screen_local.sh

Now open newznab_screen_local.sh in a text editor and modify NEWZNAB_PATH near the top to point to our installation path:

nano newznab_screen_local.sh

Set NEWZNAB_PATH:

export NEWZNAB_PATH="/var/www/newznab/misc/update_scripts"

Now we can run the script via screen:

screen sh newznab_screen_local.sh

You should see the script download headers for the groups that you have enabled and then run various stages that will attempt to group and catalogue the headers. For now, just leave the script running and detach from screen by typing cntl a d.
You can reattach the screen by first entering 'screen -ls'. This gives your the process number. To reattach you enter 'screen -r [process number].

My newznab is setup on a netbook with 1gig of ram and 160gig of harddrive. It's closed up sitting on a shelf and I access it via ssh.

This is what I did, but used Byobu instead of screen
Byobu is quite handy as you can add stats to the screen e.g. network (u/d), disk io, system load etc etc
When the script seems stuck, you can check the stats quickly to see if it is still "busy"

Why don't you guys just use the init script and run it as a service?


RE: Newznab - Anyone using it? - farscapesg01 - 2012-12-19

(2012-12-18, 07:02)farscapesg01 Wrote: Luckily there isn't anything airing the next couple days that I "need" to grab... so I'm just going to let it run. My monthly usage is already at 225GB being reported by Comcast, so that gives me a base point to watch over the next couple days (it was 219GB last night, but I did grab about 2 GB of files yesterday so it just depends on when it hits their reporting).

I'm assuming deep-inspection for passworded files would increase the bandwidth, so I could turn that off. I'm pulling headers for about 22 newsgroups, the busiest being a.b.e-books, a.b.multimedia, and a.b.teevee.

Well... two days later and I'm up to 233GB used, an 8GB increase and it hasn't been a full 48 hours even. Looks like Newznab is chewing through about 5 GB of bandwidth a day just by itself. 150GB per month just to run an indexer just doesn't sound right and isn't going to be maintanable with my other Internet usage.

Anyone have any suggestions for limiting the bandwidth usage, before I give up and use existing options? I guess I could cut out TV and rely on Sickbeard's indexer to pull my shows that way. I really like the idea of running my own indexer though....


RE: Newznab - Anyone using it? - zepfan - 2012-12-19

(2012-12-19, 17:24)farscapesg01 Wrote:
(2012-12-18, 07:02)farscapesg01 Wrote: Luckily there isn't anything airing the next couple days that I "need" to grab... so I'm just going to let it run. My monthly usage is already at 225GB being reported by Comcast, so that gives me a base point to watch over the next couple days (it was 219GB last night, but I did grab about 2 GB of files yesterday so it just depends on when it hits their reporting).

I'm assuming deep-inspection for passworded files would increase the bandwidth, so I could turn that off. I'm pulling headers for about 22 newsgroups, the busiest being a.b.e-books, a.b.multimedia, and a.b.teevee.

Well... two days later and I'm up to 233GB used, an 8GB increase and it hasn't been a full 48 hours even. Looks like Newznab is chewing through about 5 GB of bandwidth a day just by itself. 150GB per month just to run an indexer just doesn't sound right and isn't going to be maintanable with my other Internet usage.

Anyone have any suggestions for limiting the bandwidth usage, before I give up and use existing options? I guess I could cut out TV and rely on Sickbeard's indexer to pull my shows that way. I really like the idea of running my own indexer though....

Check and see if your host allows compressed headers. I turned it on and it helped a lot.


RE: Newznab - Anyone using it? - slithers - 2012-12-19

(2012-12-19, 17:24)farscapesg01 Wrote:
(2012-12-18, 07:02)farscapesg01 Wrote: Luckily there isn't anything airing the next couple days that I "need" to grab... so I'm just going to let it run. My monthly usage is already at 225GB being reported by Comcast, so that gives me a base point to watch over the next couple days (it was 219GB last night, but I did grab about 2 GB of files yesterday so it just depends on when it hits their reporting).

I'm assuming deep-inspection for passworded files would increase the bandwidth, so I could turn that off. I'm pulling headers for about 22 newsgroups, the busiest being a.b.e-books, a.b.multimedia, and a.b.teevee.

Well... two days later and I'm up to 233GB used, an 8GB increase and it hasn't been a full 48 hours even. Looks like Newznab is chewing through about 5 GB of bandwidth a day just by itself. 150GB per month just to run an indexer just doesn't sound right and isn't going to be maintanable with my other Internet usage.

Anyone have any suggestions for limiting the bandwidth usage, before I give up and use existing options? I guess I could cut out TV and rely on Sickbeard's indexer to pull my shows that way. I really like the idea of running my own indexer though....

are you having it pull down back fills? obviously your bandwidth will be greater pulling down things from the past, then it will be just for grabbing newly added files. I suspect you are downloading files from a while back. Once you have grabbed all the older information you requested, then your bandwidth usage should drop significantly as it just checks and downloads new items



RE: Newznab - Anyone using it? - richierea - 2012-12-19

If I was hitting anywhere near 5GB a day I wouldn't bother either. Not sure which groups you have but I have 11 TV / Movie and music groups totaling around 200MB a day.
Am I missing some of the scraping options that use more bandwidth? or just a lot less groups?. What I have seems to be fine so far for the TV shows I need, but will probably add some more groups.




RE: Newznab - Anyone using it? - farscapesg01 - 2012-12-19

(2012-12-19, 19:14)zepfan Wrote:
(2012-12-19, 17:24)farscapesg01 Wrote:
(2012-12-18, 07:02)farscapesg01 Wrote: Luckily there isn't anything airing the next couple days that I "need" to grab... so I'm just going to let it run. My monthly usage is already at 225GB being reported by Comcast, so that gives me a base point to watch over the next couple days (it was 219GB last night, but I did grab about 2 GB of files yesterday so it just depends on when it hits their reporting).

I'm assuming deep-inspection for passworded files would increase the bandwidth, so I could turn that off. I'm pulling headers for about 22 newsgroups, the busiest being a.b.e-books, a.b.multimedia, and a.b.teevee.

Well... two days later and I'm up to 233GB used, an 8GB increase and it hasn't been a full 48 hours even. Looks like Newznab is chewing through about 5 GB of bandwidth a day just by itself. 150GB per month just to run an indexer just doesn't sound right and isn't going to be maintanable with my other Internet usage.

Anyone have any suggestions for limiting the bandwidth usage, before I give up and use existing options? I guess I could cut out TV and rely on Sickbeard's indexer to pull my shows that way. I really like the idea of running my own indexer though....

Check and see if your host allows compressed headers. I turned it on and it helped a lot.

I'm using Newshosting.com, and unfortunately it appears Astra is about the only one that works "reliably" for compression. Tried turning on compression and it just hangs trying to download the headers.


RE: Newznab - Anyone using it? - farscapesg01 - 2012-12-19

Haven't done any backfilling for the last several days so I can figure out how much bandwidth is actually being used. Also suspended all SABNZBD and torrent activity. Here are the groups I'm currently indexing...

Quote:a.b.documentaries
a.b.e-book
a.b.e-book.flood
a.b.e-book.technical
a.b.ebook
a.b.etc
a.b.games.nintendods
a.b.games.wii
a.b.hdtv
a.b.hdtv.x264
a.b.misc
a.b.moovee
a.b.movies
a.b.movies.divx
a.b.movies.xvid
a.b.multimedia
a.b.multimedia.cartoons
a.b.multimedia.disney
a.b.teevee
a.b.tv
a.b.tvseries
a.b.x264

Some of the slower ones I'll probaby drop, like multimedia.disney, a.b.tvseries, and a.b.x264 that I'm getting very few results from.