I'm fairly new to this so please be gentle...
My server has 7 day retension but the amount of headers I have to download in certain groups (alt.binaries.nl, alt.binaries.dvd... etc) are just too many. My hard drive starts "thrashing" and my header downloads slow down to 3 - 10 K/Sec from 390 - 400K/Sec. I was told the problem was the amount of headers I was downloading and processing at 1 time.(like 2,000,00 or so) To fix this I've changed my server and newsgroup retension to be "1 - 3" but it still downloads all the headers. I'm sure I'm just missing something. Any suggestions would be greatly appreciated. (Will more RAM help? I have a P4 1.6, 512MB DDR.)
Header downloads
I am also a newbie but perhaps my experiences will help. I initially set retention to "natural". It downloaded everything. The system often seemed to bog down, including many long periods of 100%cpu (XP2600Barton running at 2.3gb, 512mb ram running at 387mhz)
So I changed the retention to 1-3 . I expected that it would automatically delete everything older than 3 days. Big mistake! Nothing changed.
So I deleted everything and started over from absolute scratch, deleting the db and re-installing NP. This time I initially set retention (both server and group) to 1-3. I then told it to get new headers.
It appears to read EVERYTHING in the group(s), though, now, at least, it only keeps stuff that's newer than 3 days old.
For example, I had put a port of "0" (zero) on my Astraweb server but didn't notice till today (I set this all up from start again yesterday) that it was unable to read it. (The error messages are in the "error" tab to the right of the "header" tab in the "tasks".) So far, the bandwidth monitor for that server says 293MB, though it has kept only about 30,000 headers, very near to what it has for my other (Verizon) server.
I am not at all sure where the problem lies - only that there is one.....
Share your findings if you have successes!
Kristi
So I changed the retention to 1-3 . I expected that it would automatically delete everything older than 3 days. Big mistake! Nothing changed.
So I deleted everything and started over from absolute scratch, deleting the db and re-installing NP. This time I initially set retention (both server and group) to 1-3. I then told it to get new headers.
It appears to read EVERYTHING in the group(s), though, now, at least, it only keeps stuff that's newer than 3 days old.
For example, I had put a port of "0" (zero) on my Astraweb server but didn't notice till today (I set this all up from start again yesterday) that it was unable to read it. (The error messages are in the "error" tab to the right of the "header" tab in the "tasks".) So far, the bandwidth monitor for that server says 293MB, though it has kept only about 30,000 headers, very near to what it has for my other (Verizon) server.
I am not at all sure where the problem lies - only that there is one.....
Share your findings if you have successes!
Kristi
Shuttle AN35N Ultra, XP2600Bart(11.5x400), 512mb, GeF4Ti4200, WinXPproSP1.
Thank you very much. It worked. Now for my next question. Does it affect my speed depending on what groups I download from? I can download headers at 390K/Sec now (Thank you very much Kristi) but my data downloads start about 200K/Sec and level off at around 70 - 80K/Sec. Is there a setting I need to check? I use astraweb and the advertise a speed of 600 Kbit/s per connection with 4 connections. Am I getting what I should be?
Thank you again for your help.
Thank you again for your help.
Yay! Progress!!!
I have a Verizon DSL connection that tests at 1.69 Mbps (1.45 to megapath) and usually maxes at 190KBps. Going against Astraweb, the plot looks pretty shabby with one thread running, but is solid 190 with 3 threads. I run 6 threads max, for what it's worth.
So if I click update groups in NP, the graph (I use tweakmeter but there are lots of free network graph proggies)(XP has one built in but I never use it) looks shabby, and after a minute is solid at 190, and when it gets down to the last two groups it looks shabby, and when it gets down to the last group it looks really shabby! This is ""mostly"" not NP's fault, but Astraweb's fault - seems to average roughly 60 (?) per thread. I say ""mostly"" cause just about any other multithread newsreader that I try does a bit better. The difference is probably in what (what?????) NP is doing when it reads a group either originally, or for updates - it apparently reads a lot more in each case. I am not sure of the benefit since I don't know what it's doing!
I confess to comparing results of NP with Grabit newsreader - lots of options vs almost no options. But on Astraweb where completion rate is so high and retention so (relatively) long, it tells me very quickly when I have a prob with one or the other!
Best!
Kristi
I check divx, vcd, and svcd binary groups - 7 in all
2,000,000 headers shouldn't be a problem and fie on the newsreader that can't handle it!
I have a Verizon DSL connection that tests at 1.69 Mbps (1.45 to megapath) and usually maxes at 190KBps. Going against Astraweb, the plot looks pretty shabby with one thread running, but is solid 190 with 3 threads. I run 6 threads max, for what it's worth.
So if I click update groups in NP, the graph (I use tweakmeter but there are lots of free network graph proggies)(XP has one built in but I never use it) looks shabby, and after a minute is solid at 190, and when it gets down to the last two groups it looks shabby, and when it gets down to the last group it looks really shabby! This is ""mostly"" not NP's fault, but Astraweb's fault - seems to average roughly 60 (?) per thread. I say ""mostly"" cause just about any other multithread newsreader that I try does a bit better. The difference is probably in what (what?????) NP is doing when it reads a group either originally, or for updates - it apparently reads a lot more in each case. I am not sure of the benefit since I don't know what it's doing!
I confess to comparing results of NP with Grabit newsreader - lots of options vs almost no options. But on Astraweb where completion rate is so high and retention so (relatively) long, it tells me very quickly when I have a prob with one or the other!
Best!
Kristi
I check divx, vcd, and svcd binary groups - 7 in all
2,000,000 headers shouldn't be a problem and fie on the newsreader that can't handle it!
Shuttle AN35N Ultra, XP2600Bart(11.5x400), 512mb, GeF4Ti4200, WinXPproSP1.
as downloading bodies it should utilize connection in full, newspro just copies data from network almost directly to files, the bottleneck should be the hard disk speed.
also i recall there is a setting in properties->tasks, set socket buffers, you may try to increase it value (e.g. to 64000) to see whether it changes something. it can only influence the network part since as to file part the system has its own buffering so the data is automatically aggregated to optimize writing speed to the hard drive.
also i recall there is a setting in properties->tasks, set socket buffers, you may try to increase it value (e.g. to 64000) to see whether it changes something. it can only influence the network part since as to file part the system has its own buffering so the data is automatically aggregated to optimize writing speed to the hard drive.
Kollinsb:
Update:
I said "So if I click update groups in NP, the graph (...) looks shabby, and after a minute is solid at 190"
I deleted the secondary server and now when I click to get new headers NP starts right out at 190KBs and stays solid. So something about adding a secondary server messes up throuput on getting headers. NP seemed very sweet as a single server newsreader.
Another thing - feeling good (like a fool)(grin) I changed NP's retention from 1-3 to 1-5. I then told it to get new headers. It looks like it is reading *every* header on the groups to check it . Been going solid for 10 minutes plus plus on my 7 groups (about 1.8M headers)
hth
Kristi
Update:
I said "So if I click update groups in NP, the graph (...) looks shabby, and after a minute is solid at 190"
I deleted the secondary server and now when I click to get new headers NP starts right out at 190KBs and stays solid. So something about adding a secondary server messes up throuput on getting headers. NP seemed very sweet as a single server newsreader.
Another thing - feeling good (like a fool)(grin) I changed NP's retention from 1-3 to 1-5. I then told it to get new headers. It looks like it is reading *every* header on the groups to check it . Been going solid for 10 minutes plus plus on my 7 groups (about 1.8M headers)
hth
Kristi
Shuttle AN35N Ultra, XP2600Bart(11.5x400), 512mb, GeF4Ti4200, WinXPproSP1.
TY for the update Kristi. I'm still kind of lost. I tried what Alex suggested buit I still seem to get about 75K/Sec. The one thing I didn't do after the changes Alex is create a new database. Is that a requirement? I guess my next question now would be "Does 4 threads at 600Kbit/Sec per thread equal out to around 75KBytes/Sec?" If not, I would think it would be an Astraweb issue. I do have the $10 a month unlimited. They have a $15/month DSL plan I could go to. Is that the route I should take? Like I said before, The speeds used to be good but now they aren't so I'm hoping it's a setting.
Thanks to you two for all your help!
Thanks to you two for all your help!
Hi, at the moment I am beginning to think that disk fragmentation may be a huge prob - I will have to check it frequently. It is made worse that I have been testing other newsreaders lots in the same partition. Just a thought for you.
re you - Given the choice, I would recreate the database. I deleted *everything* and re-installed NP. I don't know if there is an easier way - haven't checked. Only did it once. 1/2 of my prob was probably fragmentation.
I got brave and added Verizon server back in as a secondary. I only want to use it for fills, if something is missing on Astraweb. (it finished getting headers so I stopped NP and am defragging that partition (I use and love Raxco's PerfectDisk)
How long does it take your NP before the bandwidth degrades to 60? When I am doing group new headers, it will start at 190 full, but when it gets down to the last 2, or 1 (usually alt.binaries.svcd since it's the biggest of the groups I check), my graph could easily average only 60. Check Tasks, Task Manager, and then pick the tab of whatever you are doing (Articles or Headers, I presume) - if there's only 1 thread left, that's your reason. That is the one prob with Astraweb. Also check the "errors" tab to the right of each to assure that you don't have any errors.
Numbers FYI:
Edit / Properties / Task tab: Total 12, Easy 6,1,1 Hard 6,6,2 socket buffers receive=64000 per Alex
I could send you gifs of my properties if that would help. Remember, I'm a beginner at NP - these are just my guesses at the moment.
hth
Kristi
re you - Given the choice, I would recreate the database. I deleted *everything* and re-installed NP. I don't know if there is an easier way - haven't checked. Only did it once. 1/2 of my prob was probably fragmentation.
I got brave and added Verizon server back in as a secondary. I only want to use it for fills, if something is missing on Astraweb. (it finished getting headers so I stopped NP and am defragging that partition (I use and love Raxco's PerfectDisk)
How long does it take your NP before the bandwidth degrades to 60? When I am doing group new headers, it will start at 190 full, but when it gets down to the last 2, or 1 (usually alt.binaries.svcd since it's the biggest of the groups I check), my graph could easily average only 60. Check Tasks, Task Manager, and then pick the tab of whatever you are doing (Articles or Headers, I presume) - if there's only 1 thread left, that's your reason. That is the one prob with Astraweb. Also check the "errors" tab to the right of each to assure that you don't have any errors.
Numbers FYI:
Edit / Properties / Task tab: Total 12, Easy 6,1,1 Hard 6,6,2 socket buffers receive=64000 per Alex
I could send you gifs of my properties if that would help. Remember, I'm a beginner at NP - these are just my guesses at the moment.
hth
Kristi
Shuttle AN35N Ultra, XP2600Bart(11.5x400), 512mb, GeF4Ti4200, WinXPproSP1.
deleting database won't change bandwidth on article downloads, i'm not sure about fragmetation. the speed of your hard disk should be more or less the same as when copying a file.
check whether the speed cap depends on the number of connections you make to astraweb. when you have a stable speed from astraweb - try to download from some other server (e.g. isp) and see whether the overall speed increases. if it does, most probably it is the astraweb issue, then try to contact them about it, maybe the route to their server has a bottleneck or it is something with the server.
check whether the speed cap depends on the number of connections you make to astraweb. when you have a stable speed from astraweb - try to download from some other server (e.g. isp) and see whether the overall speed increases. if it does, most probably it is the astraweb issue, then try to contact them about it, maybe the route to their server has a bottleneck or it is something with the server.
an other thing you can do is to say to newspro how many headers it should take from a particular newsgroup.
so for example for alt.binaries.nl newsgroup:
click right on alt.binaries.nl newsgroup
go to "newsgroup properties"
go to "last headers per server"
You will see "default"
Change that by typing a number for example 100000
and press apply.
Then it will only take the last 100000.
You can practice and then decide to take 130000 for example if you want.
so for example for alt.binaries.nl newsgroup:
click right on alt.binaries.nl newsgroup
go to "newsgroup properties"
go to "last headers per server"
You will see "default"
Change that by typing a number for example 100000
and press apply.
Then it will only take the last 100000.
You can practice and then decide to take 130000 for example if you want.
Thanks.
I take them all cause I seem to be in there every day so a new header run for my 7 movie groups, on 2 servers, seems to take less than a minute.
My prob is that I don't notice something (pauses to start NP to finish downloading queue!) and so I have to go back a few days to get it. 1-3 days would work just fine if I would pay attention 
but I'll probably settle on 1-5. At that, all headers and 2 servers takes up only about 80mb of headers, which is fine. I do have to defrag it at least once a week, but I haven't quite gotten ahold of that - I'm playing with a dedicated NTSF partition at the moment. (my first NTSF ever!) In a couple of weeks I'll change that partition to a FAT32 and see if there's a diference. When it's just defragged, the database is fast!!! 
Kristi
I take them all cause I seem to be in there every day so a new header run for my 7 movie groups, on 2 servers, seems to take less than a minute.
My prob is that I don't notice something (pauses to start NP to finish downloading queue!) and so I have to go back a few days to get it. 1-3 days would work just fine if I would pay attention but I'll probably settle on 1-5. At that, all headers and 2 servers takes up only about 80mb of headers, which is fine. I do have to defrag it at least once a week, but I haven't quite gotten ahold of that - I'm playing with a dedicated NTSF partition at the moment. (my first NTSF ever!) In a couple of weeks I'll change that partition to a FAT32 and see if there's a diference. When it's just defragged, the database is fast!!! Kristi
Shuttle AN35N Ultra, XP2600Bart(11.5x400), 512mb, GeF4Ti4200, WinXPproSP1.
for defragging, goto www.sysinternals.com and download their contig.exe tool. It's a command-line tool that can defrag just one directory/subdirectory etc. You can schedule it to run every day via an AT job (start -> run -> cmd and type AT.exe /? for usage)
Newspro is essentially a giant database that is stored on the hard drive. The number one bottleneck will be that hard drive.
As far as hard drive interfaces go, IDE uses CPU cycles. If your processor is spiked because of doing various CPU intensive tasks (processing headers, for instance) and then you throw into the mix accessing an IDE hard drive, the overall speed will most likely be affected. I was running 3 large newspro instances on a PIII 550, with 1 gig of ram. BUT I had newspro on a 10k RPM SCSI drive. If I moved the newspro database over to a 7200 RPM ATA 100 IDE drive, performance was utterly pathetic.
Even with the SCSI drive, it still took quite some time to get all those headers processed when there are over 200k headers in a group.
If you're diligent in saving your article bodies out, you can get a 10k RPM 9 or 18 gig scsi drive relatively cheap nowadays. You'll be amazed at the performance difference in newspro.
Newspro is essentially a giant database that is stored on the hard drive. The number one bottleneck will be that hard drive.
As far as hard drive interfaces go, IDE uses CPU cycles. If your processor is spiked because of doing various CPU intensive tasks (processing headers, for instance) and then you throw into the mix accessing an IDE hard drive, the overall speed will most likely be affected. I was running 3 large newspro instances on a PIII 550, with 1 gig of ram. BUT I had newspro on a 10k RPM SCSI drive. If I moved the newspro database over to a 7200 RPM ATA 100 IDE drive, performance was utterly pathetic.
Even with the SCSI drive, it still took quite some time to get all those headers processed when there are over 200k headers in a group.
If you're diligent in saving your article bodies out, you can get a 10k RPM 9 or 18 gig scsi drive relatively cheap nowadays. You'll be amazed at the performance difference in newspro.