How to Cach Up?

jaapf · Post by **jaapf** » Wed Nov 19, 2003 3:35 pm

Many newsreaders have a function called "CatchUp".

You use it after you've been away some time, and don't want to download all the headers you missed.
Especcialy if you using a Payserver that count's Headerdownload as used data, en does not allow Xpat, it's a very handy option.

Also after A crash (wich leaves you with a empty database) you don't have to download headers for the last 20 day's or so.
It would be great if it was possible to only download headers for the time you have put the retention for the newsgroup.

The way I see it now, Newspro does download, but immidiateliy discard headers that are older as the given retention.

Mayby I am missing something. If so, please inform me of that?

bruce73 · Post by **bruce73** » Thu Nov 20, 2003 2:46 pm

My understanding of retention is that, assuming you're starting with a new newsgroup or empty database, if you set it for, say, 1-2 days/delete the rest, NP will pull headers only for the past 48 hrs. and delete anything older than that, but leave those on the server. Then, in future pulls, headers on the local database that become older than 2 days will be deleted, but still be tracked locally for as long as is set in the "Track deleted/read..." option.

Or, to get as close to a catch up function as possible, use a low figure (500) in the "Last headers per server" option/delete the rest.

I could be wrong on this, but I think that's how it should work.

jaapf · Post by **jaapf** » Thu Nov 20, 2003 4:23 pm

bruce73 wrote:My understanding of retention is that, assuming you're starting with a new newsgroup or empty database, if you set it for, say, 1-2 days/delete the rest, NP will pull headers only for the past 48 hrs. and delete anything older than that, but leave those on the server. Then, in future pulls, headers on the local database that become older than 2 days will be deleted, but still be tracked locally for as long as is set in the "Track deleted/read..." option.

Nope. If you start with an empty database (or new group) and you put the retention on 1-2 instead of natural, NP WILL read ALL available headers, the only difference is that they won't be stored (they are discarded).

For the second part, you're right. NP will keep your database smaller that way.

I haven't tried your second option (last headers per servers) yet. I will when the occaison appears. I'm afraid this is something completely different though. The highest selectable number is 3000, the newgroup on mij payserver contains over 2.000.000 articles though. Which means I would only get the latest 3000 headers? instead of all 2.000.000?

blackdog56 · Post by **blackdog56** » Thu Nov 20, 2003 6:36 pm

You can type into both areas, last headers or retention and it will honor that. So you aren't restricted to what is selectable by default and can get the last 500,000 headers or get from 4 to 5 days ago with 4-5. If you want all 2,000,000 headers, just make it natural.

For the retention, I do think it reads all the headers to figure out which ones fall within your selected retention range and then gets those only.

bruce73 · Post by **bruce73** » Fri Nov 21, 2003 12:41 am

Nope. If you start with an empty database (or new group) and you put the retention on 1-2 instead of natural, NP WILL read ALL available headers, the only difference is that they won't be stored (they are discarded).

Yes, jaapf, NP will poll the server for all messages in order to filter out the ones that don't fall within the set retention range (which is what blackdog56 is also saying, I believe), and, agreed, this takes considerable time for large newsgroups. However, I don't believe the actual headers are downloaded to the local DB and then discarded, are they? Aren't they simply marked as deleted on the server for NP's purposes, so it doesn't have to start from scratch at the next pull of headers?

As for the "Get latest headers..." option (which I don't use, since, like you, I deal with large newsgroups), I only mentioned it as a thought to minimize the number of headers downloaded, in order to get caught up to date. On retrospect, I suppose, it wouldn't minimize the time necessary for NP to go through all of the headers on the server to establish which are, in fact, the latest headers. So, bottom line, perhaps a poor workaround for the catchup function.

[/b]

jaapf · Post by **jaapf** » Sun Nov 30, 2003 8:21 pm

Thx guys, I'll try it with the "get latest headers" options next time I have to catch up from a large newsgroup.

I know for sure that the "retention" option indeed DOES download the headers, because i can see the number of downloaded MB's for the newsgroup grow while no headers actually show up. The downloaded MB's are in the correct range for the headerdownload to be responsible.

bruce73 · Post by **bruce73** » Sun Nov 30, 2003 8:40 pm

I know for sure that the "retention" option indeed DOES download the headers, because i can see the number of downloaded MB's for the newsgroup grow while no headers actually show up. The downloaded MB's are in the correct range for the headerdownload to be responsible.

OK, that seems reasonable enough. I don't usually check those figures and so didn't think to look there. You make a valid point.

jaapf · Post by **jaapf** » Mon Dec 01, 2003 11:04 am

Ok, so the "Get latest Headers" option does the trick.

I first download ALL headers from my free ISP newsserver to get a reasonable idea how much Headers are posted each day. Then multiply that with the retention I need on my Payserver to download those missing files (with about 10% extra

), enter that value in the "Get latest Headers" box in the newsgroup proporties box en get the headers from my payserver.

This works fine, without downloading all the headers I don't want

Thx for your input guys