how to download headers starting at a specific date?

Post Reply
bassie
Posts: 490
Joined: Sat Jul 19, 2003 9:58 am
Contact:

how to download headers starting at a specific date?

Post by bassie »

is there a way to download headers for example starting 30 days ago?
I am a member of the FTD community and thus i know for example that then a movie is posted.

The problem is that giganews does not have xpat support, so I have to download the headers.
Another problem is that the downloading of the headers is very slow (i have an ISP subscription to giganews with 60 kilobytes per second) and then it might take some days for some newsgroups.
I know for example that a movie is posted 30 days ago, and i am only interested in that movie.
my other payserver supports xpat but does not have that many days on the server.

I know that there is an option to download only the last x numbers of headers (for example 100000 headers). But that does not help me much because i don't know how many headers i should download. It is much of a trial and error procedure: let's try 100000, no it is not enough, let's try 200000 etc,

Maybe it would be possible that the Newspro makes an estimation by dividing the total number of headers by the total amount of days.
(for example: "5 million headers divided by 50 days = hundred thousand, so for 30 days it means i will have to download 3 million headers")
Maybe newspro can "ask"the server only to download the headers of 30 days ago for example?

Does somebody have an idea?
alex
Posts: 4558
Joined: Thu Feb 27, 2003 5:57 pm

Post by alex »

there is no fast command to tell server to download headers from 30 days ago.

you can set retention (newsgroups->properties) to "30-" so it will download headers older than 30 days, but it invokes also xhdr command, still 60 kbytes per seconds may be be bearable (if it will be indeed 60 - on some servers xhdr is slow), since it doesn't download complete headers.

also you can guess the last headers by setting approximate number and then correcting it by dividing intervals in 2 in the right direction (like 100000, 140000, 120000, 110000 115000...) by changing last headers and invoking 'get new headers' and looking into what appears there.

when you see the right headers you just cancel get headers, already downloaded headers are not affected.

and the easiest way is to get an .nzb file but the free search service www.binaries.nl doesn't have such a retention, only commercial once.

ftd community these are people running some program to know about binary posts? probably they need to include in the program the ability to provide .nzb files of their posts.
bassie
Posts: 490
Joined: Sat Jul 19, 2003 9:58 am
Contact:

Post by bassie »

Alex, thanks for your answer.

I will study and practice this in detail when my downloadlist is finished.

Sometimes in FTD somebody (the poster of the spotter) also gives a link to a site where i can download an nzb file. Unfortunately after 30 days usually it is a dead link, and i get a message like "file not found ".

And indeed it would be nice if FTD would provide for nzb files, but maybe they don't want to compromise their legal status. I think they could be out of bussiness if they would give the nzb files directly.

And i indeed might consider a commerical nzb site subscription, because it usually does not cost much (something like 2 euro a month).

Thanks again for your help.
Post Reply