Usenet Explorer

Im not sure if it is connected but when i had unpack tasks limit set to 500KB and save incompletes partials activated in watch\auto headers i noticed this:
Stuff gets autodownloaded, not all articles are there yet, UE does constant scanning which is OK but the speed drops to this very 500+KB very often. When i remove the limit it bumps to 10+MB while scanning all the way down to the end

So i was wondering if anyone else had this.

And if it really happens(i tested it only one time) then is it a normal behavior?.
I presumed that this limiter was to be applied only while unpacking\repairing.
COuld it be that extensive scanning of partials is perceived by UE as "unpacking\reparing"?

cheers

Scanning is done by unpack to find valid data blocks, if you set unpack task bandwidth limit - technically it is applied, although maybe it could be made an exception.

I see
Im glad and sad at the same time that i guessed it right

Is it technically possible somehow to make a destinction between partial scanning and unraring\repairing of the whole set ?

TIA

I'll look at the code.

Maybe what you are asking for doesn't make sense?

Bandwidth limit during unpack tasks is to let UE to maximize hard drive usage for the unpack tasks by limiting load on hard drive from download tasks.

For scanning - the bottleneck is the hard drive (unless the file is still mainly cached), it means UE will max hard drive by scanning only.

Using unpack task bandwidth only makes sense when connection speed is comparable with hard drive speed (say at least 1/4), thrashing is then much more serious issue than losses on connection slowdown and subsequent acceleration, when using many tasks connection recovers quite fast (at least it is a short process comparing to constant disk thrashing).

With such fast connection scanning will also likely to slow download and it will be also slow by itself because of download tasks disk writes, even when unpack task bandwidth limit is not set (so you limit the bandwidth to end scanning with its hard drive maxing faster).

I think thrashing resulting from max read speed (scanning) running concurrently with relatively random disk writes (download) will have more negative effect as TCP multiple connection speed drop/recovery.

You can make a test compare the time from the start of the download until the start of repair with the bandwidth unlimited and limited if there is a noticeable difference which is faster, even if about the same time on safe side better to leave it as it is to avoid the risk of thrasing.