UE is saving incompletes

jungario
Posts: 6
Joined: Wed Jan 16, 2008 5:19 am

UE is saving incompletes

Post by jungario »

HI,
Not sure which settings to put, but UE is saving the articles before they reach 100% some do reach it but the most of the articles don't.
I am using multiple servers.
Any help?
Thanks
jaapf
Posts: 203
Joined: Thu Sep 11, 2003 3:06 pm
Contact:

Post by jaapf »

Find a server that does hold all parts?
Dutch? Visit the Dutch UE/Newspro forum at: http://www.binaries4all.nl
Nederlandse UE handleiding op http://www.binaries4all.nl/ue
English UE tutorial online at http://www.binaries4all.com/ue/
Handy links at: http://jpfx.zapto.org/
Josef K
Posts: 534
Joined: Thu Feb 27, 2003 7:29 pm

Post by Josef K »

Compare the age of the posts with the retention of your servers. If you are downloading articles close to the end of the retention they may be appearing as 'No such article' to UE and it will take whatever course of action you have specified in the settings.

Look at Properties->Articles->Save Queue and see if you have 'Ignore 'No such article' errors' checked. If it is, whenever you download something that has some segments available but not all, UE will save the articles anyway. This is designed so that things like PAR2 can work (you can make use of the partially corrupted files). If you uncheck this setting, the articles will stay in UE until you manually save/delete them. However, if the articles are out of retention you will never be able to complete them. It depends on whether or not you download things that mostly have a corresponding PAR set which setting you use for this.
jungario
Posts: 6
Joined: Wed Jan 16, 2008 5:19 am

Post by jungario »

Thanks Josef k
The post are same day posts, I unchecked ignore no such articles and along with retry failed every 1 minute ,it did the trick the article kept coming back till it completed.
I do have another question, the main reason I am trying UE is because its support for multiple servers and combining their speeds which is great something grabit wihich I've been using fails to do simultaneously, however, I notice when the number of articles goes down so does the speed, dramatically and often comes to 0 kb/s, this never happened with grabit, is there another setting I should do in this case? or is it non related.
Thanks
jungario
Posts: 6
Joined: Wed Jan 16, 2008 5:19 am

Post by jungario »

Thanks Josef k
The post are same day posts, I unchecked ignore no such articles and along with retry failed every 1 minute ,it did the trick the article kept coming back till it completed.
I do have another question, the main reason I am trying UE is because its support for multiple servers and combining their speeds which is great something grabit wihich I've been using fails to do simultaneously, however, I notice when the number of articles goes down so does the speed, dramatically and often comes to 0 kb/s, this never happened with grabit, is there another setting I should do in this case? or is it non related.
Thanks
alex
Posts: 4514
Joined: Thu Feb 27, 2003 5:57 pm

Post by alex »

If you mean when the number of tasks drops (the running task pane in the task manager - the green computer icon pane to the right of the "articles" pane) the speed may go down initially and stay lower if you get slower speed when the number of connections is smaller, depending on what speed your server can provide.

The speed reading is drawn over short period of time (I think about 3 to 6 seconds) so you may see more exact speed readings, but in the end they just reflect what is going on, if the server can max your bandiwdth on a single connection you won't see prolonged speed drop, the speed will pick up shortly after the number of tasks got less, it is all tcp/ip lower level logic which no newsreader can change.
jungario
Posts: 6
Joined: Wed Jan 16, 2008 5:19 am

Post by jungario »

Thanks Alex,
It is exactly what is happening, only thing is the server that picks up the last few tasks is the slowest one for some reason, as if the fastest servers refuse to indulge on fewer tasks if i do add more tasks they suddenly wake up and speeds get back, I guess as you said tcp/ip logic.
alex
Posts: 4514
Joined: Thu Feb 27, 2003 5:57 pm

Post by alex »

if some tasks are hanging it could be server or firewall malfunction if they disappear after timeout it is more likely server, then if there are other tasks they go ahead so in the end those hanging tasks could be left with zero bandwidth, then after they time out with the next retry they are downloaded. nothing can be done about it, you could try to check with the server support what it might be if it is on their side.

if you have several servers and tasks hanging only on one server in the end you could identify the server, if you have a slower server it is likely that the server tasks will finish the last.
jungario
Posts: 6
Joined: Wed Jan 16, 2008 5:19 am

Post by jungario »

Thanks for the clarification,now that I resolved (or seem to )my main issue which is the incompletes, I will just have more patience waiting on the last taks because eventually they will finish.
As for the loss of the connection over the http protocol when I was trying to augment the default total tasks of 40 I guess because the brandwidth limit was reached, or the routeur wasn't able to handle the flood and hung up, may be not, because UE was still downloading.This issue seems to be resolved when I put the tasks back to 40.
All in all a pleasant experience with the capacity of this software easy enough when you get familliar with it .
alex
Posts: 4514
Joined: Thu Feb 27, 2003 5:57 pm

Post by alex »

to save the most resources it is more effective to keep the number of tasks towards the lower end, just somewhat higher than the number which ensures the bandwidth is reliably maxed.

as to UE no problem to handle more tasks simultaneously, you save mostly on overhead because of fewer operating system context switches and maybe less load on network lower protocol levels/hardware, i'm not sure how much though if at all. as to context switches also you don't save much since network tasks almost don't consume processor time, most time they are sleeping/waiting for input.
jungario
Posts: 6
Joined: Wed Jan 16, 2008 5:19 am

Post by jungario »

Great support, and fast responses, thanks again.
Time for me to call it a night.
xtrips
Posts: 28
Joined: Mon Mar 10, 2003 3:37 pm

Same symptom, different cause!

Post by xtrips »

Hello,

I also rarely encounter the problem of incomplete saved files.
But the prolem I am refering to is occuring when for some reason my connection to the net is reset (power surge or whatever).
UE stalls on a file, then when the connection is restored, it resumes to download. But invariably when this happens the file is corrupted, incomplete.
Why is it so? It seems that UE doen't know how to resume a download correctly.

Thanks :cry:
Josef K
Posts: 534
Joined: Thu Feb 27, 2003 7:29 pm

Re: Same symptom, different cause!

Post by Josef K »

If you lose your connection, UE will begin the download of the segment that was in progress as the connection was terminated. Therefore, UE would not save an incomplete segment to disk until it was downloaded 100%.
xtrips wrote:...connection to the net is reset (power surge or whatever).
A power surge is more likely to cause hardware damage unless you have surge protection in place. If your computer were to crash, then the data may have been partially saved, in which case UE may think it's downloaded fully. Whether or not UE writes each segment's data to disk during download or flushes it from memory after it's completed I don't know, so I can't give a definitive answer.

Which is it: your connection dropping or computer crashing and why do you say power surge?
xtrips
Posts: 28
Joined: Mon Mar 10, 2003 3:37 pm

Re: Same symptom, different cause!

Post by xtrips »

Josef K wrote:If you lose your connection, UE will begin the download of the segment that was in progress as the connection was terminated. Therefore, UE would not save an incomplete segment to disk until it was downloaded 100%.
Wrong. Sorry. What I described is what happens actually.
Josef K wrote: Which is it: your connection dropping or computer crashing and why do you say power surge?
But here, you are right. My mistake. I didn't precise that the failures I refered to didn't apply to the computer itself, only to the adsl modem, therefore the connection.
Josef K
Posts: 534
Joined: Thu Feb 27, 2003 7:29 pm

Re: Same symptom, different cause!

Post by Josef K »

xtrips wrote:Wrong. Sorry. What I described is what happens actually.
In which case it would be a matter for Alex to look into. UE has always (personally speaking) retried any articles that failed after a dropped connection (though I very rarely have had those except for network timeouts). There may be some other unforeseen reason why it isn't working as expected for you.

Maybe you can simulate this again so you have something reproducible. Do a search for or import any file with a large segment size and a corresponding PAR file. Preferably choose a very large single segment file to be sure you're dealing with exactly the same part each time. Download it a couple of times and verify it. If it verified OK, then download it again a couple of times but while it's downloading, literally pull the plug on it and then reconnect. UE should time out and resume downloading to complete the article. Verify the file and see if corruption still occurs.
Post Reply