Pending server upgrade

03 Oct 23: One server was relocated, the server currently doesn't have a public IP address since in the meantime it is running on starlink. I wrote some additional tunnel code running separately to handle this. When the server is engaged the connection is 5600+ days uniform (since around 24 July 2008). In the unlikely case I disengaged the server because of some problem the retention is 1200-3800 days depending on newsgroup. If you experience any issue please let me know.

29 Nov 23: Because of encrypted and obfuscated posts flood mass posted by a few nzb indexing websites using usenet servers as private storage for their members to download the posts - the situation with the content is pretty chaotic since their purpose is to post in such a way, so users must use their website exclusively, when the website disappears - the encrypted posts just eat usenet providers' hard disk space uselessly. If you can't find something specific please let me know at alexbirj at gmail dot com what exactly you can't find for me to check how it is possible to retain the posts. Non-obfuscated posts shouldn't be affected at all, let me know if you notice anything missing.

29 Nov 23: The search protocol had to be upgraded to extend 3 byte (16M+) limit on the server side for number of sets per instance, so at least the version 5.8.3 is needed to access it, otherwise should be no any difference.
Post Reply
alex
Posts: 4514
Joined: Thu Feb 27, 2003 5:57 pm

Pending server upgrade

Post by alex »

If all goes well in 7-10 days we will have a server with twice more RAM.

To fully utilize all available hardware we would need a client update which only will be possible after the next major release, since it is more time consuming to make changes in the current and future versions at the same time (I already got once some work accidentally deleted by swapping project directories).
alex
Posts: 4514
Joined: Thu Feb 27, 2003 5:57 pm

Post by alex »

There were some delays with hardware, now we have the components but due to a pending vacation (not mine) the computer will be assembled in the end of the next week.

I need to make a small adjustment in the software, but it is also possible I'll go for an advanced search service design and use the new platform to test it while keeping the old server as reserve. Another option I'll just run the new server with a higher retention and reuse the old server for tests, how I will proceed depends whether I'll finish with par2 option by the time the server is assembled or not.
alex
Posts: 4514
Joined: Thu Feb 27, 2003 5:57 pm

Post by alex »

I've successfully removed some periodic heavy disk/RAM operation which was a kind of impediment for pending server upgrade plan, I've just restarted the server to engage the change.

Given this I could increase the retention on the current server, but right now the server runs absolutely unattended because of vacations, so I'll be keeping it at the current level (at least I won't decrease it which otherwise would be necessary to provide more safe ground).

It is likely we'll have the client upgrade soon, I'll try to do it even before the hardware upgrade will be available (the focus will be on adding additional brains to the client, it is the optimal way to go from the point of speed and minimizing bandwidth). If all is ok after the hardware upgrade I'll be able to provide maximal hardware utilization in the matter of one or two days.

As the part of the upgrade I'm planning to replace the search server address with web page address which will contain search server name/port, it is necessary as a part of the new design.

The new design won't require server software change except for a simple protocol update, so the server is ok as it is now and it is good since it works flawlessly.
alex
Posts: 4514
Joined: Thu Feb 27, 2003 5:57 pm

Post by alex »

I'll try to implement new service design in v1.9.7 shortly, just it is more easy to do now, the code right now is well isolated so I can easily make changes on the current working state and update v1.9.6 code in parallel. Later some code will be changed so it won't be possible any more to make such changes easily.
alex
Posts: 4514
Joined: Thu Feb 27, 2003 5:57 pm

Post by alex »

the communication core code completed, what is left are few other details, maybe all will be finished tomorrow.

i also made small adjustment to the server code from July, 14, restarted it.

so in contrary to my suggestion in the beginning it looks like we'll have the indexing client code update first.
alex
Posts: 4514
Joined: Thu Feb 27, 2003 5:57 pm

Post by alex »

the work has been completed, i'll recheck and probably publish it tomorrow.

there is almost no change from the interface point of view, only the search service protocol/host/port settings have been replaced with the reset "panic" button (the process is fully automatic). internally though it is now more sofisticated.

since there is no server database change we'll run it first on the current database, when based on what hardware we'll have i'll make use of it, in short all now will be client and server ready.

i masked unfinished par2 options in the latest working code rather than applying all the changes to v1.9.6 code which was derived from earlier code (the spawning had place around v1.9.1), so it is a single project again.
alex
Posts: 4514
Joined: Thu Feb 27, 2003 5:57 pm

Post by alex »

it is released as v1.9.7 (software part)
alex
Posts: 4514
Joined: Thu Feb 27, 2003 5:57 pm

Post by alex »

i've connected the new server. for picture groups (group names containing "pict") i'm still using the old server, too much spam there, maybe i'll check the possibility to filter it out but not right now. the old server - despite returning picture group results only - is gathering headers for all newsgroups so i can switch back to it if i need to reconfigure something.

as to the new server groups i set the retention to 130 days, there is still some reserve left, the reserve means also fewer restarts so better to keep it this way, or i'll use it i don't know.

we'll check whether there will be requests from users regarding managing their firewalls so the need to add manual options to load/show search service configuration, right now i don't have any consistent user requests about that, so i don't see a need to add the options.
alex
Posts: 4514
Joined: Thu Feb 27, 2003 5:57 pm

Post by alex »

it seems all is working ok.
alex
Posts: 4514
Joined: Thu Feb 27, 2003 5:57 pm

Post by alex »

i restarted it with 150 days retention, in short with this configuration probably i'll keep it between 140 and 150 days in the terms of current usenet volume, i'll take a break from this stupid thing in the meantime.
Post Reply