Page 1 of 1

Need a filter for garbage posts

Posted: Wed Mar 31, 2021 3:16 pm
by olav
Hi,
One of my favourite newsgroups - alt.binaries.e-book is being filled with garbage posts like
721ae6867a004949bfb9c77793125b82 [5/13] "721ae6867a004949bfb9c77793125b82.part4.rar"

Any given day has hundreds of them.

How to kill these with a filter, like if there are no blanks within the 30 first characters, then kill it...

regards
Olav

Re: Need a filter for garbage posts

Posted: Sat Apr 03, 2021 7:38 am
by alex
I'll check later today maybe they are filtered out automatically on the server and I need to update the client version then as the code is shared.

More likely it was already filtered out, but they started posting to additional newsgroups.

I'll let you know here later today.

Re: Need a filter for garbage posts

Posted: Tue Apr 06, 2021 1:10 pm
by alex
Try this version:

http://www.usenetexplorer.com/temp/ue571e32.zip

Someone needed those posts for search, so on the server side it blanks the poster, since the poster string is random, on the client side it was left as it is, in this version I added the call to the built in filter for the client to filter out those posts as part of the built-in filter.

Edit menu->Kill Filters->Built in kill filter should be checked

The filter applies only to newly downloaded headers

Re: Need a filter for garbage posts

Posted: Thu Apr 15, 2021 10:21 am
by alex
It is likely to be the final solution:

http://www.usenetexplorer.com/temp/ue571h32.zip

It also contains a more flexible filter which I can update dynamically.

Re: Need a filter for garbage posts

Posted: Sat Apr 24, 2021 9:05 pm
by arthropod
Alex, you move too fast for me. By the time I get some documentation together, there's a new iteration to test.
Brief summary: these random headers are flooding almost every group I monitor. There are two problems. 1. Filtering the headers out and 2. Not accidentally taking out headers of interest.

I'm still trying to decide if the new UE version is addressing #1 sufficiently. That will take more time.

On #2, however, I have a newsgroup alt.binaries.mp3.abooks which is one of the few groups NOT to be hit by the header floods. Instead, posts in that group have all but disappeared. I suspect the built-in filters are taking them out. I don't know of a way to compare headers I receive with the built-in filters on versus off for that group so I can identify what is/is not happening.

Re: Need a filter for garbage posts

Posted: Sun Apr 25, 2021 5:04 pm
by alex
Try this version:

http://www.usenetexplorer.com/temp/ue58b32.zip

I've added the log option in edit menu->properties->kill filters, built in filter. It needs to be enabled on every run as it may write a lot.

You can access the log file fast edit menu->properties->general, database folder, click "Open" and then click on log.txt

I downloaded last 5M headers, what I see it removed 44K single messages posted on January 19 which are parts with randomized subject and author spread over many newsgroups.

But if you will see something wrong let me know.

I'll add it in the next version, I had a thought about it since I have such log option on the server, but it didn't make it through.