Post by thread:[OT] Filtering new style of SPAM

part 1 996 bytes content-type:text/plain; (decoded 7bit) I recently mentioned html based spam which is getting through my ISP's otherwise excellent filters. I have a simple method of trapping it. They code the words in html strings so that the text appears in vertical columns spread amongst n strings. There is a space between each letter. This is rebuilt on reception. eg Viagra Cialis Valium would be coded V C V i i a a a a g l i etc To add confusion they reorder the names in various emails. While complex filtering would catch this I have a simple method using Outlook Expresses standard message filters. Basically I generate 3 letter sequences of many combinations of the first letters of the various products arranged in many different ways. As these sequences, with spaces between the letters, are very rare in English, I expect very few false positives. So far it works 100% (small sample only). Method most easily understood by looking at the attached GIF. RM part 2 3992 bytes content-type:image/gif; (decode) part 3 35 bytes content-type:text/plain; charset="us-ascii" (decoded 7bit)

Russell wrote regarding '[OT] Filtering new style of SPAM' on Fri, Nov 11 at 01:27: > I recently mentioned html based spam which is getting through my ISP's > otherwise excellent filters. > I have a simple method of trapping it. > > They code the words in html strings so that the text appears in > vertical columns spread amongst n strings. There is a space between > each letter. This is rebuilt on reception. Speaking of spam filtering: I've switched from SpamAssassin to DSpam recently, and it works somehat like that - building up a list of tokens likely to be in spam and ham. It's sort of like a Bayesian filter, but it's more intelligent than most implementations I'm familiar with (it's not a 100% traditional byesian filter). For example, since this message came from a mailing list that I don't typically get spam from, this messages was marked as having a 1 in 110 probability of being spam - which means that there's a good chance it's spam, but it's probably not. This is in contrast to the other poster whose Bayesian filter improperly marked it as spam. :) Over the last week and a half, I've run about 7500 messages through my dspam install and it's had a 99.818% accuracy rate classifying spam and ham. Granted, I trained it with 18K spams and 28K hams from my personal mail archive (doesn't everyone keep 6 months worth of spam?), but I've heard reports of accuracy in the 80% range out of box. It can be set up as a proxy, but might be a little difficult to run on a windows box without cygwin - so it's not for everyone. The benefits are considerable, in that it's easy to keep up-to-date with "modern" spam and it's pretty scalable. The mail server that runs on handles something like 5000-8000 messages/day and it's just a 133MHz AMD 5x86 with 48MB RAM. The system never sees load averages over 10%, and stays below 1% most of the time (it was usually in the 10-15% range running SA). :) It's at http://dspam.nuclearelephant.com/ --Danny

> -----Original Message----- > From: .....piclist-bouncesKILLspam@spam@MIT.EDU On Behalf Of Danny Sauer > Sent: Friday, November 11, 2005 10:49 AM > > Speaking of spam filtering: Adding a shameless plug for my favorite. I've been using POPFile as my spam filter for 3 years now with excellent results. It's available for any platform the can run Perl and has support for many languages. There is also a Win32 specific easier to setup version. I reset my stats at the beginning of this year and with 14,643 messages classified the accuracy is 99.59%. (Note I don't run the PICList through POPFile) http://popfile.sourceforge.net/ Paul <snip> > > It's at http://dspam.nuclearelephant.com/ > > --Danny

Danny Sauer(piclistKILLspamdannysauer.com) is reported to have said: {Quote hidden}> Russell wrote regarding '[OT] Filtering new style of SPAM' on Fri, Nov 11 at 01:27: > > I recently mentioned html based spam which is getting through my ISP's > > otherwise excellent filters. > > I have a simple method of trapping it. > > > > They code the words in html strings so that the text appears in > > vertical columns spread amongst n strings. There is a space between > > each letter. This is rebuilt on reception. > > Speaking of spam filtering: > > I've switched from SpamAssassin to DSpam recently, and it works > somehat like that - building up a list of tokens likely to be in spam > and ham. It's sort of like a Bayesian filter, but it's more > intelligent than most implementations I'm familiar with (it's not a > 100% traditional byesian filter). For example, since this message > came from a mailing list that I don't typically get spam from, this > messages was marked as having a 1 in 110 probability of being spam - > which means that there's a good chance it's spam, but it's probably > not. This is in contrast to the other poster whose Bayesian filter > improperly marked it as spam. :) Over the last week and a half, I've > run about 7500 messages through my dspam install and it's had a > 99.818% accuracy rate classifying spam and ham. Granted, I trained it > with 18K spams and 28K hams from my personal mail archive (doesn't > everyone keep 6 months worth of spam?), but I've heard reports of > accuracy in the 80% range out of box. It can be set up as a proxy, > but might be a little difficult to run on a windows box without cygwin > - so it's not for everyone. The benefits are considerable, in that > it's easy to keep up-to-date with "modern" spam and it's pretty > scalable. The mail server that runs on handles something like > 5000-8000 messages/day and it's just a 133MHz AMD 5x86 with 48MB RAM. > The system never sees load averages over 10%, and stays below 1% most > of the time (it was usually in the 10-15% range running SA). :) > I used to use spamassassin but found, in my mind, a better solution. Rather then download the spam to have SA scan it, I switched to mailfilter. Using my 'spam library', it took a few days to get it to drop 99% of the spam I was receiving. My logs showed >100 spam messages per day back then. I now have to tweek once a month or so and average 1-2 spam msgs/week. Life in now much easier. <http://mailfilter.sourceforge.net> and a new one that has even more filtering options <http://murx.sourceforge.net> Wayne -- Pascal Users: To show respect for the 313th anniversary (tomorrow) of the death of Blaise Pascal, your programs will be run at half speed. _______________________________________________________

Russell wrote regarding 'Re: [OT] Filtering new style of SPAM' on Fri, Nov 11 at 21:23: > >You message was succesfully cataloged as spam by SPAMBAYES :) > > No doubt :-). > BUT probably on the plain text content which I included. It would be > interesting to see if it catches the original. Interesting note - DSpam on my system caught at least three of those messages with the vertical drug names in an HTML part (two had 6 drugs, one had 7) with a "1 in 333510 chance of being ham" on all of them this afternoon, wheras when it tagged the one from the list is was like 1 in 110 change of being spam. DSpam rocks. :) --Danny, noting that there are two projects called DSpam, and I have no experience with the one that's an Outlook plugin/proxy/whatever