tune-up

Looking at stats in several BlogEngine.Net 1.6 based sites, it seems like Akismet scores around 80% in accuracy on average, killing some 8 out of 10 spam comments. Which is pretty good but not outstanding. "Stop forum spam" filter blocks may be another 5%. Still, quite a lot of spam comments get through the filters. You can eliminate most of automated spam by adding captcha to your site, but it does require some basic skills and time. May be, given a surge in recent spam bots activity, it is time to include captcha to the main distribution out of the box? Also, number of enhancements to the current comment management have been suggested in the forum and few bugs found that need to be fixed. I think I might put together a little patch that improves and polishes comment management. Here is my to-do list, if you see good fit - feel free to add your suggestion.

  • Integrate captcha control with option to enable/disable.
  • Do not generate email if added comment marked as spam
  • Spammers IP automatically added to the filter with "delete" action (black list)
  • If admin restores comment, IP and email added to the filter (white list)
  • Add option to show/hide pingbacks and trackbacks in the comments list
  • Fix comment counter in the post footer (spam doesn’t count)
  • Fix spam messing recent comments widget #
  • Add "Delete all spam" button
  • Disable action buttons when grid filter applied
  • Make "author" field editable
  • Integrate with intense debate or disqus to provide an option to outsource comments all together?
Share/Save/Bookmark
Signature
custom

Spam comments are annoying and sometimes looking at comment allowed by Akismet or some other anti-spam service you think – I’m sure I would do better. It is just so hard to get the plumbing in or I would made my very own solution. With BlogEngine.Net 1.6 it is not, you can easily implement your own anti-spam filter or add existing if you like it more than built-in Akismet. Here is how.

First, open BE 1.6 solution in VS or VWD or whichever tool you are using to work with source code in .NET. Create new class in the App_Code folder, I called mine SpamBlocker. This class should implement ICustomFilter interface, it has just 4 members and looks like this:

namespace BlogEngine.Core
{
    public interface ICustomFilter
    {
        bool Initialize();
        bool Check(Comment comment);
        void Report(Comment comment);
        bool FallThrough { get; }
    }
}

Below is very simple implementation. It based on idea that most reliable way to identify spam is to check if comment contains link to site selling products or services and known for hiring spammers to push their ads all over the internet.

using System.Collections.Specialized;
using BlogEngine.Core;

public class SpamBlocker : ICustomFilter
{
    private static bool _passThrough = true;

    static StringCollection MockService()
    {
        StringCollection spammers = new StringCollection();

        spammers.Add("conference-call-providers.co.uk");
        spammers.Add("telebisyonserye.info");
        spammers.Add("qmw.com.au");

        return spammers;
    }

    public bool Initialize()
    {
        if(MockService().Count > 0) return true;
        return false;
    }

    public bool Check(Comment comment)
    {
        bool spam = false;

        foreach (string s in MockService())
        {
            if(comment.Content.Contains(s))
            {
                spam = true;
                break;
            }
        }

        _passThrough = (spam) ? false : true;
        return spam;
    }

    public void Report(Comment comment)
    {
        if(comment.IsApproved)
        {
            foreach (string s in MockService())
            {
                if(comment.Content.Contains(s))
                {
                    MockService().Remove(s);
                }
            }
        }
    }

    public bool FallThrough
    {
        get { return _passThrough; }
    }
}

Mock service has few sites that spammers promote. Obviously, you would have to put some more work here for real deal, but for now it will do. All we need is a way to identify spam. For the sake of this example, if comment mentions any of these sites – it is spam.

Now lets go over each method and explore what it does.

Initialize() – here you can verify that service is running, do authentication or any other kind of required verification. If return true, that means you are ready to handle comments. We just check if we have any spam sites in the list, if we do – good to go.

Check(Comment comment) – this is where you do your custom thing checking if comment is spam. BlogEngine will pass comment to this method, so you can examine all it’s properties. For simplicity, we check if comment has reference to any of spam sites in the list. If it has, we return “true” meaning that comment is spam and set fall through to false (more on it later).

Report(Comment comment) – this is a way to provide feedback to the service, so that you can make it smarter and identify spam better. If blogger restores comment marked by SpamBlocker as spam, we remove it from the list of spam sites. In the real world, here you can examine comment and try to figure out why you did a mistake marking it as spam or verifying that it is not spam and try to put more smarts in your service/filter.

FallThrough – blog might run several custom filters, and we need a way for them to play well together. When fall through set to true, you saying “I think it is spam/not spam, but I’m ok with others to check on it and provide second opinion”. In this case, last judge wins – so the order in which you set filters to run (priority) is important.

filters spamfolder

Here we pretty sure that when comment has spam site in it – it is spam. So we set fall through to false, telling BlogEngine that, in this case, we want to be the ultimate judge and don’t pass comment to other custom services down the list.

If you go and make a comment with one of those sites in the comment body, it will be rejected by SpamBlocker. So here you have it – your very own custom anti-spam filter in 60 lines of code. Ok, it’ll be much more lines when you done with custom logic to identify spam, and you might even implement web service and make it available to other BlogEngine users for this. But handle it on BlogEngien side is a snap.

One more thing, if you want blogger to be able turn your service on and off, you can easily do so by implementing it as extension, just the way Akismet is implemented in BlogEngine. Then, blogger can turn it off without removing it from the web site all together. I hope you’ll try it for yourself, and good luck fighting those nasty spam comments.

Share/Save/Bookmark
Signature

be16 After few delays new version of BlogEngine.Net has been released. Although this release is mostly incremental with minimum breaking changes, there are quite a few improvements, enhancements and bug fixes to make it worthwhile for those stuck on previous versions to upgrade. Upgrade instructions can be found here, and you can download new version from the project site on CodePlex.

  • Centralized Comment Management
  • Automated Comment Spam Filtering with ability to plug-in custom Filtering modules
  • Multiple Widget Zones (details)
  • Referrers data and Blogroll items now stored in Database when using the DB blog provider.
  • Unsubscribe Link in Comment Notification Emails
  • Referrer Data can be Stored for more than 7 days.
  • Blogroll items can now be Ordered.
  • Newsletter Widget more Intelligent - Emails sent when a post is going from an Unpublished to Published state.
  • Twitter Widget - New options and improvements
  • Page Slugs now saved in Database.
  • New Logging system to Track events and errors.
  • Unhandled Exception Handling
  • Fixes to Comment Notification Emails not being sent out correctly in some cases.
  • Outgoing Email improvements
  • Many other improvements and fixes.
Share/Save/Bookmark
Signature
vote

resultsOne real surprise for me was that not many people use BlogEngine as their sandbox and playground. To me as a developer from the beginning it was mostly a toy and only over time it grew up into something bigger with responsibilities attached, so I’m sort of impressed. It seems like what most folks really want is a stable mature product with great features and scalability built it. There is understandable desire for cool new tech on part of some but most seems really care about how it serve their blogging needs (ye, I know, shocking…). So in the end it looks to me like loud and clear call to power along with demand for more advanced and polished tools and features.

Share/Save/Bookmark
Signature

mobile Largely as a result of iPhone revolution, mobile devices today more wide spread and connected to the web than ever. It is increasingly important for web sites to look good on the small screen to attract mobile users. BlogEngine.Net has mobile theme that looks ok, but we need more and better. I’ve been looking for options and played a little with some of the frameworks available for other platforms. There are some interesting development, and I’m interested in doing more here. As first experiment, I updated my previous iPhone admin add-on and combined it with simple but fully functional mobile theme. It got a list of categories on front page, from where you navigate to the posts. You can also login and publish posts using admin add-on that became part of the theme. Take a look at screen shots, they are pretty self-explanatory. More...

Share/Save/Bookmark
Signature

This poll is not official and may be a little early, but it will provide BlogEngine team with some ideas on where we all, as a community, want to move. Are we looking for Wordpress.net? NewCrazyCool.net? Something in between? Please share your opinions and thoughts, it's all greatly appreciated. More we know, easier it is to make decisions and change gears. You can vote either using poll in the side bar or following the link. Feel free to drop a comment if you think of other options or just want to let everybody know what's on your mind. Don't be too harsh though, it still Christmas ;)

Share/Save/Bookmark
Signature

mailr Thanks to the magic of Metaweblog API and BlogMailr service, you can email posts to BlogEngine just as easily as publishing them from Windows Live Writer, Word or iPhone.  Here is what you need to do. Open personal account with BlogMailr, it only takes few minutes and it is free for personal use. Once your account setup you'll be able to add blogs to it. You have to provide user id and password so that BlogMailr can auto-login and publish posts on your blog. More...

Share/Save/Bookmark
Signature

SharpBtn My previous post was about awesome code syntax highlighter JS library I have converted into extension for BlogEngine and no so awesome button I’ve added to TinyMCE to insert code snippets. Let’s make it better by building custom plugin that will let us select language and enter actual code in the pop-up window eliminating messing up with TinyMCE editor itself. More...

Share/Save/Bookmark
Signature

logo

There are several implementations of this extension for BlogEngine, but I wanted it to work with latest JS library and be more configurable. And also I wanted it play well with TinyMCE - yes, it might come as a shock but some people don't use Windows Live Writer. I don't know why... So, if you go through usual routine downloading and moving files to corresponding location (you’ll need to overwrite couple) this is what you’ll get. More...

Share/Save/Bookmark
Signature

Admen2

Some people complain about Akismet not killing spam as effectively as they would like or hope. I think Akismet doing fine job identifying spam bots, but recently a lot of it coming from real people trying to make a little extra money, and those are challenging to deal with. Sometimes I get totally valid comment and the only odd thing about it is that guy’s name is “buy cheap watches” or something like it. It is tough to filter out programmatically. Probably, the best solution for a moment is similar counter-move when people tired of spam come together and create list of known spammers so that spam blocking applications can use it. More...

Share/Save/Bookmark
Signature
<<  March 2010  >>
SuMoTuWeThFrSa
28123456
78910111213
14151617181920
21222324252627
28293031123
45678910
Enhanced with Snapshots

Subscribe to Rtur.net