Bloqer Internet Filter

The Efficiency of Keyword Filtering

Keyword filtering differs from the normal approach used by Internet filtering software. What most internet filtering software products do is compile a huge list of internet pages and then categorize them. Then, they proxy the traffic through their own network, they compare the request to their huge list of sites (which are generally cached) and then they decide whether to block or not, based on the rules set up by the user. The basic issue with this method is that it is conceptually flawed in a couple of ways. The first is that there are millions of web pages built every day. So their database and cache needs to keep growing exponentially, and to infinity. And it can’t possibly include every page that exists. The second inherent flaw that it is generally overkill. If you look at people’s actual web surfing habits, they visit a handful of sites on a regular basis. Let’s say most people visit 10 domains per day. Also, most people visit the most popular sites. Facebook, Twitter, Google, Yahoo, MySpace, Baidu, etc. Maybe 100 sites that are mostly visited, and then the top 100,000 sites get the lions share of the visits. This cuts down the number of actual sites to block by a large factor. Which leads me to the original topic, keyword blocking. For the purpose of keyword blocking, let’s use pornography blocking as target category. Generally speaking, the top 1000 porn sites are going to contain about 100 or so common keywords. I won’t mention them, it’s embarrassing, and we know what those words are. These keywords are contained in the URL structure, in the meta keywords, and in the page titles. So, if you block the 100 top keywords in the three main areas they are contained, you are going to cut out porn pages in the high 90 percentile range. Yes, the most dedicated, driven, and enthusiastic porn surfers may find the odd, obscure site which does not contain a porn keyword. But the task is made much more difficult if they are searching on a smart phone. So the point of this discussion is that keyword blocking is as effective as proxy/list blocking, and it removes the necessity of compiling the infinite list of every website. A much more efficient way of blocking, and also removing the need to proxy to an outside cache.

Join the forum discussion on this post