This chapter discusses Newt (acronym for News Tailor), a personalized news filtering system implemented based on the algorithms described in the preceding chapter. The architecture of Newt is described in detail followed by a discussion of some efficiency issues.
The system is essentially composed of three modules which encapsulate fairly distinct functions.
The database from which Newt retrieves articles is USENET. This is a collection of bulletin-board like systems (called ``newsgroups'') on the Internet. While most of the newsgroups are for discussion among users, there are a large number of newswire-like newsgroups. The basic difference between the two is that, while anybody can post an article to the discussion newsgroups, access to the latter is typically restricted to a few (often, only the administrators). Newt works much better with newswire-like newsgroups. This is for a variety of reasons. The first reason is that a keyword-based system depends on the fact that words are spelled correctly. Secondly, keyword-based inferencing is much more powerful if there is consistency in the usage of terms. The third reason is that Newt performs a computation-intensive analysis on the articles in the newsgroups being searched. Moderated newsgroups typically have fewer articles, thereby being more amenable to intensive analyses. Due to these reasons, the examples that have been presented in the following sections mostly involve moderated newsgroups carrying newswire articles.