Next: Results Up: Experimental Results Previous: Experimental Results

Tests with real users

Right from the initial conceptual stages, the intention of this project was to go beyond a ``proof of concept''. The objective was to actually implement a system which people could use on a daily basis and would want to make it a part of their news-reading habits. This goal was adopted because the underlying system as well as the abstract concepts cannot be well conveyed without an implementation which makes the presentation much more concrete.

``Real'' user testing was, therefore, an important component of the performance evaluation. The project was advertised locally and nine volunteers were hired. Those who already spent a substantial amount of time reading Usenet newsgroups were preferred. Users were explained the various components of the system and were taught how to use it. They were then provided access to the system and were asked to use it on a daily basis. Since there wasn't sufficient time to port the code onto different Unix machines, the system just resided on the host machine. Users were asked to remotely login to the host computer and display the output on the remote monitor. This is possible with the network transparent X Window system. The filtering for each of the users was performed on the host machine.

The users were provided with an initial set of agents so that they had a reasonable starting point in the system and so that they could build other agents by analogy. They were particularly encouraged to construct newer agents suited to their interests. They were asked to read the news retrieved by the agents and provide a lot of feedback. In addition to this, the users were also asked to go through the newsgroups searched by the agents and find articles that the agent should have retrieved but did not. Thus, the users were expected to actively provide as much feedback as possible. As regards to the genetic operators, the users were explained the purpose and consequences of these operators and were given the freedom to decide if and when to use them.




Next: Results Up: Experimental Results Previous: Experimental Results


MIT Media Lab - Autonomous Agents Group - agentmaster@media.mit.edu