Next: Conclusions Up: Testing the complete Previous: Experiment

Results

The results of the experiment are shown in figure . The graph reveals some interesting patterns. Recall typically changes only when the next generation is created. At other times, it tends to stay steady. Within a generation, precision tends to rise upto a certain point and levels off after that. Precision sometimes shows a slight dip when the next generation is created.

The behavior of the system recall can be explained as follows. A higher proportion of the relevant articles will be retrieved only if the profiles search all the right newsgroups i.e. newsgroups which contain the relevant articles. If the set of newsgroups being searched is limited, recall is constrained and tends to stay steady. The set of newsgroups being searched changes when the mutation operator is applied. The mutation operator is applied only when a new generation is created. If the newly introduced profiles contain the right newsgroups, recall will show sudden improvement (see the new generations created during sessions and in figure ). If the new profiles search newsgroups which do not contain any irrelevant documents, recall will not show much change in the new generation. The only exception to the preceding analysis is in the initial stages of the experiment where the profiles are almost empty and do not retrieve many relevant documents. In the extreme case when the profile is completely empty, it would not recommend any document for reading and the recall would be zero. Relevance feedback in the initial stages helps both precision and recall.

The behavior of system precision depends on relevance feedback. The feedback provided by the user helps the system to distinguish between relevant and irrelevant documents. In particular, relevance feedback helps the system to assign lower scores to irrelevant documents, thereby improving precision. This effect is similar to the specialization described in Section . When a new generation is created, the precision may suffer a bit. This is because the newly introduced profiles are likely to recommend some irrelevant documents. However, relevance feedback in the succeeding sessions helps to ``fine tune'' the new profiles. This restores the precision to its original level.

Thus, experimental evidence suggests that relevance feedback helps improve the overall precision of the system while genetic operators help improve overall recall.



Next: Conclusions Up: Testing the complete Previous: Experiment


MIT Media Lab - Autonomous Agents Group - agentmaster@media.mit.edu