Two-stage model for information filtering

Two-stage model for information filtering

This thesis presents a novel two-stage model that integrates the theories and techniques from the fields of information retrieval/filtering (IR/IF)and the fields of machine learning and data mining to provide more precise document filtering and retrieval. The first stage is topic filtering. The topic filtering stage is intended to minimize information mismatch by filtering out the most likely irrelevant information based on term-based profiles. Thus, only a relatively small amount of potentially highly relevant documents remain for document ranking. The second stage of the presented method uses pattern mining approach. The objective of the second stage is to solve the problem of information overload. The most likely relevant documents were assigned higher ranks by exploiting patterns in the pattern taxonomy. The second stage is precision oriented. Since relatively small amount of documents are involved at this stage, computational cost is markedly reduced, at the same time, with significant improved results. The new two-stage information filtering model has been evaluated by extensive experiments. The tests were based on well-known IR bench-marking processes, using the latest version of the Reuters dataset, namely Reuters Corpus Volume 1 (RCV1). The performance of the new model was compared with both of the term-based and data mining-based IF models. The results show that more effective and efficient information access has been achieved by combining the strength of information filtering and data mining method.

© 2008 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.