- An Experimental Study of Relevance Judgments in Literature Searching Process
- No.12, p.259-274
In the problems of literature searching process, the cases in which question contents cannot match with index terms are observed, and the author supposed some matching levels in this process. Since they were always related to relevance judgment by searchers, the relevance was especially examined. It was insisted that the judgment level of objective relevance was not observed in users having questions, but in searchers.
Examinations were carried out experimentally on librarians, students and researchers in medicine to examine whether judgment of relevant documents was influenced by difference of subject speciality (knowledge). Search questions on basic and clinical medicine were determined, and collected data were analyzed by variance analysis.
As the result, there was no significant difference among the judgment groups in the first test on thymus study. In the second test on cerebrovascular accidents study, however, a significant difference was observed. The results on the second test seems to be due to the problems in researcher group. However, further analysis revealed that judgment was rather correlated to each other among groups, and that judgment of student group was correlated more closely to that of researcher group, than judgment of librarian group was. This seemed to indicate the difference of the amount of subject knowledge. The examinations on the difference of the type of question in basic and clinical medicine using post-questionnaires, revealed that the judgment on clinical study was thought to be easier than basic study by the subjective groups, while judgment was better agreed among groups on basic study than on clinical study. Therefore, it was concluded that librarians show more similar judgment pattern to researchers and students when questions had more limited range.
Finally, the necessity of effort to aiming at more effective service through basic study of searching process like this was pointed out.