​WSDM Workshop 2015.

User's examination behavior study on Web and vertical search

 

Min Zhang, Tsinghua University

 

User’s examination of search results is a key concept involved in all the click models. However, most studies assumed that eye fixation means examination and no further study has been carried out to better understand user’s examination behavior. In this talk, we will introduce our observations on the user’s results examination behavior via eye-tracking, mouse-movement and click-through data. To our surprise, a large proportion (45.8%) of the results fixated by users are not recognized as being "read". Looking into the tracking data, we found that before the user actually “reads” the result, there is often a “skimming” step in which the user quickly looks at the result without reading it. We thus proposed a two-stage examination model which composes of a first "from skimming to reading" stage (Stage 1) and a second "from reading to clicking" stage (Stage 2). We found that the biases (e.g. position bias, domain bias, attractiveness bias) considered in many studies impact in different ways in Stage 1 and Stage 2, which suggests that users make judgments according to different signals in different stages. We also show that the two-stage examination behaviors can be predicted with mouse movement behavior, which can be collected at large scale. Relevance estimation with the two-stage examination model also outperforms that with a single-stage examination model. Our recent study shows that the two-stage examination behavior also exists in vertical search, which will also be shared in this talk. We believe that user’s examination of search results is a complex cognitive process that needs to be investigated in greater depth and this may have a significant impact on Web search.