How Hearing Impaired People View Closed Captions of TV Commercials Measured By Eye-Tracking Device

We conduct experiments to measure effects of various types of closed captions for Japanese TV commercials using eye-tracking device. The experiment materials are real TV commercials, and 92 hearing impaired people participate in the experiments. We prepare six TV commercials, each of them captioned in four different ways: no closed captions and three types of captions, which are fully captioned (horizontal), partially captioned (horizontal), and fully captioned vertically. The participants watch them in silence using eye-tracking device, accompanied by a questionnaire asking about caption readability for two CMs. We define Areas of Interest (AOIs) as the caption lines and record fixation counts in AOIs and visit counts to AOIs. The results show that there are no statistically significant differences among the three types of captions on both fixation and visit counts, which means the participants watch the CMs similarly. However, further analyses of the data suggest some characteristics of the way the hearing impaired participants view the CM captions. The questionnaire reveals that there are significant differences among the three types of captions for a detergent CM, where partially captioned CM scores higher than the others in readability.