The Web is More Than Text

Nevertheless, the next generation Web should not be based on the false assumption that text is predominant and keyword-based search will be adequate for all reasonable purposes [127]. Indeed, the issues relating to navigation through multimedia repositories such as video archives and through theWeb are not unrelated: both need information links to support browsing, and both need engines to support manual link traversal. However, the keyword approach may falter in the multimedia context because of the greater richness of many non-textual media [264]. The Google image search approach relies on the surrounding text for an image, for example, which allows relatively fast search, and again in general the user is often able to make the final choice by sifting through the recommendations presented (keyword-based image searches tend to produce many fewer hits, which may mean they are missing many plausible possibilities). The presence of the human in the loop is hard to avoid at the moment: human intervention in the process of integrating vision language with other modalities is usually required [224], although there are a number of interesting techniques for using structures generated from texts associated with image collections to aid retrieval in restricted contexts [7].

All of our search technologies, semantic explorations, and other online conversations are all dependent on text, but they must grow to be able to read images, audio, and video as well.