[Domeshek, 92][Johnson, 00]) where human analysts identify the specific points that are made by a story, and encode these points using formal representation to facilitate their retrieval. Point-based indexing of this sort is particularly suited for cross-domain retrieval, e.g. where a story about conducting contract negotiations with a company in a supply chain illustrates a point that is directly relevant to establishing a clean water supply in an impoverished region. In our own work, we have not pursued this style of point-based indexing. Aside from the difficulties in automating approach, cross- domain stories have not proven to be particularly useful in the development of the training applications mentioned earlier in section 2. Instead, we have seen that the stories most relevant to a learning need are within the same task domain or activity, e.g. the stories that are most useful for learning how to conduct contract negotiations are about the task of conducting contract negotiations. The technical challenge is to automatically select stories that describe experiences in the context of task or activity.
Our approach to this automation challenge was to use techniques for textual information retrieval, where textual descriptions of activities or tasks are used as queries for the retrieval of textual stories from a collection. To explore the feasibility of this approach, we constructed a large-scale retrieval system for finding stories automatically extracted from Internet weblogs [Gordon, 08]. In this system, called StoryUpgrade, users described an activity as a paragraph-sized description of events expected to occur in the task, written as a past-tense first-person narrative (a “boring story” of the activity). The system then encoded the query as a weighted vector of lexical features, and then ordered the relevance of stories in the collection using vector-based similarity measures. Two versions of this system were built that differed in the way that they collected stories from Internet weblogs. In the first version, we created a single collection of stories by applying our story capture technology (described earlier in section 3) to 3.4 million weblog entries, yielding a text corpus of over one billion words. In the second version, we constructed specialized story collections for each query by first sending fragments of the activity description to a commercial weblog search engine, and then applying our story capture technology to hundreds or thousands of the top search results.
We evaluated the effectiveness of the StoryUpgrade system (version 1) as a tool for finding stories relevant to the development of simulation-based training applications. The Intelligent Convoy Operations Support (ICOS) project at the University of Southern California’s Institute for Creative Technologies was an effort to develop a tutoring module to augment a training simulation for military convoy operations. Four members of the ICOS project team used the StoryUpgrade system to find relevant stories about military convoy operations told in Internet weblogs. In one forty-five minute session, these participants authored five activity descriptions of the convoy operations task and judged the relevance of 67 retrieved weblogs. There were 23 relevant stories, 6 relevant non-stories, 14 non-relevant stories, and 24 non- relevant non-stories (55% story precision, 43% relevance precision). We concluded that the performance of our approach to pairing task descriptions to stories in the collection was not high enough to fully automate this process, but that a developer could efficiently use this technology to find a sufficient number of task-relevant stories in very large collections in less than an hour.