Current and Recent Projects:


Storied Navigation

Edward Shen, Henry Lieberman
, Glorianna Davenport

Today, people are able to tell stories by composing, manipulating, and sequencing individual media artifacts using digital technologies. However, these tools offer little help in developing a story's plot. In this project we present a novel approach with commonsense reasoning technology to this problem - "Storied Navigation" - where media sequences are tagged with free-text annotations and stored as a collection. To tell a story, the user inputs a free-text sentence and the system suggests possible segments for a storied succession. This process iterates progressively, helping the user to explore the domain of possible stories.

Scenario-Oriented Recommendation

Edward Shen, Henry Lieberman

Although Electronic Commerce on the Web is thriving, they still have trouble finding products that will meet their needs and desires. Based on commonsense reasoning tools, our new recommendation technique, Scenario-Oriented Recommendation, works even when users don't necessarily know exactly what product characteristics they are looking for. It breaks down boundaries between products' categories, finds the "first example" for existing techniques like Collaborative Filtering, and helps promote independent brands.

Your Memory, Connected

James Chao-Ming Teng, Edward Shen, Henry Lieberman
, Pattie Maes

Imagine a situation where you are an artist, and you are able to ask a question or give a statement to all the people in the world. What if you could check the memories that are being evoked when the people are confronted with your question or statement, and you could "steal" those memories to create an artistic collage? Our system uses natural language processing, concept reasoning, and textual affect sensing techniques to collect all the related memories from people on Flickr. We create this computational "memory retrieval" procedure that simulates the process of evocation when people's brains are triggered with the typed-in statement from the artist.

A Goal-Oriented User Interface for Personalized Semantic Search

Alex Faaborg, Henry Lieberman

Users have high-level goals when they browse the Web or perform searches. However, the two primary user interfaces positioned between users and the Web, Web browsers and search engines, have very little interest in users' goals. Present-day Web browsers provide only a thin interface between users and the Web, and present-day search engines rely solely on keyword matching. This thesis leverages large knowledge bases of semantic information to provide users with a goal-oriented Web browsing experience. By understanding the meaning of Web pages and search queries, this thesis demonstrates how Web browsers and search engines can proactively suggest content and services to users that are both contextually relevant and personalized.

Reducing Complexity in Consumer Electronic Interfaces

Jose Espinosa, Henry Lieberman

Consumer electronics devices are becoming more and more complicated to the point that the user is scare to manipulate them.  These devices do not know anything about every day life and human goals and they show irrelevant menus and options. By using EventNet, a comonsense reasoning plan recognizer, we build an interface with knowledge about the userís intentions. This knowledge helps the device to display relevant information to reach the userís goal. For example, a living room set that knows how to configure itself to watch the news. This leads to a more human like interaction with these devices.

A Commonsense Approach to Predictive Text Entry

Tom Stocky, Alex Faaborg, Henry Lieberman

People cannot type as fast as they think, especially when faced with the constraints of mobile devices. The focus of this project is developing an alternative approach to predictive text entry using Open Mind Common Sense.

i-Seek: An Intelligent System for Eliciting and Explaining

Ashwani Kumar, Henry Lieberman

We propose i-Seek, an Intelligent System for Eliciting and Explaining Knowledge that leverages the Open Mind Commonsense knowledgebase in conjunction with domain specific knowledge in Personal Finance, Technical Help, and Health domains to act as an advisory system for novice users.

Commonsense Investing: Bridging the Gap between Expert and Novice

Ashwani Kumar, Henry Lieberman

This project aims to develop an intelligent personal-finance advisory agent that bridges the gap between the novice user and the expert model of the finance domain. The agent uses common-sense reasoning and inference for associating the user's personal life, financial situation, and goals with the attributes of the expert domain model and vice versa. The agent interface provides a natural-language interface for elicitation and explanations of design and process rationale. The architecture of the system is domain independent and consequently, can be used for any novice-expert domain model.

Anticipating User Tasks Using Commonsense Reasoning

Alex Faaborg, Chris Tsai, Henry Lieberman

The goal of this project is to create an agent that predicts tasks users may be interested in adding to their to-do list, based on the context of appointments they are adding to their calendar. The agent uses Commonsense Reasoning to predict tasks that users are likely to add. For instance, if the user makes the calendar appointment ďgo fishing? the agent may recommend the task ďpack fishing equipment.?By automatically generating potential tasks, the agent will save userís time, and will help them remember actions they may commonly forget.

Using Commonsense Reasoning in Video Game Design

Andrew Wang, Alex Faaborg,
Jose Espinosa, Henry Lieberman

When computers automatically generate video game environments, the result is often as simplistic as a randomly generated maze. Using the Open Mind Common Sense knowledge base, we are exploring ways to automatically generate virtual environments that are immersive and intelligently designed. By allowing video games to dynamically create their own environments, developers will be able to reduce the time they currently spend crafting environments by hand and focus on higher level design issues. More importantly, games that are capable of changing their own environments will feel less static, resulting in a more believable experience and an increased replay value.

Using Commonsense Reasoning to Improve Voice Recognition

Waseem Daher, Jose Espinosa, Alex Faaborg, Henry Lieberman

Current voice recognition software relies on statistical techniques to determine which words a user has said. In this project we are attempting to leverage the semantic context of what the user has previously said to improve future predictions. We are using ConceptNet, a semantic network created from the Open Mind Common Sense knowledge base to disambiguate phonetically similar words and improve overall recognition accuracy.

Using Commonsense Reasoning to Enable the Semantic Web

Alex Faaborg, Sakda Chaiworawitkul, Henry Lieberman

Current efforts to build the Semantic Web have been based on creating machine readable metadata, using XML tags and RDF triples to formally represent information. It is generally assumed that the only way to create an intelligent Web agent is to build a new Web, a Web specifically for machines, using a unified logical language. We are attempting to solve this disparity between humans and machines from the opposite direction, by enabling machines to understand and reason on natural language statements, and giving them knowledge of the world we live in.

Real Time Searches on a Local Social Network

Alex Faaborg, Chris Tsai, Henry Lieberman, Judith Donath

Cell phone contact lists represent a very large peer-to-peer network. We are creating a cell phone based application that allows users to perform real time searches on their local social network, against pieces of information that their contacts have provided about themselves. Our matchmaking agent uses Open Mind Common Sense to understand users?goals, and logically expand on their queries.

Using Commonsense Reasoning to Find Cultural Differences in Text

Jose Espinosa, Henry Lieberman

Because commonsense knowledge differs in each culture, misunderstandings frequently occur. Since differences can be subtle, there has been little work in trying to detect places in text where cultural differences might arise. We explicitly represent commonsense knowledge of each culture in separate knowledge bases. By analyzing the text, we can find the differences between each culture's knowledge concerning the subject of the text.

GloBuddy 2

Jose Espinosa, Alex Faaborg, Henry Lieberman

When traveling in foreign countries, people often rely on traditional phrase books for language translation.  However, these phrase books only work in a limited number of common situations, and even common situations will often deviate from the predefined script the phrase book relies on.  Using a vast knowledge base of commonsense facts and relationships, GloBuddy 2 is able to expand on the userís translation request and provide words and phrases related to the userís situation.


Earl Wagner, Henry Lieberman

"Follow the Money" Woodstein, the E-Commerce Debugger, shows users the processes that they're working with over the web. As users fill out forms and submit data, it graphically represents the structure of their actions. That way, when problems occur, they're able to bring up the record of actions they took and are able to see what went wrong.

Aria: An Agent for Integrated Annotation and Retrieval of Images

Hugo Liu, Henry Lieberman

ARIA (Annotation and Retrieval Integration Agent) is a software agent that acts as an assistant to a user writing email or Web pages. As the user types a story, it does continuous retrieval and ranking on a photo database. It can use descriptions in the story text to semi-automatically annotate pictures based on how they are used.

Emotus Ponens

Hugo Liu, Henry Lieberman, Ted Selker

Psychologist William James noted that the recognition of emotion is intimately related to traditions and culture. Based on the thesis that much of our emotional attitudes are dictated by our culture's "common sense" about everyday situations, this project uses large-scale affective commonsense from Open Mind to analyze the broad emotional qualities of sentences.

MakeBelieve: Interactive Computer Story Generation

Hugo Liu, Push Singh

MAKEBELIEVE is a story generation agent that uses Open Mind knowledge to interactively compose short fictional texts with a user. While a user must start a story, MAKEBELIEVE will attempt to continue that story by freely imagining possible sequences of events that might happen to the character the user has chosen. The agent uses "commonsense" about causality and how the world works, mined from the Open Mind Common Sense corpus, and combines this with very simple lingustic techniques for story generation to produce pithy but interesting stories. MAKEBELIEVE also uses common sense to evaluate and critique a story it has written to catch logically inconsistent, incoherent events and actions.

Goal-Oriented Web Search User Interfaces

Hugo Liu, Henry Lieberman, Ted Selker

A novice search engine user may find searching the Web for information difficult and frustrating because she may naturally express search goals rather than the topic keywords search engines need. GOOSE (goal-oriented search engine interface) is an adaptive search engine interface that uses natural language processing to parse a userís search goal, and uses "common sense" reasoning to interpret this goal, and reason from it an effective query.

Previous Projects


  Copyright ?@