Multimedia Explorer

Multimedia Explorer is a framework for content-based multimedia exploration. Multimedia explorer makes multimedia document more searchable by allowing detailed enrichment of the content information. It allows annotation and search of multimedia contents at different levels of granularity.

Business Challenges

Multimedia applications hosted/accessed over broadband and mobile networks are gaining increasing popularity. A few examples of such applications include IPTV, professional collections and peer-to-peer social networks, such as YouTube and Truveo. Most of these multimedia applications employ tags, annotations, and other forms of metadata, at a superficial level, to ensure and enhance the accessibility of the media. Usually, such metadata is too general to reveal the contents/appropriateness of the media, which can only be gathered by the actual physical access of the media. The difficulty in extracting semantics from such media data, popularly known as the “semantic gap”, makes it difficult for an accurate search and navigation of such media content.   

The lab’s Multimedia Explorer solution seeks to effectively address the above challenges through content-based interaction with multimedia data.    

Technical Details

The Multimedia Explorer employs a human-computer collaborative framework for semantic annotation and access of media content. Multimedia Explorer automatically extracts features from the media content and sets up an interactive and collaborative environment for media annotation.

The tool comprises a three-level media description for interactive access:

1. Video based
2. Scene based
3. Object based

Each of these levels includes media features as well as semantic annotations. The framework provides a workbench that automatically segments the video into shots based on the color features. The workbench allows the user (content creator) to annotate and tag at scene and shot levels (segments of a video).  This way the metadata gets enriched and this further facilitates the text-based search (content retrieval) along with the feature-based search. A multimedia ontology is used to assist search and navigation in the semantic space.  

Benefits

The Multimedia Explorer framework allows associating an enriched metadata with the multimedia artifacts. It makes the documents more searchable by allowing text-based and feature-based search interfaces. The end user can take more informed purchase decision using this framework.

Use Case Scenarios

The Multimedia Explorer solution has been customized for an online book and documentary selling portal. It can be used for the following applications:

• Educational Portal

  • Containing any type of multimedia collection like Video, PPT, PDF, etc.
  • Sharing slides and remarks
  • Add and share personalized notes on documents
•  Online video libraries and stores
• Cultural Heritage Portals
  • Dance mudras, musical instruments used, etc.
• Personalized video sharing

To know more/download artefacts about the Multimedia Explorer view:  

Sujal S Wattamwar, Surjeet Mishra and Hiranmay Ghosh. Multimedia Explorer: Content Based Multimedia Exploration. IEEE Tencon, Hyderabad ( India) November 2008