Sunday, May 23, 2010

Maltparser: A parser to build parser: Introduction & Installation

While a traditional parser-generator constructs a parser given a grammar, a data-driven parser-generator constructs a parser given a treebank. MaltParser is a system for data-driven dependency parsing, which can be used to induce a parsing model from treebank data and to parse new data using an induced model.
MaltParser implements seven deterministic parsing algorithms:
  • Nivre arc-eager
  • Nivre arc-standard
  • Covington non-projective
  • Covington projective
  • Stack projective
  • Stack swap-eager
  • Stack swap-lazy
 You can choose whichever algorithm you intend to apply from command promt using Options(see further)

MaltParser currently includes two machine learning package :
  • LIBSVM - A Library for Support Vector Machine.
  • LIBLINEAR -- A Library for Large Linear Classification

    The latest version of MaltParser is available from the MaltParser download page
I discuss installation of maltparser on Ubutu Platform. I guess it works on all other platform , if you face any problem, write to me.
Download the Zip File  : malt 1.3.1 
Check if there are newer versions available.
Download the zip file to the directory where you want to extract it or copy it to the directory.
open your command prompt and set the path to the directory . Now type
prompt> unzip malt-1.3.1.zip

To run MaltParser type the following at the command line prompt (it is important that you are in the malt-1.3.1 directory):

prompt> java -jar malt.jar
If MaltParser displays something like the message below, the distribution has been installed successfully.
-----------------------------------------------------------------------------
                          MaltParser 1.3.1
-----------------------------------------------------------------------------
         MALT (Models and Algorithms for Language Technology) Group
             Vaxjo University and Uppsala University
                             Sweden
-----------------------------------------------------------------------------

Usage:
   java -jar malt.jar -f  
   java -jar malt.jar -h for more help and options

help                  (  -h) : Show options
-----------------------------------------------------------------------------
option_file           (  -f) : Path to option file
-----------------------------------------------------------------------------
verbosity            *(  -v) : Verbosity level
  debug      - Logging of debugging messages
  error      - Logging of error events
  fatal      - Logging of very severe error events
  info       - Logging of informational messages
  off        - Logging turned off
  warn       - Logging of harmful situations
-----------------------------------------------------------------------------
Documentation: docs/index.html


To be able to build MaltParser you need Java SE Development Kit (JDK) 6 and Apache Ant 1.7.0. If you dont have these tools please download and install them by following the instructions on their web sites.

To build the MaltParser distribution:
prompt> cd 
prompt> ant dist
should be replaced by the actual path to the directory where malt-1.3.1 is installed.

Friday, May 21, 2010

Morphological Parsing

Parsing in general means, taking an input and producing some sort of structure for it. In NLP the structure could be in various forms like:
1)Morphological
2)Syntactic
3)Semantic
4)Pragmatic

Morphological Parsing takes as input the inflected "surface form" ofLink each word in a text. As output, it produces the parsed form consisting of a canonical form (or lemma) of the word and a set of "tag" showing its syntactical category and morphological characteristics.

Morphological generation is the inverse of morphological analysis, namely the process of converting the internal representation of a word to its surface form. For example, if our internal representations of the words mice and move are:

Root: MOUSE  PartOfSpeech:Noun   Number:Plural
Root: MOVE PartOfSpeech:Verb Tense:Past
then morphological generation would convert these to the character strings mice and moved


The morphological parser gets its information from the extraction of:
1)Lexicon: It lists stem and affixes together with basic information about them
2)Morphaotactics: It describes way morphemes are arranged or touch each other
3)Orthographic Rules: These are spelling rules that specify changes that occur when the ordering when two given morphemes combine.

IIIT-Hyderabad And NLP

I am at the International Institute of Information Technology, Hyderabad and working on NLP for LTRC(Language Technologies Research Center)
ICON(International Conference on Natural Language Processing) conference series is a forum for promoting interaction among researchers in the field of Natural Language Processing in India and abroad. Various interesting papers, have been presented lately in the conference.
The SIEL(Search and Information Extraction Lab) has various interesting project. One which I found very interesting was cloud computing course. The lab is collaborating with Yahoo!, which are the biggest users and promoters of Apache Hadoop. SIEL has developed a very close research partnership with Yahoo! R&D Bangalore.
Link

Sunday, May 16, 2010

New links and materials


OpenLibrary Relaunch- A redesigned OpenLibrary website was launched last week, designed by Caterina Flake. The new site is like Wikipedia meets a Library catalog. See their blog for details.

LikeLike - An implementation of LSH written for Hadoop. There isn't much documentation on how it is parallelizing the computation.

Inferring Web Searcher Intent Tutorial - The slides from Eugene Agichtein's WWW 2010 tutorial are now available. The first part of the tutorial provides an overview of user task and behaviour models. The second part focuses on utilizing implicit feedback from clicks and other interaction activity.