Google Scribe : An autocompletion service

Google Scribe was launched this september. Its beta phase looks deficient but promising. I had the notion of such a auto completion of text . Imagine yourself typing at a document and the autocompletion service saving your time. I am trying to extend the same idea over to Rhyme - Seeker ,"a ranked autocompletion of poetry". I saw a similar idea at ISI for Creative language use. I believe that the technology of Scribe could be extended to achieve the task.
       Google Scribe seems to be straightforward application of web n-gram language models covered in an AJAX interface. Some of its mistakes demonstrate the drawbacks of not utilizing long range word dependencies and topical context. It also adds numerous options for searching.

  Google states, “Scribe’s suggestions indicate correct or popular phrases to use". Most probably , as I aslo experimented with it, Google Scribe doesn't personalize the autocompletion yet. It would be amazing to see scibe getting enhanced and added to google docs .

The new Lucene Based Search Architecture of Twitter

A new backend search architecture that was launched few weeks ago. The previous system was based on the original Summize search system that Twitter acquired in 2008. The old system was based on MySQL. This system became difficult to scale for such large data. They turned toward a new, modern search architecture that is based on a highly efficient inverted index instead of a relational database, which is the obvious choice for large data indexing for quick search.
                   They did lots of tuning for improving the current version of  Lucene. Some of the highlights of the  changes include:

• significantly improved garbage collection performance
• lock-free data structures and algorithms
• posting lists, that are traversable in reverse order
• efficient early query termination 

We can hope to find these contributions to the Lucene soon.

According to the blog, the new search system was designed to handle over 1,000 TPS (Tweets/sec) and 12,000 QPS (queries/sec) = over 1 billion queries per day . Besides the challenging query volume, the data needs to be available quickly, a tweet needed to be searchable in less than 10 seconds.

The main benefit to users is that the new system is much more scalable and can support an index that is twice as large as previous versions which means that you can search for tweets further back in time.