Note that there are some explanatory texts on larger screens.

plurals
  1. PO
    primarykey
    data
    text
    <p>It's tough to say without more information about your infrastructure and desired scaling targets. You may find this slide deck about <a href="http://www.slideshare.net/kevinweil/hadoop-pig-and-twitter-nosql-east-2009" rel="nofollow noreferrer">How Twitter Uses Hadoop</a> to be instructional. It was presented by <a href="http://twitter.com/kevinweil" rel="nofollow noreferrer">Kevin Weil</a> at the recent <a href="http://paulstamatiou.com/recap-nosql-east-conference-2009" rel="nofollow noreferrer">NoSQL East conference</a>.</p> <p><a href="http://paulstamatiou.com/wp-content/uploads/2009/11/nosqleast_weil_twitterpig.jpg" rel="nofollow noreferrer">alt text http://paulstamatiou.com/wp-content/uploads/2009/11/nosqleast_weil_twitterpig.jpg</a></p> <p>Borrowing ideas from what Twitter is doing you could consider an architecture split into collection, analysis and render phases.</p> <p><strong>Collection Phase</strong>: Super low latency. Very scalable. Lots of binding choices. Developed at <a href="http://github.com/facebook/scribe" rel="nofollow noreferrer">facebook</a>.</p> <blockquote> <p>Processing Node Log Event -> <a href="http://www.cloudera.com/blog/2008/11/02/configuring-and-using-scribe-for-hadoop-log-collection/" rel="nofollow noreferrer">Scribe</a> -> HDFS </p> </blockquote> <p><strong>Analysis Phase</strong>: SQL-like query language that will allow you to do exploratory ad-hoc queries as well. </p> <blockquote> <p>HDFS -> <a href="http://www.cloudera.com/hadoop-training-pig-introduction" rel="nofollow noreferrer">Pig</a> -> MySQL</p> </blockquote> <p><strong>Render Phase</strong>: Implemented in your current web framework</p> <blockquote> <p>MySQL -> JSON -> Memcached -> Flash Charting</p> </blockquote> <p>There have been some posts here on SO regarding choice of Flash charting components for thew web. I personally have had good success with <a href="http://www.amcharts.com/" rel="nofollow noreferrer">AmCharts</a>.</p> <ul> <li><a href="https://stackoverflow.com/questions/2543">What are the best solutions for flash charts and graphs?</a></li> <li><a href="https://stackoverflow.com/questions/153303">What are some good toolsets for graphing/charting in a web application?</a></li> </ul>
    singulars
    1. This table or related slice is empty.
    plurals
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. This table or related slice is empty.
    1. VO
      singulars
      1. This table or related slice is empty.
    2. VO
      singulars
      1. This table or related slice is empty.
    3. VO
      singulars
      1. This table or related slice is empty.
 

Querying!

 
Guidance

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload