IBM's latest anomaly-finding algorithm crunches 9TB of numbers in 20 minutes
This is one of many surprises you can always expect from big blues, in fact IBM has always focused to develop the efficient computational algorithms in order to reduce the computational complexity and cost related to the processing of the enormous pools of data. What is new about this latest breakthrough from IBM is the development of energy efficient method to crunch the big data. The mathematical algorithm, developed by IBM’s laboratories in Zurich, is tricky and quite general in nature, in fact it can be used to solve several domain specific problems including bioinformatics one. According this IBM news announcement:
In a record-breaking experiment, IBM researchers used the fourth most powerful supercomputer in the world — a Blue Gene/P system at the Forschungszentrum Julich in Germany — to validate nine terabytes of data (nine million million or a number with 12 zeros) in less than 20 minutes, without compromising accuracy. Ordinarily, using the same system, this would take more than a day. Additionally, the process used just one percent of the energy that would typically be required.
Mathematical description of algorithm written on the back of the glass
Image Credits IBM Research Zurich Flickr stream
Image Credits IBM Research Zurich Flickr stream
In a post-Hadoop scenario this work has major implications especially when most of Hadoop implementations are tuned for availability and computational efficiency, not for the energy efficiency.
4 Responses to “IBM's latest anomaly-finding algorithm crunches 9TB of numbers in 20 minutes”
Latest Tweets
-
Dear Entrepreneur, CXO of unknown start-up with uncertain business model, be grounded and humble. cheersFebruary 10, 2012 at 2:53 am -
Apache Giraph, a framework for large-scale graph processing on Hadoop, reaches 0.1 milestone http://t.co/ymaoSXtTFebruary 10, 2012 at 1:57 am
-
Got this today, nice collection of Studio Ghibali Songs. Yeee http://t.co/mtauzVI7February 10, 2012 at 1:46 am
IBM's latest anomaly-finding algorithm crunches 9TB of numbers in 20 minutes http://bit.ly/aaV7vX
IBM's latest anomaly-finding algorithm crunches 9TB of numbers in 20 minutes http://bit.ly/aaV7vX #science
IBM's latest anomaly-finding algorithm crunches 9TB of numbers in 20 minutes http://bit.ly/aaV7vX #fisheye
IBM's latest anomaly-finding algorithm crunches 9TB of numbers in 20 minutes- by Fisheye Perspective http://bit.ly/99N8Z6