2013 Year in review: pushing the limits of Big Data

2013 was a busy year for Google Cloud Platform. Watch this space: each day, a different Googler who works on Cloud Platform will be sharing his or her highlight from the past year.



My highlight from 2013 was speaking to developers from across the world at the Big Data Spain conference. I used the opportunity to share with people everything that we have accomplished with Big Query. Although every tool has its limits, it was a joy to review how many big data limits we broke during 2013. Some of the highlights included how to grow a database that already can be as tall you need it to be - in 2012 each row could contain up to 64kB of data. Today, that number is up to 20MB. Other updates include the ability to combine and join 2 insanely huge tables, aggregate values in cases that were considered to have too many groups to group by, or return results of arbitrary sizes (JOIN EACH, GROUP EACH BY, and the allowLargeResults flag). BigQuery not only got bigger this year, it also got smarter: The new window and analytic functions allow users to run richer queries while the new correlation function allows surfacing and discovering relationships previously invisible. What a good way to close this year!



-Posted by Felipe Hoffa, Developer Programs Engineer