Compute Engine Load Balancing hits 1 million requests per second!

The C10k problem is so last century. How about serving 1 million load balanced requests per second in the cloud? Using publicly available Google Cloud Platform services, including Google Compute Load Balancing, we have done exactly that. Within 5 seconds after the setup and without any pre-warming, our load balancer was able to serve 1 million requests per second and sustain that level.



This number is 20 times greater than the throughput in last year’s Eurovision Song Contest, which served 125 million users in Europe and was hosted on Google Compute Engine. The Eurovision setup used DNS load balancing, which increases the complexity of setup, maintenance and cost. We were able to simplify this process using Compute Engine Load Balancing, which avoids these issues by providing simple APIs and allowing a single IP address to serve all traffic at a lower cost than do-it-yourself options. In addition, Compute Engine Load Balancing has the ability to detect unhealthy instances and dynamically add and remove new instances to serve traffic. This addresses the hard-to-solve problem of cached DNS entries in end-user browsers. No more “404’s”.

Starting with an empty Compute Engine project and ending with 456 cores provisioned (for the load generator and web servers VMs) and one load-balanced IP address actively processing 1.016M Requests Per Second (+-0.007M) took a total of 7 minutes 30 seconds. The 1M number is measuring a complete request and successful response. You can read more on how to set up the Compute Engine Load Balancing and how to provision Compute Engine VMs on the Google Cloud Platform website.



The following depicts the setup used:

This setup demonstrated a couple of features, including scaling of the Compute Engine Load Balancing, use of different machine types and rapid provisioning. For generating the load we used 64 n1-standard-4’s running curl_loader with 16 threads and 1000 connections. Each curl_loader ran the same config to generate roughly the same number of requests to the LB. The load was directed at a single IP address, which then fanned out to the web servers.



To demonstrate scaling of the Compute Engine Load Balancing fanout we used 200 n1-standard-1’s Web Server running Apache v2.2.22 on Debian 7.1 Wheezy Images. Users are encouraged to use larger VM types for better single machine backend web serving, however here we demonstrated the scaling of the load balancer to backends and were not concerned with the backends themselves using every cycle to serve responses. Each backend web server received ~5K requests per second, which is an even distribution.



Compute Engine Load Balancing distributed the load by using a tuple of source address+port, destination address+port and protocol. Each web response was 1 byte in size not including the http headers. All of this was configured from an empty Compute Engine project. To reproduce the data yourself you can use the following Gist.



This entire setup and test cost just $10 USD!



-Posted by, Anthony F. Voellm, Performance Engineering Manager

Read More

Dedicated memcache is now generally available in App Engine 1.8.8

With the 1.8.8 App Engine release, we are glad to announce that dedicated memcache is now generally available.



Dedicated memcache is an App Engine feature that lets you scale your caching capacity indefinitely without having to manage a server farm of memcached machines. After going into Preview in July, hundreds of customers have deployed many terabytes in production, including a single application using six terabytes.



Customers have told us that with higher cache hit rates, dedicated memcache reduces their Datastore costs and makes their applications faster. Ben Kamens, Lead Developer at Khan Academy said, “Dedicated memcache helped us take more control over the performance of our site — we'll almost certainly be using memcache more and more as a result."



Since the preview announcement we’ve added several features to help customers manage their cache, such as hot key warnings in the memcache viewer, and cache size and performance graphs in the dashboard.



To get started using dedicated memcache, select dedicated memcache on the App Engine admin console’s application settings page.



The complete list of features and bug fixes for 1.8.8 can be found in our release notes. For App Engine coding questions and answers check us out on Stack Overflow, and for general discussion and feedback, find us on our Google Group.



-Posted by Logan Henriquez, Product Manager

Read More

Development tips for Google Cloud Endpoints for Android

You’ve been struck with an idea for the perfect idea for a mobile game on Android, and you’re sure it will be a hit. The details are a little fuzzy, but it definitely has something to do with cats. But, being an experienced engineer, you know how much work needs to go into creating and maintaining a scalable backend. You’re going to need servers, and then you’ll have to load-balance them. And you’re going to need to write protocols to enable your multiple servers to communicate with your mobile clients. So much work, and all you really want to do is create attractive content.



How about using Google Cloud Endpoints, a service for building and exposing your custom APIs? You can create a new App Engine backend for your project with a single command and generate a client library from your API with another. Cloud Endpoints is easy to use and enables flexible and agile development, which is important (because remember, so far your idea consists of just “cats”).



One of the benefits of being able to easily build your own API is that you can design it any way you want. Cloud Endpoints gives you this flexibility—it allows REST- or RPC-style APIs, or a combination of both. Even better, you can change the design as often as you like and implement those changes quickly.



To help you get started, we have published a new article about Google Cloud Endpoints for Android to complement the developer docs in Java and Python. The article walks you through the development process and answers many common questions. For example, what is representational state transfer, anyway? Which combinations of Endpoint class notations should you use? How do you support enums in your API?



This paper is a useful -- and occasionally entertaining -- addendum to aid you in navigating Google Cloud Endpoints. It features some programming gems that you won’t want to miss, such as “lobster.dance();”.



Good luck, have fun! We look forward to playing your mobile cat game.



- Posted by Helen T Chou, Solutions Application Engineer

Read More

Offline Disk Import and the OmNomNom Machine

Yesterday, we announced that we are expanding our offline disk import service to better serve users globally. With new disk upload centers in Switzerland, Japan and India, as well as our US center, it’s easier for people around the world to import large data sets by mailing hard drives to us rather than sending hundreds of terabytes over their slow, expensive or unreliable Internet connection.



But importing large amounts of data at scale isn’t simple. Our engineers have been working on the challenge for years. Originally, offline disk import was handled at our data centers as a way to efficiently import large amounts of data from the hard drives in our Street View cars - vehicles that capture terabytes of photographs and information about the landscape as they build a navigable, visual database of the world.



And although it’s technically challenging, the system we built for rapidly ingesting and processing these large data sources has a playful name. We call it OmNomNom. Here is one of the test OmNomNom machines that the disk import team has at its offices in Mountain View:

But rapidly ingesting and processing large data sets and making them usefully available is a bit more complex than gobbling down a cookie. As we’ve improved our ability to quickly import these drives, we dramatically reduced the time between capturing these images and making them available to users around the world.



Now, we are helping people across the world take advantage of the speed, scale and global availability of Google Cloud Storage as well as this rapid disk-upload technology. Even though it might sound like something out of Sesame Street, this is another example of how Google Cloud Platform is making the advantages of Google-sized scalable infrastructure available to you. All you need to do is send us your EncFS encrypted hard drives, and we will let you know once your encrypted bytes are imported to your designated GCS bucket. Once uploaded, we can mail your drives back to you, or if you prefer, safely and securely handle disk destruction free of charge. Check out our website for how you can be part of the Limited Preview of International Offline Disk Import.



(Oh, and for those of you who want to use Google Cloud Storage but don’t need offline disk import, you always have quick access Google Cloud Storage from the command line using gsutil).



-Posted by Benjamin Bechtolsheim, Product Marketing Manager

Read More

International Offline Disk Import now available with new disk-upload centers in Japan, India and Switzerland

Google Cloud Platform customers have told us that transferring large data sets (in the hundreds of terabytes and beyond) can be expensive and time-consuming over unreliable Internet connections. Today, we are pleased to announce that Offline Disk Import for Google Cloud Storage is available internationally in Limited Preview. Now, you can send disks to our disk-upload centers in Switzerland, Japan and India as well as the United States. From there, we will upload your data to your Cloud Storage regional buckets (US or Europe).



It begins when you send your encrypted data on physical disk drives (2.5” and 3.5” HDDs or SSDs) to a Google disk-upload center using a mail carrier. From there, we upload the data into an empty Cloud Storage bucket that you designate. Once your data is imported to your designated bucket, the standard Cloud Storage storage fees apply. This is a good option if you’re limited to a slow, unreliable or expensive Internet connection and with new international disk import centers, it is easier than ever.



Offline Disk Import is offered in all our disk-upload centers around the globe at a flat fee of $80 per disk, irrespective of the drive capacity, data size, your designated bucket location or the disk-upload center location. There are no additional fees for loading the data or per-byte charges.



Back in June, we announced the Limited Preview of the Cloud Storage Offline Disk Import service for U.S. customers and we’re glad that we can expand this offering to our customers around the world. Use the form on our Offline Disk Import website to let us know your feedback or sign up to participate in the Limited Preview.



-Posted by Lamia Youseff, Software Engineer

Read More

Google App Engine integration features in IntelliJ IDEA and PyCharm

Today’s guest post comes from Maarten Balliauw, Technical Evangelist at JetBrains, the vendor of smart developer tools such as IntelliJ IDEA, PyCharm, PhpStorm, Android Studio and many more.



We build tools that enhance developer productivity and code quality. We do this by providing plugins that support integration with databases, frameworks, and services, such as Google App Engine. In this post, we will look at how we’ve made it easier to deploy to Google App Engine from PyCharm or IntelliJ IDEA.



PyCharm Professional Edition comes with Google App Engine Integration built in. For IntelliJ IDEA Ultimate you need to enable it by installing the Google App Engine Integration plugin.



Once integration is enabled, you can create a new Google App Engine application, open a sample shipped with the Google App Engine SDK or open an existing project you may have.

Creating a new project in PyCharm

The IDE will ask for some additional information, such as the Google App Engine application id. Once that is done, the IDE will create a new project structure in which you can start working.



All the various coding assistance features in the IDE are available: syntax and error highlighting, automatic indentation, code completion and more. The Google App Engine SDK directory is automatically added to Libraries in your IDE so all the APIs and classes will provide autocompletion.

Code completion in IntelliJ IDEA knows about App Engine APIs

The IDEs also offer code inspections for common problems and issues in code. For example, PyCharm features some inspections that report the use of language features restricted on GAE and so on. IntelliJ IDEA provides similar inspections.

Google App Engine specific code inspections in Pycharm

During development, you can run your application locally without having to upload it to Google App Engine. The IDE provides a special run/debug configuration that launches your application locally.

Once started, you can navigate to it in your browser at the http://localhost:8080 URL, by default. The Run tool window at the bottom of the IDE will show the output of the GAE application server you launched locally.

Running Google App Engine Java app locally

If instead of using the Run button you use the Debug button (or Shift+F9), the IDE attaches a debugger to your application server. You can now add breakpoints in your application and inspect variables and so on using the Debug tool window once a breakpoint has been hit.

Debugging Google App Engine application in IntelliJ IDEA

Once you feel your application is ready to go live, you can deploy it to Google App Engine Appspot. From the Tools > Google App Engine > Upload App Engine app menu, you can trigger the upload. This will open a new tool window in which you enter your Google Account credentials. The IDE will run the appcfg.py tool and display the output.

App deployment from an IDE

Once the upload and deployment is finished, your application will be live. Congratulations!



As you’ve seen in this blog post, the Google App Engine integrations for IntelliJ IDEA and PyCharm enhance our tools by adding a local test Google App Engine development environment, debugging support and more. We’re not stopping here though, so keep an eye out for even more added functionality. We have some detailed tutorials available for Java and Python. Give it a try (trial versions are available for IntelliJ IDEA and PyCharm) and let us know your thoughts through the comments below.



Develop with pleasure!



-Contributed by Maarten Balliauw, Technical Evangelist, JetBrains

Read More

Connecting mobile developers to the cloud with Google Cloud Endpoints

As a mobile application developer, some projects demand building your own backend, while others can move faster with a ready-made solution. Two updates we are making to Google Cloud Platform help you with either scenario. To help you build your own backend, Google Cloud Endpoints has now moved to General Availability. If you are interested in a ready-made solution, the new version of the Mobile Backend Starter is now available with support for large media files in addition to updated iOS and Android clients.



Google Cloud Endpoints is now Generally Available

In a multi-platform, multi-client and multi-screen world, it's often important to think about building APIs first and using a shared backend to connect to client applications later. At Google, we have a history of providing APIs for products such as Maps, Translate and Gmail, which have led to the creation of new applications that are used by millions of users.



But we know that building an API for your own clients isn't easy. Scaling, authentication and tooling are all issues that need to be addressed. Google Cloud Endpoints provides developers with a simple way to create, expose and consume APIs served from App Engine.

Cloud Endpoints simplifies multi-client access to a shared Google App Engine backend

Blossom.io, a project management tool, built their product using App Engine and Cloud Endpoints. Here is what their CEO, Thomas Schranz, had to say:



“Cloud Endpoints certainly has enabled us to offer an API that is on par with the best APIs out there. It has enabled us to build an API that feels and behaves like APIs from YouTube, Google Maps and other Google services...We would never have been able to offer something comparable in the same amount of time. Building all of that without Google Cloud Endpoints would have been unthinkable.”



Server Side Annotation Language

For backend developers, simple annotations turn native Java and Python code into APIs that can be easily deployed and consumed by Android, iOS and web clients. Deployed APIs gain the resilience and scalability that other Google APIs have with features like DoS protection, OAuth support and client key management.

package guestbook;

import javax.inject.Named;

import com.google.api.server.spi.config.Api;

@Api
public class GreetingsEndpoint {

  public Greeting getGreeting(@Named("id") Integer id) {
    return createGreeting(id);
  }

  public Greeting[] listGreetings() {
    Greeting[] greetings = new Greeting[] {
      createGreeting(1),
      createGreeting(2)
    };
    return greetings;
  }

  private Greeting createGreeting(Integer id) {
    Greeting greeting = new Greeting();
    greeting.setMessage("Hello " + id);
    return greeting;
  }
}

Client Side Development

Client developers on Android, iOS and web can use automatically generated client libraries that make API access simple and natural. Cloud Endpoints libraries take a lot of the complexity away by handling marshaling and unmarshaling, authentication and key verification. Furthermore, Android Studio offers great support for consuming Cloud Endpoints.

Automatic strongly-typed client library generation in Android Studio to simplify API consumption

Mobile Backend Starter Update

If you would rather focus on your client application, Mobile Backend Starter (MBS) is a one-click deployable, complete mobile backend built on Cloud Endpoints. MBS provides a ready-to-use, cloud backend and client-side framework for Android and iOS. We have distilled the best practices from hundreds of successful mobile backends on App Engine to give developers a turnkey solution that requires no server side development.



MBS includes a server that stores your data with App Engine and a client library and sample app for Android and iOS that make it easy to access that data. You can also use the built-in support for Google Cloud Messaging (GCM) for Android, Apple Push Notifications (APN) for iOS and continuous queries to notify your app of events you are interested in. In addition, MBS includes built-in support for Google Authentication to keep user data secure.

The new version of MBS includes:

• Handling Large Media Files: Many mobile applications let users view and upload videos and high resolution images. But storing and serving this content can be cumbersome. Google Cloud Storage offers high durability and availability at low cost. MBS now allows you to easily manage user-isolated and secure access to data in Cloud Storage directly from your iOS or Android application, with no server coding required.
  
  
  
  Imagine that you are building an expense reporting mobile app and you want to allow your users to upload pictures of their receipts. This is very straightforward. Your Android or iOS app can obtain a secure upload URL that only your application can use and then use standard client libraries to stream bytes to Google Cloud Storage. In this snippet of code for Android notice that in this case the uploaded file will be available only to the user who uploaded it.
  
  

  // get a secure URL for uploading a private file
  URL url = blobEndpoint.getUploadUrl("receipts",  
                               getReceiptFileName(),"PRIVATE")
                         .execute().getShortLivedUrl());
  

  
  
  
  Downloading images and other files is also easy. You start with obtaining a secure download URL and then you can use standard client libraries to download bytes from Google Cloud Storage. MBS will authenticate the caller and check if it is allowed to download the file.
  
  

  // get a secure URL for downloading a file
  URL url = blobEndpoint.getDownloadUrl("receipts", 
                               getReceiptFileName())
                         .execute().getShortLivedUrl());
  

  
  


• Updated Mobile Clients: We have updated the template mobile clients as well. Both Android and iOS display an updated user interface and incorporate best practices. The Android client sports the newest version of Google Cloud Messaging.
  
  
  
  
  
   

So whether your mobile project demands rolling your own backend with Cloud Endpoints or you are able to start a ready-made backend of MBS, the Google Cloud Platform just got better for you. Get started with Cloud Endpoints or Mobile Backend Starter today!



-Posted by Tzachi Ben-Ayoun, Product Manager

Read More

Rovio, Freshplanet, Pocket Gems and others talk about building games on Google Cloud Platform

Steven Hauk of Rovio says that the most important thing to Rovio is “to delight our fans with great games and good products. And Google Cloud Platform allows us to focus on just that.”



When Rovio ships a new version of Angry Birds, they don’t have to provision new hardware in order to serve over 250 million active users. And since Songpop launched and eventually reached nearly 100 million users across the world, its creators at Freshplanet never worried about whether their servers will be able to handle the load.



That’s because these games are built on Google Cloud Platform - which is designed to scale smoothly whenever you need it.



Hear more from Rovio, Freshplanet, Pocketgems and others talk about building games on Google Cloud Platform in this video:

Read More

Build amazing real time games on Google Cloud Platform with Node.js and Websocket

Can you imagine the challenges associated with building a real-time game that has the potential to receive global media attention and a large number of simultaneous players?



Wouldn’t it be great to have a working example that ties it all together? A highly-available, auto-scaling, cloud-based game with interactive mobile clients, a real-time browser display, backend database and compute-intensive services?



We have published a new solutions article that shows how to use Google Cloud Platform services to solve a complex problem in real time and scale during an onslaught of demand. Our example uses free open source (FOSS) technologies, including WebSocket with Node.js, OpenCV and PhantomJS. The article looks into the design of a game called World Wide Maze that lets users turn any website viewed in the Chrome browser into a 3D maze and navigate through the maze using their mobile phone as a tiltable game controller. The game is intended to demonstrate the power of HTML5 and WebGL, and the article takes you behind the scenes to see how the Google Cloud Platform makes it work.

World Wide Maze: Real-time 3D maze game playable with Chrome and mobile devices

One of the biggest challenges when building a real-time game is handling the persistent connections between the clients and servers. In World Wide Maze, real-time communication from the game controller to the game screen needs to be handled within 200 ms. This real-time communication includes:

• The request from the controller received by the server


• The request routed to the screen from the server


• The maze rendered on the screen, based on the request

The article delves into how World Wide Maze is designed and how it solves the real-time communication challenges. On the server side, World Wide Maze uses a combination of App Engine and Compute Engine. Compute Engine accepts WebSocket connections that are used to implement the bi-directional low latency communication between the controller and the screen. Compute Engine is also used to host the database servers and to build game stages. App Engine accepts HTTP requests, orchestrates Compute Engine instances and connects the clients with available Compute Engine instances.



Read more in the article Real-time Gaming with Node.js + WebSocket on Google Cloud Platform.



- Posted by Jeff Peck, Cloud Solutions Technical Account Manager

Read More

Just Develop IT Migrates Petabytes of Data to Google Cloud Storage

We interviewed Brooke Bryan, CTO of Just Develop It Ltd (JDI), one of the leaders in device data backup with popular applications including My PC Backup. In early 2013, the company migrated 5 petabytes of data from Amazon S3 to Google Cloud Storage. Since then, JDI’s business has expanded their use to over 10 petabytes at a rate of 800 terabytes a month.



What does Just Develop IT do?



Just Develop It (JDI) was founded with the mission of developing online services to help consumers across the Internet. Today, JDI has numerous companies in its portfolio, ranging from Skylark Golf & Country Club, a beautiful and award winning golf and country club in the south of Hamphire, England, to Just Cloud, a service that backs up music, videos, documents, emails, iTunes, photos and pictures, and stores them in the cloud.



Why did JDI want to migrate to the cloud?



Maintaining physical storage servers in the data center was a nightmare and the downtime was unbearable. As our portfolio of services grew, especially the size and scale of our backup products, we knew that a move to cloud infrastructure was imminent. This move to the cloud was very strategic for our growth as a company because the cloud is much easier to manage and required far fewer system administrators than our old storage servers. We initially selected Amazon S3, but unfortunately, the migration took nearly four months due to the speed of the physical storage servers.



Why did you switch from Amazon S3 to Google Cloud Storage?



From a product perspective, we were happy with Amazon S3. However, the customer service left a lot to be desired, and we found that we were locked into a contract without flexibility.



We researched suitable replacements and decided to take a good look at Google. We discovered that Google Cloud Storage was not only a great product but also affordable. Google was willing to work closely with JDI during the migration and partner with us long-term to ensure our needs were being met. We were also very impressed from a security standpoint as Google storage systems offer full encryption ensuring that our users’ data is secure at all times, a feature that is essential to our business.



Initially, we tested switching applications in developer environments and conducted testing to ensure that Google could handle the load. After this was confirmed, we began to migrate over five petabytes of data to Google Cloud Storage.



What challenges did the migration present?

Because our users rely on our products for 100% uptime, we wanted to execute the move to Google Cloud Storage within a short timeframe and in a way that would go unnoticed to users. This presented a few challenges:

1. We needed high network bandwidth and parallel data transfer mechanisms to ensure that the migration could be completed as soon as possible.


2. Since JDI has multiple storage/backup products, the process had to be invisible to our users, as a disruption of service would directly affect the confidence in our products.


3. We had to keep costs low during the process, which raised two issues: We paid twice the fees to store objects in both Amazon and Google systems in order to ensure that services would not be disrupted; and if an object was copied multiple times, we incurred unnecessary AWS egress network costs.

How did Google meet these challenges?



Google went above and beyond our previous storage providers. Google’s network bandwidth proved to be reliable. Also, we were able to move all of our storage and workloads, such as versioning and file composition, into Google Compute Engine to take advantage of Google’s networks and compute resources.



From a migration and testing perspective, everything was completed within 2-3 weeks. The implementation took only 24 hours and was available in 3-4 days. Our users did not notice a disruption, and the quick turnaround ensured that migration costs were low. We now host over 10 petabytes of data in Google Cloud Storage, and our data is growing at the rate of over 800 terabytes every month.



How has your experience with Google been since the implementation?



We view Google as a strategic long-term partner based on our experience with Google Compute Engine and Google Cloud Storage. The company’s technical, customer relations and support teams have met our every need, and if there is a problem, we know they will work quickly to resolve it.



We are looking forward to expanding our relationship with Google to take advantage of all the tools that Google Cloud Platform has to meet our present and future needs.



-Contributed by Brooke Bryan, CTO, Just Develop It

Read More

App Engine 1.8.7 and a platform that is great for game developers

This week, we are highlighting the great apps that game developers have built on Google Cloud Platform. App Engine in particular is used for a variety of game types, ranging from casual mobile and web games to real-time games on the web. Check out this paper, which describes an implementation that uses Google Compute Engine for the backend and App Engine for the frontend of a real-time game. For the rest of our gaming content, including technical solutions articles, customer stories and videos, stay tuned to the blog or follow us on Google+ and Twitter.



Adding to these highlights, you can now download the SDK for App Engine 1.8.7. To see what’s new, please refer to the release notes.



-Posted by Chris Ramsdale, Product Manager

Read More

Google App Engine helps MAG Interactive handle explosive growth of word game, Ruzzle

Today we hear from Daniel Hasselberg, co-founder and chief executive officer of mobile game development company, MAG Interactive, based in Stockholm, Sweden. MAG Interactive produces some of the most popular games in the world, including Ruzzle, which has more than 45 million players in 142 different countries.



When we launched our word game Ruzzle in 2012, we had no idea it would become an international sensation almost overnight. We initially promoted the game only to our family and friends, but within two weeks of our launch, Ruzzle was the No.1 game on the Swedish App Store.



I believe if we hadn’t used Google App Engine to build the backend of Ruzzle, we wouldn’t have been able to scale fast enough with our own servers, which would have killed the app in the marketplace. There were about a million downloads of Ruzzle per month in the Nordic region, Holland, Spain and Italy through 2012. As we refined the game’s social integration through channels like Facebook and Twitter, we grew rapidly in Italy and the United States. In 2013, Ruzzle became the No. 1 game download on Google Play and the App Store in Italy, Sweden, the United States and many other countries.



Things were especially crazy at the end of last year. We were seeing about 700,000 new players each day from December 2012 through January 2013. We added 20 million users in a single month! It was incredible to see App Engine scale – and just keep on working – as we grew from about 5 million players to 25 million players in just a few weeks.



Our decision to use App Engine as the platform for Ruzzle and our new game, QuizCross, was strategic. Some of us at MAG Interactive helped develop the server platform for one of the most popular music download services in the Nordic region, so we knew about the challenges of having to scale quickly. While we didn’t anticipate Ruzzle’s popularity, we did recognize even before creating the game that we could face scaling problems if we were successful. So we decided from day one to use a cloud solution.



We looked at Amazon’s platform but preferred Google’s approach to cloud solutions. Google’s scalability was an important factor in our decision, but we also appreciated the company’s transparent pricing. The more efficient we became with App Engine, the less we paid.



The Google Cloud Platform team has been great to work with, as well. They are very supportive and appreciate our feedback. The technical support experts at Google are amazing, too – very hands-on. They know the platform extremely well and can help us work through any challenge.



We’re also using Google BigQuery for business intelligence. We track millions of events in the game every day so we know what users are doing – or not doing – and how we should improve the experience. We really like that we can throw enormous amounts of data at BigQuery, and it still performs. It only takes a few seconds to get results, and there are no scaling issues. It’s also easy to use. We have just one data analyst doing all the work with BigQuery but could probably use more people. If there are a few brilliant data mining experts out there who can imagine a future in Stockholm, please give us a call!



One thing we’ve learned from our BigQuery analysis is that the more users play Ruzzle, the more they improve their skills. New players typically find about 18 words in the two-minute time frame they’re given. After they play 100 games, they can find about 50 words, on average. I think that tracking player improvement is what keeps people playing and has helped to make Ruzzle so popular.



BigQuery offers our company a lot of insight into the use of our games and how we can improve them. We’re looking forward to expanding our relationship with Google as App Engine and Cloud Platform evolves.



-Contributed by Daniel Hasselberg, CEO, MAG Interactive

Read More

Applibot builds social games on Google Cloud Platform

Around the world, developers are taking advantage of Google Cloud Platform’s scale and speed to develop games. In this video, we talk to Koki Ukita, Executive Vice President of Applibot, about how Google Cloud Platform has allowed the company to remain “user focused.”



In Japan, Applibot uses Google Cloud Platform to build and deploy their social and mobile games. With millions of downloads, these are among the most popular apps on the Google Play Store in Japan.









-Posted by Benjamin Bechtolsheim, Product Marketing Manager

Read More

How to build scalable mobile games on Google Cloud Platform

You launch your mobile game and get a million downloads in a few days. Your dream is coming true! But your success is bittersweet, because your servers struggle to keep up with demand and your users, frustrated by timeouts, start publishing negative reviews while you scramble to increase server capacity. It doesn't have to be that way. The platform you choose to build on can help determine whether your launch is successful.



Using Google Cloud Platform can help you build an application that scales seamlessly from hundreds to millions of users. We have published two open source games, along with a technical paper and reference architecture on developing mobile games on Google Cloud Platform. Here's a preview of the reference architecture:

The first open source game is Griddler, which allows you to solve riddles against the clock. It demonstrates how to build a casual mobile game with both single-player and multi-player modes, invite friends using push notifications, store data in a NoSQL datastore and manage some basic game statistics.

The full source code for Griddler is published on GitHub: Java game backend, Android client with iOS client.



The second mobile game sample is Cloud Adventure, a text-based adventure game that takes after the tradition of Colossal Cave, Zork and other classics. It showcases a few interesting scenarios, such as a pre and postgame lobby for multiple players, unique nickname selection and saving points in the game.

The full source code for Cloud Adventures is also published on GitHub: Java game backend and Android client.



Both of these games are ready for you to download, deploy and play. You can explore the source code and extend them too, such as adding a visual clue to each riddle in Griddler.



There are already many successful games powered by Google Cloud Platform, and we can’t wait to hear about your success. Let us know what you think in the comments.



- Posted by Grzegorz Gogolowicz, Solutions Architect

Read More

Kaplan builds online education platform KAPx with Google App Engine

Today's guest post is from Charlie Ward, VP of Platform Development at Kaplan Inc., a global educational services company that operates in 170 countries and serves one million students annually. Charlie describes why Kaplan chose Google App Engine as the foundation for a scalable, flexible, and robust education platform that allows anyone to easily deliver learning content in both live and recorded formats.



The Drivers for Change

Ongoing disruptions in the Higher Education space -- including open courses, free content, regulatory pressures and protests over the increasing cost of education -- have spurred students to look for alternative paths to achieving their education and career goals and have driven institutions to look for innovative ways to lower the cost of education.



In response, Kaplan launched the KAPx initiative, whose original objective was to develop a scalable and robust platform for delivering content effectively to large audiences, using only low-cost and open components. Initial prototypes relied on traditional development technologies or required significant investments in infrastructure. When developers began to load-test these prototypes, the team recognized that Google App Engine could scale faster and further than the other products being used. So the decision was made to port the whole thing to Google App Engine.



First impressions with Google App Engine

Our initial introduction to Google App Engine was a real-time commenting service that we designed around the Channel API. While load testing our prototypes, we immediately saw that Google App Engine could scale faster and further than the rest of our solution. At that point we decided to port the entire application to the Google App Engine platform.



Using HTML templates, the NoSQL datastore, and memcache for dynamic web content, three engineers with zero python experience were able to port the application and go from prototype to production in just three weeks. The team also leveraged task queues to manage message distribution and the channel API for real-time message delivery. It was App Engine’s tight integration of these components and associated Python libraries that simplified typically tedious tasks and allowed us to flatten out the learning curve.

Adding on Google Hangouts & Google Apps

With Google App Engine powering the scalable foundation for KAPx, we leveraged Google Hangouts on Air for live streaming video and built a Hangouts App for the presenter to interact with participants. We also turned to Google Apps -- Docs, Presentations, and Drawings -- and the Google Drive Picker API to allow the presenter to select, share, and incorporate lecture materials into the live experience

App Engine provided us with the tools to rapidly go from concept to production at lightning speed, without the overhead of managing a traditional infrastructure or the need to wire up disparate technologies. After starting out with live lecture delivery on KAPx, the flexibility and ease of development on Google App Engine has allowed us to expand quickly, tackling structured learning with our first set of open courses for Kaplan University and a completely new online institution, Mount Washington College.



Tips for getting started with Google App Engine

1. There are many excellent examples of App Engine solutions in the GoogleCloudPlatform repository on GitHub to get you started.


2. Use webapp2 and the jinja2 template engine for serving up content unless you need something heavier. This lightweight but flexible combination is simple to use and provides an easy way to deliver MVC style HTML templates as well as raw data for RESTful APIs.


3. Use the NDB datastore libraries to get memcache for free. NDB was experimental when we first launched our platform, so we went with GAE’s DB datastore having to write our own handlers for front loading DB queries with memcache. With NDB, this is provided for you, dramatically improving performance and shortening development time.


4. Plan ahead if you are going to need to aggregate data (join, counts, etc). Operations that you would typically rely on in a relational database (joins, groupings and aggregate functions) are not as straightforward when working with the NoSQL DB or NDB datastore. You’ll want to plan out your models with the appropriate keys / properties to support aggregation in code foregoing joins and grouping when possible.


5. When relying heavily on the built in datastore and memcache implementations, you’ll want to understand their upper limits and design around them. There are excellent articles available on eventual consistency, sharding data, reducing datastore contention, async task queues, and MapReduce in the developers.google.com/appengine site that can provide you with techniques to work within the boundries of service quotas and the constraints of a highly scalable environment.


6. Invest time upfront to set up automated extracts of your log data to Google BigQuery on a regular basis. This will pay huge dividends the first time you need to track down a bug in production, as the dashboard tools out of the box are cumbersome to work with at scale.


7. Use the --datastore_consistency_policy=random option on dev_appserver.py when developing locally to identify potential problems with delayed datastore writes before deploying.

-Contributed by Charlie Ward, VP of Platform Development, Kaplan

Read More