Cloud World

  • Subscribe to our RSS feed.
  • Twitter
  • StumbleUpon
  • Reddit
  • Facebook
  • Digg

Tuesday, 16 February 2010

Scalability means flexibility

Posted on 14:44 by Unknown

From the beginning we've worked hard to ensure that building and maintaining applications on Google App Engine is as easy as possible and that those applications can be made to scale very well. Scalability can mean different things to different people though. For us, platform scalability (today) means efficiently serving 250 million daily page views for 250,000 developers. For you, application scalability means being able to serve a large number of requests to your web site when you need it.






But serving all of that traffic is just half the challenge. Even for applications with very predictable traffic patterns it's often impossible to know how much traffic your site will see at any given time in the future. You need flexibility and a platform which will automatically adjust to your needs, minute by minute. This is especially true for sites that are used in conjunction with large media events. We've been working with one such company and we'd like to share their scalability story with you today.






Gigya’s technology enables their customers to connect seamlessly to popular social network platforms for user authentication, sharing and engagement. One product feature is a social live chat widget used by their customers to accompany live-streamed events. When I spoke with Gigya's Raviv Pavel, VP Research and Development, he told me, "Although we typically host all our services in-house, on our own infrastructure, we felt that GAE would be a better fit for the live chat feature because of its unique traffic pattern, which is characterized by very low traffic most of the time with very high bursts during high profile events."






Below is a graph from the January 22nd Hope for Haiti Now: A Global Benefit for Earthquake Relief during which Gigya had deployed their social gadget to multiple web properties. The vertical scale shows number of requests or queries per second, which peaked at around 1600 QPS. Each of their events has its own unique characteristics. So far we've seen Bactrian camels, Dromedary camels, parts of the Rocky Mountains and this next graph which vaguely resembles a submarine poking through the polar icecap.













Gigya wins because Google App Engine maintains the infrastructure needed to support those traffic bursts for the live chat widget. And because of App Engine's granular billing model, they only pay for the resources they use. When Gigya's live chat sees reduced traffic, we make sure our machines are busy serving other requests.






If you have an interesting App Engine story you'd like to share, let us know. We'd like to hear about it.






Posted by Fred Sauer, App Engine team

Read More
Posted in | No comments

Wednesday, 10 February 2010

App Engine SDK 1.3.1, Including Major Improvements to Datastore!

Posted on 15:09 by Unknown
We are excited to announce the release of version 1.3.1 of the App Engine SDK for both Python and Java. While this release contains plenty of new features and fixes, we've concentrated on using our very first SDK release of 2010 to improve the heart of many App Engine applications: the Datastore.



Here are the three major improvements that 1.3.1 has in store for datastore users:
  • Datastore Query Cursors - Cursors allow applications to save and 'bookmark' their progress through a query, so that it can be resumed later. This works great in combination with paging URLs, as well as processing in the Task Queue API, but there are many other uses. Watch for an upcoming blog post that explores Cursors in the near future. They're also really handy in the context of the next change...
  • No more 1000 result limit - That's right: with addition of Cursors and the culmination of many smaller Datastore stability and performance improvements over the last few months, we're now confident enough to remove the maximum result limit altogether. Whether you're doing a fetch, iterating, or using a Cursor, there's no limits on the number of results.
  • Reduced error rate with Automatic Datastore Retries - We've heard a lot of feedback that you don't want to deal with the Datastore's sporadic errors. In response, App Engine now automatically retries all datastore calls (with the exception of transaction commits) when your applications encounters a datastore error caused by being unable to reach Bigtable. Datastore retries automatically builds in what many of you have been doing in your code already, and our tests have shown it drastically reduces the number of errors your application experiences (by up to 3-4x error reduction for puts, 10-30x for gets).
But even with our focus on the Datastore, we were able to sneak in a quite a number of other new goodies into 1.3.1 as well:



For Python, we've included the AppStats RPC instrumentation library. AppStats lets users easily profile the performance of calls from their app to the App Engine backend services to identify and isolate issues such as ineffective caching, bottlenecks, and redundant RPC calls in their app. (A Java version is in beta testing now.)



And for Java, we've included a comprehensive new unit-testing framework for your App Engine apps. The unit-testing framework enables you to test your application code in a natural, fully supported manner, and also allows you to integrate your App Engine apps into other existing testing and automation frameworks.



The list of changes in 1.3.1 goes on and on (Custom Admin Console pages! Support for wildcard domain mappings! Java precompilation on by default for all applications!), so make sure to download the new version and read our release notes for the complete list (Python, Java).



Posted by the App Engine Team
Read More
Posted in | No comments
Newer Posts Older Posts Home
Subscribe to: Posts (Atom)

Popular Posts

  • Tutorial: Adding a cloud backend to your application with Android Studio
    Android Studio lets you easily add a cloud backend to your application, right from your IDE. A backend allows you to implement functionality...
  • A Day in the Cloud, new articles on scaling, and fresh open source projects for App Engine
    The latest release of Python SDK 1.2.3, which introduced the Task Queue API and integrated support for Django 1.0, may have received a lot ...
  • New Admin Console Release
    Posted by Marzia Niccolai, App Engine Team Today we've released some new features in our Admin Console to make it easier for you to mana...
  • JPA/JDO Java Persistence Tips - The Year In Review
    If you’re developing a Java application on App Engine you probably already know that you can use JPA and JDO, both standard Java persistence...
  • The new Cloud Console: designed for developers
    In June, we unveiled the new Google Cloud Console , bringing together all of Google’s APIs, Services, and Infrastructure in a single interfa...
  • Best practices for App Engine: memcache and eventual vs. strong consistency
    We have published two new articles about best practices for App Engine. Are you aware of the best ways to keep Memcache and Datastore in syn...
  • Pushing Updates with the Channel API
    If you've been watching Best Buy closely, you already know that Best Buy is constantly trying to come up with new and creative ways to...
  • Outfit 7’s Talking Friends built on Google App Engine, recently hit one billion downloads
    Today’s guest blogger is Igor Lautar, senior director of technology at Outfit7 (Ekipa2 subsidiary), one of the fastest-growing media enterta...
  • Bridging Mobile Backend as a Service to Enterprise Systems with Google App Engine and Kinvey
    The following post was contributed by Ivan Stoyanov , VP of Engineering for Kinvey, a mobile Backend as a Service provider and Google Cloud ...
  • Easy Performance Profiling with Appstats
    Since App Engine debuted 2 years ago, we’ve written extensively about best practices for writing scalable apps on App Engine. We make writ...

Categories

  • 1.1.2
  • agile
  • android
  • Announcements
  • api
  • app engine
  • appengine
  • batch
  • bicycle
  • bigquery
  • canoe
  • casestudy
  • cloud
  • Cloud Datastore
  • cloud endpoints
  • cloud sql
  • cloud storage
  • cloud-storage
  • community
  • Compute Engine
  • conferences
  • customer
  • datastore
  • delete
  • developer days
  • developer-insights
  • devfests
  • django
  • email
  • entity group
  • events
  • getting started
  • google
  • googlenew
  • gps
  • green
  • Guest Blog
  • hadoop
  • html5
  • index
  • io2010
  • IO2013
  • java
  • kaazing
  • location
  • mapreduce
  • norex
  • open source
  • partner
  • payment
  • paypal
  • pipeline
  • put
  • python
  • rental
  • research project
  • solutions
  • support
  • sustainability
  • taskqueue
  • technical
  • toolkit
  • twilio
  • video
  • websockets
  • workflows

Blog Archive

  • ►  2013 (143)
    • ►  December (33)
    • ►  November (15)
    • ►  October (17)
    • ►  September (13)
    • ►  August (4)
    • ►  July (15)
    • ►  June (12)
    • ►  May (15)
    • ►  April (4)
    • ►  March (4)
    • ►  February (9)
    • ►  January (2)
  • ►  2012 (43)
    • ►  December (2)
    • ►  November (2)
    • ►  October (8)
    • ►  September (2)
    • ►  August (3)
    • ►  July (4)
    • ►  June (2)
    • ►  May (3)
    • ►  April (4)
    • ►  March (5)
    • ►  February (3)
    • ►  January (5)
  • ►  2011 (46)
    • ►  December (3)
    • ►  November (4)
    • ►  October (4)
    • ►  September (5)
    • ►  August (3)
    • ►  July (4)
    • ►  June (3)
    • ►  May (8)
    • ►  April (2)
    • ►  March (5)
    • ►  February (3)
    • ►  January (2)
  • ▼  2010 (38)
    • ►  December (2)
    • ►  October (2)
    • ►  September (1)
    • ►  August (5)
    • ►  July (5)
    • ►  June (6)
    • ►  May (3)
    • ►  April (5)
    • ►  March (5)
    • ▼  February (2)
      • Scalability means flexibility
      • App Engine SDK 1.3.1, Including Major Improvements...
    • ►  January (2)
  • ►  2009 (47)
    • ►  December (4)
    • ►  November (3)
    • ►  October (6)
    • ►  September (5)
    • ►  August (3)
    • ►  July (3)
    • ►  June (4)
    • ►  May (3)
    • ►  April (5)
    • ►  March (3)
    • ►  February (7)
    • ►  January (1)
  • ►  2008 (46)
    • ►  December (4)
    • ►  November (3)
    • ►  October (10)
    • ►  September (5)
    • ►  August (6)
    • ►  July (4)
    • ►  June (2)
    • ►  May (5)
    • ►  April (7)
Powered by Blogger.

About Me

Unknown
View my complete profile