Automatically Process Cloud Storage Uploads with Change Notifications

Do your customers upload files to Google Cloud Storage for your applications to process? For example, a photo app may want to create thumbnails of new images as soon as they are uploaded. Normally you would have to poll for updated objects which can be a resource waste or cause you to react slowly. Most times writing and deploying custom scripts to trigger your application is cumbersome.



Today, we're releasing object change notification as a preview feature, allowing you to watch your Google Cloud Storage buckets for new, modified, or deleted objects with a webhook you provide. Now your application can be automatically triggered when an important change happens and start processing data immediately. We've also updated gsutil with a notifyconfig command. A Google App Engine webhook can be as simple as the following:

class MainPage(webapp2.RequestHandler):
  def post(self):
    resource_state = self.request.headers['X-Goog-Resource-State']
    if resource_state == 'sync':
      # Initial message that the notification channel is active.
      pass
    elif resource_state == ‘exists’:
      an_object = json.loads(self.request.body)
      bucket = an_object['bucket']
      object_name = an_object['name']
      # Take action!
    elif resource_state == ‘not_exists’:
      # Object was deleted.
      pass

We're also releasing an update to the Google Cloud Storage JSON API, bringing it into parity with our existing XML API, including exposing new methods such as Copy and Compose. As a part of this release, we are making the API available to everyone without requiring an invitation.



Enjoy, and as always, we watch StackOverflow.



- Posted by Dave Barth, Product Manager