Get started with Google Cloud Datastore - a fast, powerful, NoSQL database

At Google I/O, we announced Google Cloud Datastore, a fully managed solution for storing non-relational data. Based on the popular Google App Engine High Replication Datastore (HRD), Cloud Datastore provides a schemaless, non-relational datastore with the same accessibility of Google Cloud Storage and Google Cloud SQL.



Cloud Datastore builds off the strong growth and performance of HRD, which has over 1PB of data stored, 4.5 trillion transactions per month and a 99.95% uptime. It also comes with the following features:

• Built-in query support: near SQL functionality that allows you to search, sort and filter across multiple indexes that are automatically maintained 


• ACID transactions: data consistency (both Strong and Eventual) that spans multiple replicas and requests 


• Automatic scaling: built on top of Google’s BigTable infrastructure, the Cloud Datastore will automatically scale with your data 


• High availability: by utilizing Google’s underlying Megastore service, the Cloud Datastore ensures that data is replicated across multiple datacenters and is highly available 


• Local development environment: the Cloud Datastore SDK provides a full-featured local environment that allows you to develop, iterate and manage your Cloud Datastore instances efficiently 


• Free to get started: 50k read & write operations, 200 indexes, and 1GB of stored data for free per month  

Getting started with Cloud Datastore 



To get started, head over to the Google Cloud Console and create a new project. After supplying a few pieces of information you will have a Cloud Project that has the Cloud Datastore enabled by default. For this post we’ll use the project ID cloud-demo.

With the project created and the Cloud Datastore enabled, we’ll need to download the Cloud Datastore client library. Once extracted, it’s time to start writing some code. For the sake of this post, we’ll focus on accessing the Cloud Datastore from a Python application running on a Compute Engine VM (which is also now in Preview). We’ll assume that you’ve already created a new VM instance.

import googledatastore as datastore

def main()
  writeEntity()
  readEntity()

Next include writeEntity() and readEntity() functions:

def WriteEntity():
  req = datastore.BlindWriteRequest()
  entity = req.mutation.upsert.add()
  path = entity.key.path_element.add()
  path.kind = 'Greeting'
  path.name = 'foo'
  message = entity.property.add()
  message.name = 'message'
  value = message.value.add()
  value.string_value = 'to the cloud and beyond!'
  try:
    datastore.blind_write(req)
  except datastore.RPCError as e:
    # remember to do something useful with the exception
    pass

def ReadEntity(): 
  req = datastore.LookupRequest()
  key = req.key.add()
  path = key.path_element.add()
  path.kind = 'Greeting0'
  path.name = 'foo0'
  try:
    resp = datastore.lookup(req)
    return resp
  except datastore.RPCError as e:
    # remember to do something useful with the exception
    pass

First create a new file called “demo.py”. Inside demo.py, we’ll add code to write and then read an entity from the Cloud Datastore.  Finally we can update main() to print out the property values within the fetched entity:

def main():
  writeEntity();
  resp = readEntity();

  entity = resp.found[0].entity
  for p in entity.property:
    print 'Entity property name: %s', p.name
    v = p.value[0]
    print 'Entity property value: %s', v.string_value

Before we can run this code we need to tell the client library which Cloud Datastore instance we would like to use. This is done by exporting the following environment variable:

~$ export DATASTORE_DATASET cloud-datastore-demo

Finally we’re able to run the application by simply issuing the following:

~$ python demo.py

Besides the output that we see in console window, we’re also able to monitor our interactions within the Cloud Console. By navigating back to Cloud Console, selecting our cloud-datastore-demo project, and then selecting the Cloud Datastore we’re taken to our instance’s dashboard page that includes number of entities, properties, and property types, as well as index management, ad-hoc query support and breakdown of stored data.

And that’s really just the beginning. To fully harness the features and functionality that the Cloud Datastore offers, be sure to check out the larger Getting Started guide and the Cloud Datastore documentation.



Cloud Datastore is the latest addition to the Cloud Platform storage family, joining Cloud Storage for storing blob data, Cloud SQL for storing relational data, and Persistent Disk for storing block data. All fully managed so that you can focus on creating amazing solutions and leave the rest to us.



And while this is a Preview Release, the team is off to a great start. As we move the service towards General Availability we’re looking forward to improving JSON support, more deeply integrating with the Cloud Console, streamlining our billing and driving every bit of performance that we can out of the API and underlying service.



Happy coding!



 -Posted by Chris Ramsdale, Product Manager