Cloud World

  • Subscribe to our RSS feed.
  • Twitter
  • StumbleUpon
  • Reddit
  • Facebook
  • Digg

Monday, 22 July 2013

New in Google Cloud Storage: auto-delete, regional buckets and faster uploads

Posted on 06:00 by Unknown
We’ve launched new features in Google Cloud Storage that make it easier to manage objects, and faster to access and upload data. With a tiny bit of upfront configuration, you can take advantage of these improvements with no changes to your application code — and we know that one thing better than improving your app is improving your app transparently!



Today we’re announcing:



Object Lifecycle Management - Configure auto-deletion policies for your objects

Regional Buckets - Granular location specifications to keep your data near your computation

gsutil - automatic parallel composite uploads - Faster uploads with gsutil



Object Lifecycle Management

Object Lifecycle Management allows you to define policies that allow Cloud Storage to automatically delete objects based on certain conditions. For example, you could configure a bucket so objects older than 365 days are deleted, or only keep the 3 most recent versions of objects in a versioned bucket. Once you have configured Lifecycle Management, the expected expiration time will be added to object metadata when possible, and all operations are logged in the access log.



Object Lifecycle Management can be used with Object Versioning to limit the number of older versions of your objects that are retained. This can help keep your apps cost-efficient while maintaining a level of protection against accidental data loss due to user application bugs or manual user errors.



Regional Buckets

Regional Buckets allow you to co-locate your Durable Reduced Availability data in the same region as your Google Compute Engine instances. Since Cloud Storage buckets and Compute Engine instances within a region share the same network fabric, this can reduce latency and increase bandwidth to your virtual machines, and may be particularly appropriate for data-intensive computations. You can still specify the less-granular United States or European datacenter locations if you'd like your data spread over multiple regions, which may be a better fit for content distribution use cases.



gsutil - Automatic Parallel Composite Uploads

Gsutil version 3.34 now automatically uploads large objects in parallel for higher throughput. Achieving maximum TCP throughput on most networks requires multiple connections, and this makes it easy and automatic. The support is built using Composite Objects. For details about temporary objects and a few caveats, see the Parallel Composite Uploads documentation. To get started, simply use 'gsutil cp' as usual. Large files are automatically uploaded in parallel.



We think there’s a little something here for everyone: If you’re managing temporary or versioned objects, running compute jobs over Cloud Storage data, or using gsutil to upload data, you’ll want to take advantage of these features right away. We hope you enjoy them!



-Posted by Brian Dorsey, Developer Programs Engineer
Email ThisBlogThis!Share to XShare to Facebook
Posted in | No comments
Newer Post Older Post Home

0 comments:

Post a Comment

Subscribe to: Post Comments (Atom)

Popular Posts

  • SaltStack for Google Compute Engine
    SaltStack was built for the performance, reliability, security and scale delivered by Google Compute Engine . Needless to say we are very e...
  • 2013 Year in review: making Google Compute Engine Generally Available
    2013 was a busy year for Google Cloud Platform. Watch this space: each day, a different Googler who works on Cloud Platform will be sharing ...
  • Google Compute Engine and App Engine give Evite freedom to expand services anytime
    Today’s guest blog comes from Dan Mesh, Vice President of Technology at Evite, the pioneer in online invitations and social planning. Evite ...
  • Why We Deployed Zencoder on Google Cloud Platform
    Today's guest post comes from Jon Dahl, VP of Encoding Services at Brightcove. Brightcove’s Zencoder transcodes millions of video and au...
  • International Offline Disk Import now available with new disk-upload centers in Japan, India and Switzerland
    Google Cloud Platform customers have told us that transferring large data sets (in the hundreds of terabytes and beyond) can be expensive a...
  • Google Compute Engine is now Generally Available with expanded OS support, transparent maintenance, and lower prices
    Google Cloud Platform gives developers the flexibility to architect applications with both managed and unmanaged services that run on Googl...
  • 2013 Year in review: bringing App Engine to the PHP community
    2013 was a busy year for Google Cloud Platform. Watch this space: each day, a different Googler who works on Cloud Platform will be sharing ...
  • Update on Datastore Auto IDs
    In the upcoming Google App Engine 1.8.1 release, the  Datastore default auto ID policy in production will switch to scattered IDs to impro...
  • 2013 Year in review: giving time back to developers
    2013 was a busy year for Google Cloud Platform. Watch this space: each day, a different Googler who works on Cloud Platform will be sharing ...
  • DataStax Enterprise feels right at home in Google Compute Engine
    Today’s guest post comes from Martin Van Ryswyk, Vice President of Engineering at DataStax. The cloud promises many things for database user...

Categories

  • 1.1.2
  • agile
  • android
  • Announcements
  • api
  • app engine
  • appengine
  • batch
  • bicycle
  • bigquery
  • canoe
  • casestudy
  • cloud
  • Cloud Datastore
  • cloud endpoints
  • cloud sql
  • cloud storage
  • cloud-storage
  • community
  • Compute Engine
  • conferences
  • customer
  • datastore
  • delete
  • developer days
  • developer-insights
  • devfests
  • django
  • email
  • entity group
  • events
  • getting started
  • google
  • googlenew
  • gps
  • green
  • Guest Blog
  • hadoop
  • html5
  • index
  • io2010
  • IO2013
  • java
  • kaazing
  • location
  • mapreduce
  • norex
  • open source
  • partner
  • payment
  • paypal
  • pipeline
  • put
  • python
  • rental
  • research project
  • solutions
  • support
  • sustainability
  • taskqueue
  • technical
  • toolkit
  • twilio
  • video
  • websockets
  • workflows

Blog Archive

  • ▼  2013 (143)
    • ►  December (33)
    • ►  November (15)
    • ►  October (17)
    • ►  September (13)
    • ►  August (4)
    • ▼  July (15)
      • How Safari Books Online uses Google BigQuery for b...
      • Multi-Channel Chat with Twilio and Google App Engine
      • Google App Engine: Hello World using Push-to-Deploy
      • SendGrid gives App Engine developers a simple way ...
      • Create dynamic web projects in Eclipse
      • New in Google Cloud Storage: auto-delete, regional...
      • Dedicated memcache preview now available on Google...
      • Google App Engine 1.8.2 released
      • Help us improve Google Cloud Platform by participa...
      • Get up and Running with Cassandra on Google Comput...
      • Lock-in, what lock-in?
      • Development in the Cloud with Codenvy and Google C...
      • See how to test drive Virtual Machines on Google C...
      • IT Operations Management for Google Compute Engine...
      • Google App Engine takes the pain out of sending iO...
    • ►  June (12)
    • ►  May (15)
    • ►  April (4)
    • ►  March (4)
    • ►  February (9)
    • ►  January (2)
  • ►  2012 (43)
    • ►  December (2)
    • ►  November (2)
    • ►  October (8)
    • ►  September (2)
    • ►  August (3)
    • ►  July (4)
    • ►  June (2)
    • ►  May (3)
    • ►  April (4)
    • ►  March (5)
    • ►  February (3)
    • ►  January (5)
  • ►  2011 (46)
    • ►  December (3)
    • ►  November (4)
    • ►  October (4)
    • ►  September (5)
    • ►  August (3)
    • ►  July (4)
    • ►  June (3)
    • ►  May (8)
    • ►  April (2)
    • ►  March (5)
    • ►  February (3)
    • ►  January (2)
  • ►  2010 (38)
    • ►  December (2)
    • ►  October (2)
    • ►  September (1)
    • ►  August (5)
    • ►  July (5)
    • ►  June (6)
    • ►  May (3)
    • ►  April (5)
    • ►  March (5)
    • ►  February (2)
    • ►  January (2)
  • ►  2009 (47)
    • ►  December (4)
    • ►  November (3)
    • ►  October (6)
    • ►  September (5)
    • ►  August (3)
    • ►  July (3)
    • ►  June (4)
    • ►  May (3)
    • ►  April (5)
    • ►  March (3)
    • ►  February (7)
    • ►  January (1)
  • ►  2008 (46)
    • ►  December (4)
    • ►  November (3)
    • ►  October (10)
    • ►  September (5)
    • ►  August (6)
    • ►  July (4)
    • ►  June (2)
    • ►  May (5)
    • ►  April (7)
Powered by Blogger.

About Me

Unknown
View my complete profile