Cloud World

  • Subscribe to our RSS feed.
  • Twitter
  • StumbleUpon
  • Reddit
  • Facebook
  • Digg

Thursday, 7 November 2013

Just Develop IT Migrates Petabytes of Data to Google Cloud Storage

Posted on 04:00 by Unknown
We interviewed Brooke Bryan, CTO of Just Develop It Ltd (JDI), one of the leaders in device data backup with popular applications including My PC Backup. In early 2013, the company migrated 5 petabytes of data from Amazon S3 to Google Cloud Storage. Since then, JDI’s business has expanded their use to over 10 petabytes at a rate of 800 terabytes a month.



What does Just Develop IT do?



Just Develop It (JDI) was founded with the mission of developing online services to help consumers across the Internet. Today, JDI has numerous companies in its portfolio, ranging from Skylark Golf & Country Club, a beautiful and award winning golf and country club in the south of Hamphire, England, to Just Cloud, a service that backs up music, videos, documents, emails, iTunes, photos and pictures, and stores them in the cloud.



Why did JDI want to migrate to the cloud?



Maintaining physical storage servers in the data center was a nightmare and the downtime was unbearable. As our portfolio of services grew, especially the size and scale of our backup products, we knew that a move to cloud infrastructure was imminent. This move to the cloud was very strategic for our growth as a company because the cloud is much easier to manage and required far fewer system administrators than our old storage servers. We initially selected Amazon S3, but unfortunately, the migration took nearly four months due to the speed of the physical storage servers.



Why did you switch from Amazon S3 to Google Cloud Storage?



From a product perspective, we were happy with Amazon S3. However, the customer service left a lot to be desired, and we found that we were locked into a contract without flexibility.



We researched suitable replacements and decided to take a good look at Google. We discovered that Google Cloud Storage was not only a great product but also affordable. Google was willing to work closely with JDI during the migration and partner with us long-term to ensure our needs were being met. We were also very impressed from a security standpoint as Google storage systems offer full encryption ensuring that our users’ data is secure at all times, a feature that is essential to our business.



Initially, we tested switching applications in developer environments and conducted testing to ensure that Google could handle the load. After this was confirmed, we began to migrate over five petabytes of data to Google Cloud Storage.



What challenges did the migration present?









Because our users rely on our products for 100% uptime, we wanted to execute the move to Google Cloud Storage within a short timeframe and in a way that would go unnoticed to users. This presented a few challenges:


  1. We needed high network bandwidth and parallel data transfer mechanisms to ensure that the migration could be completed as soon as possible.

  2. Since JDI has multiple storage/backup products, the process had to be invisible to our users, as a disruption of service would directly affect the confidence in our products.

  3. We had to keep costs low during the process, which raised two issues: We paid twice the fees to store objects in both Amazon and Google systems in order to ensure that services would not be disrupted; and if an object was copied multiple times, we incurred unnecessary AWS egress network costs.


How did Google meet these challenges?



Google went above and beyond our previous storage providers. Google’s network bandwidth proved to be reliable. Also, we were able to move all of our storage and workloads, such as versioning and file composition, into Google Compute Engine to take advantage of Google’s networks and compute resources.



From a migration and testing perspective, everything was completed within 2-3 weeks. The implementation took only 24 hours and was available in 3-4 days. Our users did not notice a disruption, and the quick turnaround ensured that migration costs were low. We now host over 10 petabytes of data in Google Cloud Storage, and our data is growing at the rate of over 800 terabytes every month.



How has your experience with Google been since the implementation?



We view Google as a strategic long-term partner based on our experience with Google Compute Engine and Google Cloud Storage. The company’s technical, customer relations and support teams have met our every need, and if there is a problem, we know they will work quickly to resolve it.



We are looking forward to expanding our relationship with Google to take advantage of all the tools that Google Cloud Platform has to meet our present and future needs.



-Contributed by Brooke Bryan, CTO, Just Develop It
Email ThisBlogThis!Share to XShare to Facebook
Posted in cloud storage, customer | No comments
Newer Post Older Post Home

0 comments:

Post a Comment

Subscribe to: Post Comments (Atom)

Popular Posts

  • A Day in the Cloud, new articles on scaling, and fresh open source projects for App Engine
    The latest release of Python SDK 1.2.3, which introduced the Task Queue API and integrated support for Django 1.0, may have received a lot ...
  • Cube Slam meets Google Cloud Platform
    The Google Creative Lab team has built another fun Chrome Experiment called Cube Slam . Cube Slam connects players into a three dimensional,...
  • GDC’13: Learn how to build games on Google Cloud Platform
    At the Game Developers Conference last month, we held a day of sessions showing developers how to take advantage of Google Cloud Platform ...
  • New in Google Cloud Storage: auto-delete, regional buckets and faster uploads
    We’ve launched new features in Google Cloud Storage that make it easier to manage objects, and faster to access and upload data. With a tiny...
  • DataStax Enterprise feels right at home in Google Compute Engine
    Today’s guest post comes from Martin Van Ryswyk, Vice President of Engineering at DataStax. The cloud promises many things for database user...
  • The BugSense hybrid app: experiences using Clojure on Google App Engine
    Today’s post comes to us from Jon Vlachogiannis and Panos Papadopoulos , founders of BugSense , a mobile error analytics service. We hope y...
  • Outfit 7’s Talking Friends built on Google App Engine, recently hit one billion downloads
    Today’s guest blogger is Igor Lautar, senior director of technology at Outfit7 (Ekipa2 subsidiary), one of the fastest-growing media enterta...
  • Jump-start your data pipelining into Google BigQuery
    Once you get your data into Google BigQuery , you don’t have to worry about running out of machine capacity, because you use Google’s machin...
  • Development in the Cloud with Codenvy and Google Cloud Platform
    Today’s post is from Tyler Jewell, CEO at Codenvy. In this post, Tyler looks at how you can leverage cloud based development tools to build ...
  • Get started at no cost with a faster, larger Google Cloud SQL database
    You want your application to be fast. Databases play a big role in your overall application speed and end-users’ experience. Reads and write...

Categories

  • 1.1.2
  • agile
  • android
  • Announcements
  • api
  • app engine
  • appengine
  • batch
  • bicycle
  • bigquery
  • canoe
  • casestudy
  • cloud
  • Cloud Datastore
  • cloud endpoints
  • cloud sql
  • cloud storage
  • cloud-storage
  • community
  • Compute Engine
  • conferences
  • customer
  • datastore
  • delete
  • developer days
  • developer-insights
  • devfests
  • django
  • email
  • entity group
  • events
  • getting started
  • google
  • googlenew
  • gps
  • green
  • Guest Blog
  • hadoop
  • html5
  • index
  • io2010
  • IO2013
  • java
  • kaazing
  • location
  • mapreduce
  • norex
  • open source
  • partner
  • payment
  • paypal
  • pipeline
  • put
  • python
  • rental
  • research project
  • solutions
  • support
  • sustainability
  • taskqueue
  • technical
  • toolkit
  • twilio
  • video
  • websockets
  • workflows

Blog Archive

  • ▼  2013 (143)
    • ►  December (33)
    • ▼  November (15)
      • Compute Engine Load Balancing hits 1 million reque...
      • Dedicated memcache is now generally available in A...
      • Development tips for Google Cloud Endpoints for An...
      • Offline Disk Import and the OmNomNom Machine
      • International Offline Disk Import now available wi...
      • Google App Engine integration features in IntelliJ...
      • Connecting mobile developers to the cloud with Goo...
      • Rovio, Freshplanet, Pocket Gems and others talk ab...
      • Build amazing real time games on Google Cloud Plat...
      • Just Develop IT Migrates Petabytes of Data to Goog...
      • App Engine 1.8.7 and a platform that is great for ...
      • Google App Engine helps MAG Interactive handle exp...
      • Applibot builds social games on Google Cloud Platform
      • How to build scalable mobile games on Google Cloud...
      • Kaplan builds online education platform KAPx with ...
    • ►  October (17)
    • ►  September (13)
    • ►  August (4)
    • ►  July (15)
    • ►  June (12)
    • ►  May (15)
    • ►  April (4)
    • ►  March (4)
    • ►  February (9)
    • ►  January (2)
  • ►  2012 (43)
    • ►  December (2)
    • ►  November (2)
    • ►  October (8)
    • ►  September (2)
    • ►  August (3)
    • ►  July (4)
    • ►  June (2)
    • ►  May (3)
    • ►  April (4)
    • ►  March (5)
    • ►  February (3)
    • ►  January (5)
  • ►  2011 (46)
    • ►  December (3)
    • ►  November (4)
    • ►  October (4)
    • ►  September (5)
    • ►  August (3)
    • ►  July (4)
    • ►  June (3)
    • ►  May (8)
    • ►  April (2)
    • ►  March (5)
    • ►  February (3)
    • ►  January (2)
  • ►  2010 (38)
    • ►  December (2)
    • ►  October (2)
    • ►  September (1)
    • ►  August (5)
    • ►  July (5)
    • ►  June (6)
    • ►  May (3)
    • ►  April (5)
    • ►  March (5)
    • ►  February (2)
    • ►  January (2)
  • ►  2009 (47)
    • ►  December (4)
    • ►  November (3)
    • ►  October (6)
    • ►  September (5)
    • ►  August (3)
    • ►  July (3)
    • ►  June (4)
    • ►  May (3)
    • ►  April (5)
    • ►  March (3)
    • ►  February (7)
    • ►  January (1)
  • ►  2008 (46)
    • ►  December (4)
    • ►  November (3)
    • ►  October (10)
    • ►  September (5)
    • ►  August (6)
    • ►  July (4)
    • ►  June (2)
    • ►  May (5)
    • ►  April (7)
Powered by Blogger.

About Me

Unknown
View my complete profile