• About This Podcast
  • Hosts
  • Channels
  • Sponsorship
  • Podroll
  • Feedback

Storage Unpacked Podcast

A weekly podcast on deploying and managing enterprise storage and data

You are here: Home / Guest Speakers / #55 – Storage for Hyperscalers

#55 – Storage for Hyperscalers

29 June 2018 by Chris Evans 9 Comments

#55 – Storage for Hyperscalers

This week we talk to Mark Carlson, co-chair of the SNIA Technical Council, about the storage needs of hyperscalers.  Mark defines hyperscalers as those companies opening multiple data centres a year, most notably Amazon, Microsoft Azure, Google and Facebook in the US and Baidu, Alibaba and Tencent in China.  These vendors are deploying petabytes of storage a year, with specific requirements on storage media.  The hyperscaler applications have issues with HDD and SSD performance characteristics, such as tail latency and the effects of garbage collection.  As a result, drive manufacturers are building in new features to meet the needs of these companies.

What exactly are the issues?  The conversation covers some of the typical problems, like reusing existing partially failed media, the problems caused by tail latency, write amplification and why non-deterministic latency is such an issue.  Some vendor solutions include Depop, Fast Fail and Open Channel.  Not sure what these are?  Listen in to find out!

In the podcast, Mark mentions a white paper on hyperscaler storage.  You can find it here (PDF).  The SNIA YouTube channel can be found here.  Mark mentions his presentation at Tech Field Day, which can be found here.

Elapsed Time:  00:27:54

Timeline

  • 00:00:00 – Intro
  • 00:01:30 – Who do we mean by hyper-scalers?
  • 00:02:30 – ODM (Not OEM) – Original Design Manufacturers.
  • 00:04:00 – Aren’t HPE and Dell selling to hyper-scalers?
  • 00:05:30 – Why do hyper-scalers need different storage media?
  • 00:07:00 – Differences of approach to resiliency compared to Enterprise.
  • 00:09:00 – Don’t we want to have partial device failures?
  • 00:10:20 – Depop – depopulate and reset drive to factory settings.
  • 00:11:30 – Back to tail latency – extended response times.
  • 00:12:30 – NVM Sets to logically partition a drive for each application.
  • 00:13:40 – Fast Fail for HDD reads.
  • 00:15:00 – putting drives into a deterministic window – putting maintenance off.
  • 00:16:30 – Open Channel approach – let the host do the maintenance work.
  • 00:17:30 –  How are changes introduced?  ECNs and Technical Proposals.
  • 00:22:00 – Will hyper-scaler features filter down to the enterprise?
  • 00:26:00 – Wrap Up

Mark’s Bio

Mark A. Carlson, Principal Engineer, Industry Standards at Toshiba, has more than 35 years of experience with Networking and Storage development and more than 20 years experience with Java technology. Mark was one of the authors of the CDMI Cloud Storage standard. He has spoken at numerous industry forums and events. He is the co-chair of the SNIA Cloud Storage and Object Drive technical working groups, and serves as co-chair of the SNIA Technical Council.

Copyright (c) 2016-2018 Storage Unpacked.  No reproduction or re-use without permission. Podcast Episode HIJ4.
 

 

Podcast: Play in new window | Download

Subscribe: Apple Podcasts | Android | Google Podcasts | RSS

(Visited 747 times, 1 visits today)

Filed Under: Guest Speakers Tagged With: HDD, hyperscalers, Mark Carlson, SNIA, SSD

Subscribe by email

Subscribe by Podcatcher

on Apple Podcastson Androidon Google Podcastsvia RSS
Listen to Stitcher

Pages

  • About This Podcast
  • Channels
  • Feedback
  • Hosts
  • Podroll
  • Sponsorship
  • Subscribe

Recent Posts

  • #189 – The Quiet Success of Software-Defined Storage 22 January 2021
  • #188 – Is Intel Optane Ready for Primetime? 15 January 2021
  • #187 – End of the Year Show 2020 – Part Two 30 December 2020
  • #186 – End of the Year Show 2020 – Part One 11 December 2020
  • #185 – Pure-as-a-Service 2.0 (Sponsored) 4 December 2020

Post Archives

  • January 2021
  • December 2020
  • November 2020
  • October 2020
  • September 2020
  • August 2020
  • July 2020
  • June 2020
  • May 2020
  • April 2020
  • March 2020
  • February 2020
  • January 2020
  • December 2019
  • November 2019
  • October 2019
  • September 2019
  • August 2019
  • July 2019
  • June 2019
  • May 2019
  • April 2019
  • March 2019
  • February 2019
  • January 2019
  • December 2018
  • November 2018
  • October 2018
  • September 2018
  • August 2018
  • July 2018
  • June 2018
  • May 2018
  • April 2018
  • March 2018
  • February 2018
  • January 2018
  • December 2017
  • November 2017
  • October 2017
  • September 2017
  • June 2017
  • May 2017
  • April 2017
  • March 2017
  • February 2017
  • January 2017
  • December 2016
  • November 2016
  • October 2016

Copyright © 2021 · News Pro Theme on Genesis Framework · WordPress · Log in