• Skip to main content
  • Skip to primary sidebar
  • About This Podcast
  • Hosts
  • Channels
  • Sponsorship
  • Podroll
  • Feedback

Storage Unpacked

A weekly podcast on deploying and managing enterprise storage and data

Home » #58 – Storage Vendor Hero Numbers

#58 – Storage Vendor Hero Numbers

20 July 2018 by Chris Evans

This week, Chris Mellor is back and the team take a dive into the subject of storage vendor hero numbers.  What are hero numbers?  We’ve all seen them, they’re the huge performance figures quoted by all-flash storage vendors aimed at putting their products forward in the best light possible.

Are hero numbers believable or should we be looking at certified vendor benchmark testing as a guide to capability?  Do users even look at benchmark or hero numbers in the first place?  Could the whole exercise be a waste of time?  The conversation moves to talk about STAC (Securities Technologies Analysis Center) and whether more application specific benchmarks have greater value.  This includes the approach Pure Storage appears to have taken with FlashArray and FlashBlade, only quoting application-specific testing like that of AIRI.

As the discussion moves on, the team talk about more consistent deterministic latency (more on that here) and whether public cloud service providers will ever subject their service offerings to benchmark testing (TL;DR, probably not).  As a final wrap up, the team asks, what can customers actually do to ensure they are seeing the real truth behind benchmark claims.

Elapsed Time: 00:20:48

Timeline

  • 00:00:00 – Intro
  • 00:01:00 – What are vendor hero numbers – ten meeelion IOPS!!
  • 00:02:00 – But certified benchmarks are expensive to perform.
  • 00:04:00 – Storage arrays aren’t interesting, do hero numbers help?
  • 00:05:00 – Do users even care about industry standard benchmarks?
  • 00:06:00 – STAC – financial application benchmarks
  • 00:07:48 – Should vendors be more realistic about hero numbers?
  • 00:09:50 – Why has Pure Storage stopped producing FlashArray performance data?
  • 00:11:40 – However Pure do produce numbers for CI like AIRI (see Soundbytes #008).
  • 00:14:30 – Do people do proofs of concept any more?
  • 00:16:00 – Maybe arrays are massively over specified anyway?
  • 00:16:30 – Or actually, it’s about consistent deterministic performance.
  • 00:17:44 – Do we need performance benchmarks for cloud vendors?
  • 00:19:00 – So what should customers do?
  • 00:20:08 – Wrap Up

Related Podcasts & Blogs

  • Pure Storage AIRI – CI for AI
  • Why Deterministic Storage Performance is Important
  • Avoiding the Storage Performance Bathtub Curve
  • Avoiding All-Flash Missing IOPS
  • Are DataCore’s SPC Benchmarks Unfair?
  • NFS Vendor Benchmarks – Caveat Emptor
  • Storage Performance: Why I/O Profiling Matters
Vendors referenced in this post: Violin Systems, Dell EMC (PowerMax), Pure Storage (FlashBlade & FlashArray), NetApp.

 

Copyright (c) 2016-2018 Storage Unpacked.  No reproduction or re-use without permission. Podcast Episode UGE3.

Podcast: Play in new window | Download

(Visited 320 times, 1 visits today)

Filed Under: Garbage Collection Tagged With: All-flash, ChrisE, chrism, Dell EMC, IBM FlashSystem, MartinG, NVMe, Pure Storage, Violin Systems

Reader Interactions

Trackbacks

  1. #95 - Quantifying Storage Vendor Guarantees with Chris Mellor and Eran Brown | Storage Unpacked Podcast says:
    26 April 2019 at 11:18 am

    […] #58 – Storage Vendor Hero Numbers […]

  2. What Next for Violin Systems? - Architecting IT says:
    23 July 2019 at 3:47 pm

    […] #58 – Storage Vendor Hero Numbers […]

  3. #123 - Hitachi Next 2019: VSP 5000 | Storage Unpacked Podcast says:
    18 October 2019 at 11:00 am

    […] #58 – Storage Vendor Hero Numbers […]

  4. Does WekaIO Have the Fastest File System on the Planet? - Architecting IT says:
    29 November 2019 at 11:03 am

    […] will always use hero numbers for marketing purposes.  If you’re not even playing in the benchmark game, then you don’t get […]

Primary Sidebar

Subscribe to Podcast

Apple PodcastsGoogle PodcastsSpotifyAndroidStitcherby EmailRSS

Categories

  • All-Flash
  • AWS
  • AWS Reinvent
  • Cloud
  • CloudFlare
  • Commvault
  • Containers
  • Data Management
  • Data Protection
  • Databases
  • Events
  • Everspin Technologies
  • Fauna
  • File Systems
  • Flash Memory Summit 2022
  • Fungible
  • Garbage Collection
  • General
  • Guest Speakers
  • Hitachi NEXT 2019
  • Hitachi Vantara
  • HPE
  • HYCU Inc
  • IBM
  • In-Memory Computing
  • Infinidat
  • Intel
  • Liqid
  • MemVerge
  • Midrange
  • NVIDIA
  • NVMe
  • Object Storage
  • Persistent Memory
  • Pliops
  • Pure Accelerate techfest22
  • Pure Storage
  • Pure//Launch
  • Scality
  • Serverless
  • SmartNICs
  • Softiron
  • Software-Defined Storage
  • Soundbytes
  • Sponsored
  • Standards
  • Storage Hardware
  • Storage Management
  • StorONE
  • Tintri
  • VAST Data
  • Vendors
  • VMware
  • Western Digital

Copyright (c) 2021 Brookend Ltd, no reproduction without permission.