#75 – It's ILM All Over Again with Chris Mellor

Data volumes have always increased over time as we store more information with the hope that one day some of it will be useful.  Even 30 years ago on the mainframe, Information Lifecycle Management or ILM was a thing with tools like DFHSM used to move content around between disk and tape.  This week Martin, Chris Evans and Chris Mellor talk about the new range of data management or ILM products that are looking to resolve the current issues of data sprawl.

How do these products work?  It is all about data ingest, or just indexing?  What happens when data moves between platforms and how do files continue to be accessed?  Companies like Komprise claim to be able to move data without impacting the application or needing to use techniques like stubs.  Others like IBM Spectrum Discover, simply appear to do content indexing.  Ultimately, perhaps we need to move everything to object stores and dispense with file services altogether.  Can we ignore public cloud and can we get the S3 API moved to an open standard?  All questions the team attempt to answer on this week’s podcast.

Elapsed Time: 00:37:26

Timeline

• 00:00:00 – Intros
• 00:01:00 – Why do we need data management or ILM?
• 00:03:00 – How do we manage so many different data silos?
• 00:04:40 – Data management gets conflated with other features
• 00:05:00 – What vendors are there in this space?
• 00:07:00 – ILM on the mainframe!
• 00:09:00 – Should ILM functions be built into the file system?
• 00:11:00 – How is public cloud influencing the ILM process?
• 00:12:30 – Data gravity (or inertia) causes problems moving into the cloud
• 00:16:00 – How can we solve the distributed data problem?
• 00:18:00 – Hammerspace – separating data and metadata
• 00:21:00 – Everyone needs ML & AI in their storage platforms
• 00:22:00 – Have we simply not defined the data management problem?
• 00:25:00 – Remember File Area Networks?
• 00:27:00 – How does IBM Spectrum Discover work?
• 00:31:00 – Qumulo QF2 – focused on fast metadata searches
• 00:32:30 – We still have a lot of sticking plaster and bandaids in place
• 00:33:40 – Should we just move everything to object stores?
• 00:35:30 – Please Jeff, can you donate the S3 API to the community?
• 00:36:30 – Wrap Up

Related Podcasts & Blogs

• #68 – Intelligent Object Storage with Scott Baker
• #65 – Challenges in Managing Unstructured Data with Shirish Phatak
• #60 – New Data Economy with Derek Dicker
• Data Gravity Pointed the Way to Data Rather than Storage Management

Podcast: Play in new window | Download