A VAST-ly Different Storage Story

Who doesn’t love an inspirational quote?!  The idea of tying one of my favorite quotes into a tech post kinda excites me.  So, here goes.

This quote pops into my head on the regular. Mostly, it’s when a project’s to-do list is way too long, and I feel overwhelmed. I think of this quote and proceeding thoughts are usually slow down, focus on tasks to accomplish and forgot about the rest (for now). Complexity can send you in a million different directions and distracts you from the execution of the tasks that need to be done. Complexity is the antithesis of simplicity.

Simplicity was a theme from VAST Data’s messaging at Storage Field Day 18.  While VAST joked that they were trying to take over the world, the fact is that they aren’t.  They are focusing on key tasks that need to happen one step at a time.

Now, that I’ve shared my inspirational quote, let’s explore who is VAST Data, what is their value proposition is, and how removing complexity has aided their ability to execute.

VAST Data’s answers to the “Who, What, and Why Now” questions
After about three years in stealth mode, VAST Data officially debuted in February 2019. Storage Field Day 18 was their first big public appearance. Renen Hallak, VAST CEO and Founder, was one of three presenters. Renan, hold 17+ patents and was a developer and architect of XtremIO.VAST claims that their target customers need hyperscaler agility and scalability. However, for reasons like cost or data gravity, they aren’t able to fully get this in a public cloud. Their customers’ application-centric workloads require large global datasets and the fast performance for machine learning and artificial intelligence.

VAST was conceived in an era where significant innovations like NVME-oF and Storage Class Memory (SCM) were just on the horizon. If you pair these disruptive advancements with breakneck network speeds like 400 Gigabit Ethernet, it’s a recipe “disaggregated share everything architecture.” This architecture removes a lot of complexity for which HCI and traditional direct attached share nothing storage have had to contend for resiliency and scalability purposes. VAST’s “disaggregated share everything storage architecture” allows compute and storage to be scaled independently. It also addresses the scalability and resiliency complexity for which traditional scale-out storage vendors have had to contend.

Currently, a typical VAST deployment is an appliance with a minimum of 1 petabyte from an NVMe JBOF (Just a Bunch of Flash) arrays and four containers in which the storage software lives. Supported protocols are NFS and S3. More protocols and deployment models are expected further down the road.  This approach is another case where VAST is focusing on the protocols that their customers have requested.

Where VAST scrapped some of the complexity:

1. Getting to the “why” first.Getting to the “why” first. VAST gathered customer pain points for 6-months before they started created requirements and designing their solution. This approach allowed them to develop storage to meet these needs rather than cobbling together solutions and looking for markets. Anyone who has ever done software development will tell you that a thorough requirement gathering process at the beginning saves you time, money and scope in the end.
2. Only NVMe will do. Explicitly designed for non-volatile memory and flash, the NVMe commandset only requires 13 commands. By forgetting about HDDs, VAST can skip the SCSI, and the complexity that comes with using commands meant a slew of devices and not only storage. NVMe-oF also lets disks connect to the fabric without storage controllers. Also, there is no need to tier between performant and capacity disks. It’s all fast storage.
3. Caching of Metadata is unnecessary. Metadata structures are stored in storage class memory. Self-describing data structures remove the need for caching and any subsequent cache consistency mechanisms. These self-describing file attributes allow VAST to handle file locking at the file attribute level, not in a traditional metadata database that would have to replicate between multiple nodes. Note: Any atomic operations happen in RAM.
4. Keepin’ it stateless. Software logic lives in stateless containers and state lives in JBOFs. Losing all containers doesn’t mean you lose data. Also, there is no need to manage availability failover/giveback/quorum that traditional scale-out storage clusters handle. This design supports disaggregation and moves the bottleneck to CPU allowing you to scale compute and storage independently. Also, it enables global algorithms for all devices in the system.

No matter where you’ve worked in IT, you probably have a story about how avoiding complexity has helped you execute. I enjoyed watching VAST weave this theme throughout their SFD presentation.

By avoiding complexity where it’s not required, VAST can execute on their plan to create scalable and resilient storage solution to rival hyperscalers, but for the masses (or those without the means to roll their own storage architecture).

Btw, if you haven’t watched VAST’s Storage Field Day 18 presentations, it’s not too late. Even if storage isn’t your jam, VAST’s videos are still worth watching to see how an excellent technical presentation should look. Simple slides without too much content, including relevant historical context, and answering all of the questions that one would need to understand who VAST data is and what their key differentiators are makes a great story.

 

Disclaimer: As Storage Field Day 18 delegate, GestaltIT paid for my hotel, airfare, and meals. With this invitation came no obligation to blog about or endorse any the presenting vendors or GestaltIT. Any content written or opinions expressed are solely mine.