Sage Weil

Sage Weil

Ceph Principal Architect, Red Hat

From DreamHost to Red Hat, Sage Weil has spent almost 20 years forging a path for open source data storage

Sage Weil started seeing the need for an open source storage alternative to proprietary data storage solutions while at college. “I co-founded a web hosting company with a bunch of other students called DreamHost. We were storing and hosting a bajillion websites, we had lots of data, and at some point we switched to using centralised storage: we started buying netapps and it was the greatest thing ever to have all our storage in one place. A couple of years down the line, however, and we'd grown to the point where we had about a hundred netapps. We were writing software to manage moving data around. It had all become nightmarishly complicated, not to mention expensive,” he laughs. “The problem was that, at that point in time, there were no open source storage systems that were scalable. You had to either cobble something together with a bunch of machines or go buy some expensive enterprise solution.” 

Around this time, Weil started Grad School at the University of California: Santa Cruz, where he ended up in the storage research group - a group which also collaborated with some of the US’ most prestigious national labs, like Los Alamos, on scalable, petabyte-scale file systems. “The Ceph architecture came out of that research group,” he explains. “There were all sorts of interesting ideas we had back then about data distribution, consistency, and so on that we assembled into a working prototype in 2007.” 

By 2011, Weil and a few colleagues started Inktank, the lead development contributor and financial backer for Ceph. He reflects that “I was pretty naive back then. I thought you just built something, open sourced it, and people would just start appearing to develop it, fix bugs, etc. and that's not how it works.” As Ceph reached a level of maturity where Sage and the Inktank team felt that “people would actually be willing to use it,” he explains that “around that time we started bringing on community managers and people who actually knew about cultivating an ecosystem, and we started talking about creating a foundation for Ceph. But then we got bought by Red Hat and that can was kicked down the road a bit.” 

Weil - along with Red Hat, and some of the biggest names in the open source tech space and beyond - officially launched the Ceph Foundation in 2018, under the Linux Foundation’s umbrella. In those intervening years (and since) Weil reflects that the sheer size, depth, and diversity that’s developed in the Ceph community is staggering. 

“We never really know how big our community is or how many people are using Ceph. We have to go on data like how many bug reports we get, how many patches we get, how active our user list is, etc. and those metrics definitely give us a glimpse into just how big and varied the Ceph user base is and how deep its penetration is into certain spaces. 

“We talk to a lot of users that love Ceph because it allows them to do things that they simply couldn't do if they were forced into an expensive proprietary solution. 

“One conversation that stands out to me was with a researcher at a University in Canada that does research for a cancer institute. They told me that Ceph is what allows them to do the research that they do, because they wouldn't have the funding to support the massive amounts of data storage that their research requires if they didn't have an open source, low-cost solution. 

“In fact, Ceph is widely used throughout a lot of the scientific community. We have a very robust relationship with the group at CERN, and we know they collaborate with scientific communities around the world. A lot of them are deploying Ceph to do all sorts of different things, like these scientists in South Africa and Australia that are building crazy big radio telescopes that use Ceph to store all their data. 

“There's a lot of Ceph adoption and a pretty robust community of companies in Europe. I think there's an absolute tonne of Ceph deployed in China. We hear stories about people building these ridiculously big clusters using Ceph, but there's a language barrier and so the communities don't talk as much. But there's no doubt that there are many, many exabytes of storage in China that use Ceph. 

“We hear all sorts of stories from all around the world about people who are running Ceph. A couple of years ago I ended up helping out an ISP in Baghdad that was running into trouble with their Ceph cluster. In a lot of parts of the world where cost is a major concern to IT service companies, Ceph - not to mention other open source solutions - are what keeps the lights on.” 

 

Read the full story HERE.

Share

Featured Interviews

Featured

Jonathan Eaves

CEO at Edge Centres

Jonathan Eaves, CEO of Australian data centre firm Edge Centres, discusses his background, experience, and even learning to fly to build his business

Read More

Pip Squire

Head of Energy and Sustainability at Ark DataCentres

Ark Data Centres’ Head of Energy & Sustainability, Pip Squire shares his industry experience in infrastructure construction over the past 40 years

Read More
We can't be very much more CO2 efficient because we have pushed the laws of physics to the point where you can't change physics.
Pip Squire
Head of Energy and Sustainability at Ark DataCentres

Friso Haringsma

Managing Director of DataCentre United

Friso Haringsma is Managing Director of DataCenter United and talks about DC expansion, work culture and why local problems require local solutions

Read More

Todd Coleman

Founder, President and CEO, eStruxture

Todd Coleman, Founder, President and CEO of eStruxture, talks strategy, speed, risk, and becoming the frontrunning data centre operator in Canada

Read More

Himmath Mohammed

Head OF Sales & Strategies, Gulf Data Hub

Gulf Data Hub’s Himmath Mohammed discusses his role at the company, what drives him to do well and the challenges he’s faced in his career

Read More

Jonathan Eaves

CEO at Edge Centres

Jonathan Eaves, CEO of Australian data centre firm Edge Centres, discusses his background, experience, and even learning to fly to build his business

Read More