Rethinking the Cloud: Public, Private, and the Trillion-Dollar Question

Unveiling the Cloud Conundrum: A Decade of Lessons and a Private Future

Hey there, I’ve been diving deep into the cloud landscape lately, and honestly—it’s been a bit of a revelation. What I uncovered felt both eye-opening and unsettling. It’s made me reflect a lot on my own decade-long journey with OpenStack while poring over some incredible industry analyses that challenge what we thought we knew about cloud infrastructure.

I wanted to take a moment to share my thoughts with you—not just as a tech enthusiast, but as someone who's lived through the real-world evolution of infrastructure. I’ve seen the promises, the pitfalls, and now, perhaps, the beginnings of a new chapter. So grab a coffee, and let’s unpack this together—starting with a paradox that’s been hiding in plain sight.

The Trillion-Dollar Twist: Cloud’s Hidden Cost

I started with a hard-hitting article from a16z titled The Cost of Cloud, a Trillion Dollar Paradox. It hit me like a ton of bricks: “Public Cloud is great early on, but as you scale, it can quietly erode your margins.”

At first, cloud computing feels magical: instant scalability, flexibility, and none of the hardware headaches. For startups, it’s an absolute gift. But fast forward a few years, and the story changes. The folks at a16z estimate that among just 50 top public software companies, cloud costs are suppressing market value by around $100 billion. Stretch that to a broader slice of the tech industry, and the figure swells to over $500 billion.

One of the clearest examples? Dropbox. They revealed in their 2017 S-1 filing how shifting workloads off public cloud to custom-built, on-premises infrastructure saved them $75 million over two years. Their gross margins soared from 33% to 67%. That’s not just a cost win—it’s a competitive advantage. They credited much of this margin growth to infrastructure optimization.

Even more striking was the point about market cap: every dollar saved could potentially boost stock value by 24-25x. That’s a massive ripple effect.

What hit home for me? It’s not about abandoning the public cloud—it’s about understanding its long-term impact. I’ve seen too many teams get hooked on the ease of public cloud, only to realize later they’re paying a premium for that convenience. Are we, as an industry, prioritizing short-term agility over long-term sustainability? It’s a question we all need to start asking.

Pricing Pains: CIOs Weigh In

Digging deeper, I stumbled on another article zeroing in on cloud pricing realities. Tracy Woo from Forrester said it best: “In the rush to public cloud, a lot of people didn’t think about pricing.”

It’s true. Cloud spend is ballooning, even as IT budgets tighten. CIOs like Ron Hollowell at Reinsurance Group of America are responding by “right-sizing” their footprint—moving steady, predictable workloads back to private clouds where costs are easier to control.

And the shift isn’t small. Radu Vunvulea from Endava shared that 60% of enterprises he works with are repatriating at least one major system from public cloud back to private infrastructure. Especially with data-heavy AI projects, where 45% of cloud costs can come from just moving data around—egress fees, inter-zone transfers—it’s like paying a premium just to shuffle your bits across town.

What we’re seeing here is a tectonic mindset shift. The glow around public cloud is dimming once you unpack the hidden fees. Control and cost visibility are taking center stage again.

Private Clouds On Demand: A Game-Changer Emerges

And then, the real lightbulb moment: Mastering Cloud Costs: On-Demand Private Clouds and the Future of Cloud Infrastructure.

Here’s the headline: On-demand private clouds are emerging as a powerful new alternative.

Built on open-source technologies like OpenStack and Ceph, these solutions (think OpenMetal and others) deliver private cloud benefits—full control, predictable costs—at speeds we once thought only public cloud could achieve. Deployment times have shrunk from months to minutes.

Back in 2018, Dell’s VMware and HPE Greenlake pioneered the trend, but open-source players have supercharged it. They stripped away the complexity and heavy upfront investments that once made private clouds daunting.

And having personally wrestled with early OpenStack deployments, I can’t help but smile at how far we’ve come. What once took me days of blood, sweat, and error logs can now be done in under an hour.

This could truly flip the infrastructure playbook—especially for companies hitting the point where public cloud bills outweigh their benefits.

My OpenStack Journey: From Kilo to Today

Now, allow me to share a personal chapter.

I’m Erwin, and my OpenStack journey began ten years ago with the Kilo release in 2015. Back then, OpenStack was powerful but notoriously complex. Setting it up felt like assembling a spaceship with instructions written in an alien language. Debugging installs? Let’s just say you had to really love the craft.

But I stuck with it. Through Newton, Pike, Queens, all the way to today. And now, as OpenStack celebrates its 15th birthday, I’m proud to have been part of its evolution.

What keeps me hooked after a decade? It’s simple: freedom. OpenStack’s open-source DNA means no vendor lock-in. Its flexibility allows me to shape infrastructure exactly how I need it—whether that’s traditional VMs, Kubernetes clusters, or AI-driven workloads.

Today, OpenStack isn’t just keeping up—it’s powering modern hybrid clouds and massive AI infrastructures, often working hand-in-hand with Kubernetes. The OpenInfra Foundation’s new alliance with the Linux Foundation only strengthens this synergy.

Looking back, I realize OpenStack didn’t just save costs—it empowered innovation. It’s matured into an enabler of choice, performance, and scale. In 2015, I was trying to optimize a few hundred VMs. In 2025, I’m dreaming up hybrid clouds that blend on-demand private infrastructure with raw bare-metal horsepower.

The public cloud offers convenience. But OpenStack—and private clouds—offer freedom and sustainability. That’s the future I see.

Looking Ahead: Your Invitation to the Private Cloud Frontier

So, where does this leave us?

After all this digging, reflecting, and hands-on experience, I’m more convinced than ever: the cloud story is evolving. Public cloud is still a fantastic launchpad. But at scale, it’s private clouds—especially on-demand, open-source-driven ones—that are stealing the show.

The numbers don’t lie: billions of dollars in market value are up for grabs. Companies like Dropbox, Basecamp, and others are showing us there’s a smarter path forward.

Our big takeaway? Infrastructure deserves to be a first-class citizen again. Track your costs like KPIs. Incentivize optimization. Design for flexibility and control from the start.

And this is just the beginning.

I’m kicking off a new series where we’ll dive even deeper—exploring:

• How private clouds can transform your business

• What’s new and exciting in OpenStack (celebrating 15 years!)

• How to lead data protection in your OpenStack and Kubernetes setups

• Best-in-class observability solutions for your infra and apps

I’ll be sharing real-world lessons, research insights, and a lot of hard-earned experience. I would love for you to join me on this ride.

Ready to rethink the cloud with us? Stay tuned—the private future is closer than you think. 🚀

That’s it! Keep innovating and stay inspired! If you think your colleagues and friends would find this content valuable, we’d love it if you shared our newsletter with them!

PROMO CONTENT

Can email newsletters make money?

With the world becoming increasingly digital, this question will be on the minds of millions of people looking for new income streams in 2025.

The answer is—Absolutely!

That’s it for this episode!

Thank you for taking the time to read today’s email! Your support allows me to send out this newsletter for free every day. 

 What do you think for today’s episode? Please provide your feedback in the poll below.

Share the newsletter with your friends and colleagues if you find it valuable.

Disclaimer: The "Tiny Big Spark" newsletter is for informational and educational purposes only, not a substitute for professional advice, including financial, legal, medical, or technical. We strive for accuracy but make no guarantees about the completeness or reliability of the information provided. Any reliance on this information is at your own risk. The views expressed are those of the authors and do not reflect any organization's official position. This newsletter may link to external sites we don't control; we do not endorse their content. We are not liable for any losses or damages from using this information.