Why I Ditched the Cloud for On-Prem Infrastructure
Key Factors Before Migration
- Pricing on Cloud: ₹40 Lakhs per month
- Network Cost on Cloud: Very high (egress-heavy traffic)
Key Factors After Migration
| Factor | On Cloud | On-Prem |
|---|---|---|
| Pricing | ₹4 Lakhs per month (for object storage & backup) | ₹3 Lakhs per month |
| Network Cost | Very high (egress-heavy) | Minimal (localized traffic) |
The Turning Point
In 2024, my previous organisation underwent significant management changes, and I was tasked with drastically cutting cloud costs. We already had substantial hardware infrastructure across various edge locations. Initially, I focused on cloud cost-optimisation methods, but the results were minimal.
So, I had a crazy but promising idea—what if we ditched the cloud completely?
Before proposing this, I researched extensively and found many examples, including Subhash Choudhary, CTO of Dukaan, who explained how Dukaan moved out of the Cloud and onto Bare Metal, whose story reignited my forgotten passion.
Without a fancy presentation, I used a whiteboard and marker to pitch this idea to management—and our on-prem journey began.
Background and Context
I've always loved servers and infrastructure. In college, I worked with IBM and Acer servers, plus firewalls. Since 2014, when NTT opened its Mumbai data centre, I’ve been fascinated by how large-scale infrastructure operates.
A favourite learning resource was the YouTube channel My PlayHouse by Morten Hjorth, which helped me understand the physical infrastructure behind the “cloud.”
I had limited knowledge of global networking until discovering BGP (Border Gateway Protocol) and ASN (Autonomous System Numbers), which was mind-blowing.
How It All Started: The Network Challenge
Managing public IPs and connecting private infrastructure to the internet was the major challenge.
I obtained a /23 public IP block from NIXI (National Internet Exchange of India).
For networking, I selected the Arista 7050SX as the main BGP gateway switch and Sophos Firewall for security.
Internet connectivity was set up via dual Internet Leased Lines (ILL) from Jio and Airtel, ensuring redundancy.
The planning and deployment took about 1.5 months, after which the cloud bill dropped from ₹40 Lakhs to ₹3 Lakhs per month.
The Hardware and Network Setup
- 6.4 Tbps local network backbone
- 2 Gbps public internet uplink
- 3,000 CPU cores
- 8 TB RAM
- 30 TB NVMe production storage
- 40 TB archival storage
- GPUs like RTX 4060, RTX 3090, NVIDIA A6000 and L40S
Software and Tools
- FreeSWITCH – voice & telephony
- NFS and MinIO – distributed storage
- Zabbix – infrastructure monitoring
- ELK Stack – centralised logging
- Prometheus + Grafana – metrics & visualization
- Ubuntu Server – operating system
- Databases: PostgreSQL, MySQL, MongoDB
- Next.js – frontend framework
- Redis, Kafka – caching and message streaming
- Backend: Go, Node.js, Python, PHP
- AI/ML: TensorFlow, RASA, NLU
- Custom orchestration and automation tools built in-house
Additional Context & Considerations
It is important to note that this approach works best for stable, predictable workloads with high data egress, like AI processing pipelines. Cloud remains preferable for rapid scaling, short projects, or workload variability.
On-prem migration requires significant upfront investment, greater management effort, and technical expertise, especially around networking and infrastructure automation.
Results will vary based on organisational scale, workload patterns, and resource utilisation efficiency.
Final Thoughts
Our AI and data processing workloads made cloud infrastructure too costly and complex. Moving on-prem drastically cut expenses and gave us full control and visibility.
While cloud offers easy scalability and convenience, for heavy processing workloads, on-prem infrastructure remains superior.
Most importantly, this journey rekindled my passion for infrastructure engineering — something no managed cloud service can provide.
"Sometimes, the best way to understand the cloud is to step outside it."
