Your CEO read about Llama being "free" and wants to know why you're paying for a closed frontier model. Your security team wants everything on-premise. Your developers want to use Claude. Here's what this decision actually means for your organization.

Understanding Open vs Closed Source AI

Open Source AI means the model's weights, architecture, and often training code are freely available. You can download models like Meta's Llama, Mistral, or DeepSeek, modify them, run them on your own servers, and even fine-tune them on your proprietary data. You have complete control, but also complete responsibility for infrastructure, maintenance, and performance.

Closed Source AI refers to proprietary models like OpenAI's GPT, Anthropic's Claude, or Google's Gemini that are accessed through APIs or web interfaces. The companies control the models, handle all infrastructure, and provide regular updates. You pay for usage but get enterprise support, guaranteed uptime, and no operational overhead.

Think of it like running your own email server versus using Gmail: one gives you total control, the other gives you convenience and reliability.

The "Free" Open Source Myth

Llama, DeepSeek, and other open-weight models are free like a puppy is free. The model weights cost nothing. Running them costs everything.

To run a 70B-parameter open model effectively, you need about 140GB of VRAM at full precision or 70GB with INT8 quantization. That means two NVIDIA A100 80GB GPUs at thousands of dollars per month in cloud costs, or a single H100 at even higher rates, before you count the engineers who keep it running.

Meanwhile, Claude and ChatGPT business plans charge a fixed fee per user per month, and Azure OpenAI runs on consumption-based pricing. No infrastructure, no maintenance, just API calls.

The math usually only works for open source at high, steady token volumes or with strict compliance requirements. Below that level, you're paying more for control you might not need.

What You Actually Control (And Don't)

Open Source Control: You can modify the model, fine-tune it on your data, and run it anywhere. Fine-tuning on domain terminology can improve accuracy for specific use cases. You own the deployment, the data never leaves your servers, and you're not dependent on any vendor.

But you also own every problem. When the model hallucinates, that's your problem. When it needs updating, that's your engineering time. When it goes down at 3 AM, that's your incident response. And with the new mixture-of-experts architectures, debugging has become significantly more complex.

Closed Source Reality: You get what OpenAI, Anthropic, or Google delivers. You can't modify the base model, though providers offer fine-tuning for selected models, and long context windows allow extensive customization through prompting. Your data goes through their servers (though enterprise agreements include specific data protection clauses).

But when a provider ships a better model, enterprise customers get the improvements without redeploying anything. When infrastructure fails, the provider's SLA covers it. When new capabilities launch, you get them immediately.

The Privacy Question Everyone Asks Wrong

"We need open source for data privacy" is what every CISO says initially. But privacy isn't about where the model runs; it's about data governance and contractual guarantees.

OpenAI's enterprise agreement includes zero data retention for training and SOC2 Type II compliance. Your prompts don't train their models. Azure OpenAI goes further with Data Zones: letting you control exactly which geographic regions process your data while still accessing the latest models. Microsoft explicitly states data is encrypted at rest (AES-256) and in transit (TLS 1.2+), with 30-day retention only for abuse monitoring (which you can opt out of).

Meanwhile, that open source model running in your data center might be logging everything to debug errors. Your contractors might be copying outputs to their laptops. Privacy comes from processes and contracts, not just premises.

A common pattern: a team spends months deploying an open model for privacy, then finds the bigger risk is employees pasting company data into personal AI accounts.

Performance Reality Check

The benchmarks tell an interesting story. OpenAI's open-weight gpt-oss-120b came close to its smaller proprietary reasoning models on reasoning benchmarks. Open 70B-class models match earlier frontier models on many benchmarks while using fewer resources. For many business use cases (document analysis, code generation, customer support) the gap is small enough that cost and control decide.

The real differences appear in specialized capabilities. Current frontier models offer context windows large enough to hold entire codebases. Some vendors put heavy emphasis on safety training, which suits regulated industries. Efficient open models fit high-volume batch processing.

When teams test models on their own tasks, the differences are often small. Total cost of ownership, not benchmark scores, should drive the choice.

The Real Decision Framework

Choose open source when:

Choose closed source when:

  • You want to start using AI tomorrow
  • Usage is low or hard to predict
  • You need consistent updates and new capabilities
  • Your team should focus on business logic, not infrastructure
  • You need enterprise SLAs and support

The hybrid approach that actually works: Use closed source APIs for prototyping and variable workloads. Deploy open source for high-volume, specialized operations. For example, a team can use a closed API for customer service and a fine-tuned open model for high-volume product descriptions.

The Hidden Costs Nobody Mentions

Open Source Hidden Costs:

  • GPU availability and procurement lead times
  • Quantization reduces costs but can degrade quality, especially on coding tasks
  • Engineering time: expect 2-3 FTEs for production deployment
  • Monitoring and observability tools add to infrastructure costs
  • Model versioning and rollback systems

Closed Source Hidden Costs:

Recent Shifts in the Market

The landscape shifted in 2025. OpenAI released gpt-oss models under the Apache 2.0 license, its first open language models since GPT-2. The smaller gpt-oss-20b runs on hardware with 16GB of memory, with performance close to OpenAI's smaller proprietary models.

Anthropic introduced tiered rate limits after a small group of heavy users overloaded Claude Code, a reminder that closed services can change terms on short notice.

Microsoft's Azure OpenAI Data Zones now let you specify exact geographic boundaries for data processing while maintaining access to all models: addressing the main enterprise concern about cloud AI.

Smaller open models now deliver performance that once required far larger ones, which changes the economics of self-hosting.

The Decision Most Make

After evaluation, here's what enterprises actually do:

Most choose closed source first, using business plans such as ChatGPT or Claude for immediate productivity gains with predictable per-user budgeting.

Many adopt hybrid approaches within a year, keeping closed source for general use while deploying open source for specific high-volume tasks.

A minority go fully open source, typically those with existing ML infrastructure and regulatory requirements.

Start with closed source APIs to understand your actual usage patterns. Track token consumption, peak loads, and use cases. If you consistently process high token volumes with predictable workloads, evaluate open source. Most discover they never reach that volume.

The question isn't "open or closed?" It's "build infrastructure or buy capability?" For most enterprises, buying the capability lets you focus on what actually differentiates your business, which isn't running AI infrastructure.