Inside OpenAI Habitat: scaling online storage past 70M requests per second and migrating from Python to Rust
OpenAI centralized its storage control plane before shifting production traffic to a Rust rewrite, with measured efficiency and latency gains.
Key takeaways
OpenAI published an engineering account of Habitat on September 11. The company says its online storage platform now handles more than 70 million requests per second, supports products used by over one billion people each week across almost 40 regions, and serves more than 500 petabytes of data. Habitat began as a Python client library for GPTs in 2023, then became a standalone service so deployments, observability, access controls, audit logging, and platform improvements could be managed centrally. In the second quarter of 2026, two engineers used Codex and GPT-5.5 to rewrite the service in Rust. OpenAI reports that the Rust service now handles 95% of production requests and is six times more CPU-efficient and fifteen times more memory-efficient than the Python version. These are internal production measurements, so other teams should adopt the migration discipline rather than assume identical gains.
Direct answer
OpenAI published an engineering account of Habitat on September 11. The company says its online storage platform now handles more than 70 million requests per second, supports products used by over one billion people each week across almost 40 regions, and serves more than 500 petabytes of data. Habitat began as a Python client library for GPTs in 2023, then became a standalone service so deployments, observability, access controls, audit logging, and platform improvements could be managed centrally. In the second quarter of 2026, two engineers used Codex and GPT-5.5 to rewrite the service in Rust. OpenAI reports that the Rust service now handles 95% of production requests and is six times more CPU-efficient and fifteen times more memory-efficient than the Python version. These are internal production measurements, so other teams should adopt the migration discipline rather than assume identical gains.
Verified facts
OpenAI says Habitat processes more than 70 million requests per second, supports products used by over one billion people each week across almost 40 regions, and serves more than 500 petabytes of data.
Habitat launched for GPTs at DevDay 2023 as a Python client library over Azure Cosmos DB, then became a standalone service to centralize deployment, observability, security controls, and platform improvements.
OpenAI reports that two engineers used Codex and GPT-5.5 to complete a Rust rewrite in Q2 2026; it now serves 95% of production requests with measured gains of 6x CPU efficiency and 15x memory efficiency.
What changed
- Storage logic moved from distributed clients into a centralized service control plane
- Tail latency, connection reuse, and downstream protection became explicit operating concerns
- A constrained NoSQL API preserves predictable work as a scaling boundary
- The Rust rewrite was validated by progressively serving production traffic
Impact for AI users
The account shows why user-perceived speed follows the slowest parts of a request chain rather than average database latency alone. For teams building high-volume AI products, the transferable pattern is to stabilize the protocol, permissions, audit trail, and observability before replacing the runtime. Shadow traffic, progressive percentages, and an explicit rollback point can then test the new implementation. OpenAI's scale, internal tooling, and reported efficiency multiples are specific to Habitat and should not be treated as a forecast for another system.
Operating checklist
- Capture a pre-migration baseline for throughput, p95 and p99 latency, event-loop delay, connection counts, and downstream errors.
- Stabilize routing, authorization, audit logging, and rate limits behind one protocol, then test compatibility with replayed traffic.
- Compare Python and Rust with shadow requests and a small production percentage, expanding only while automatic stop thresholds remain clear.
- Preserve the version, metrics, failure samples, and rollback command for each stage, and verify the old path before proceeding.
AI frontier news and analysis, AI software and model tools, AI skill tutorials and validation methods, and AI account and permission guidance
FAQ
Is Habitat a database product offered by OpenAI?
No. The article describes Habitat as an internal online storage platform for OpenAI products; this release is an engineering account.
Has OpenAI completely retired the Python service?
Not yet. OpenAI says Rust handles 95% of production requests and that Python will be deprecated over the following weeks.
Can another system assume the same 6x and 15x efficiency gains?
No. Those are internal Habitat measurements, and hardware, workload, protocol, and implementation differences can materially change the result.
Summary
Habitat's central lesson is not merely the choice of Rust. OpenAI first established a common control plane, then let the replacement prove itself progressively with production metrics. That migration order is more reusable than any single efficiency number.
This AI-assisted article is checked by ENHE AI automation for official sources, bilingual fields, media rights, page safety, and historical duplication before publication.
What this means for everyday users
The account shows why user-perceived speed follows the slowest parts of a request chain rather than average database latency alone. For teams building high-volume AI products, the transferable pattern is to stabilize the protocol, permissions, audit trail, and observability before replacing the runtime. Shadow traffic, progressive percentages, and an explicit rollback point can then test the new implementation. OpenAI's scale, internal tooling, and reported efficiency multiples are specific to Habitat and should not be treated as a forecast for another system.
Tools you may use
Related tutorials
[[zh]][[/zh]][[en]]Review a public website[[/en]]
SearchScope SEO/GEO
[[zh]][[/zh]][[en]]Access and usage guide[[/en]]
AI Prompt Management System | 418 Bilingual Prompts for Writing, SEO and AI Creation
High-Frequency AI Prompts for Work, Learning, and Teaching guide
High-Frequency AI Prompts for Work, Learning, and Teaching
Related Tools And Tutorials
Use the following ENHE AI sections to continue from the news signal into tool selection, account-service guidance, or practical learning.
Related reading
A durable Dense versus MoE model-selection checklist beyond total parameters
NVIDIA's September 15 technical article explains the central Dense versus Mixture-of-Experts tradeoff. Dense models activate all parameters for every token; MoE models keep all experts in memory but route each token through a subset, separating memory cost from compute cost. Nemotron 3.5 Lightning is cited as a 30B-total model with 3B active parameters per token. MoE can raise throughput at similar total size, while latency advantages narrow at high concurrency, fine-tuning can create router imbalance, and quantization affects routing and projection layers differently. This checklist turns those factors into a repeatable selection task using a fixed request set, target hardware, quality thresholds, concurrency steps, and cost per accepted result.
AWS HyperPod Inference Gateway routes requests with live GPU signals
AWS announced SageMaker HyperPod Inference Gateway Tier 1 on September 18. The EKS managed add-on uses a Body-Based Router to read the model field from OpenAI-compatible requests and an Endpoint Picker that scores KV-cache use, queue depth, LoRA residency, prefix-cache hits, and running requests. AWS reports tail-latency gains across several matched-replica scenarios and summarizes up to an 82% first-token reduction, while a uniform steady workload remained comparable to round robin. Teams should repeat the comparison on their own hardware, models, and traffic. A production trial should hold replicas and requests constant, include failure and 429 behavior, and preserve the prior routing configuration for immediate rollback.
NVIDIA Expands DGX Spark With 64GB: Evaluate a Local AI Workstation
On October 2, NVIDIA announced a 64GB unified-memory DGX Spark configuration and said partner availability would begin October 23 at a starting price of $4,999. NVIDIA says one unit can run models up to 100 billion parameters and that two systems can pool 128GB; it reports up to 1.7x performance in its own Qwen 3.8 27B test. Availability is forward-looking, and performance figures are vendor results. Model choice, quantization, context length, and concurrency change outcomes. This article offers a practical evaluation checklist for teams comparing local inference with cloud services. Teams should check applicability against their own environment, record assumptions, and validate decisions with representative operational evidence.
Microsoft’s 2026 Defense Report: AI Is Changing the Pace of Cybersecurity
In its October 1 commentary on the 2026 Microsoft Digital Defense Report, Microsoft says threat actors are using AI in parts of existing workflows, including reconnaissance and social engineering. The post does not claim that attacks have become fully autonomous; it highlights changes in speed and scale. Identity controls, least privilege, data protection, monitoring, testing, and secure development remain central. This article separates Microsoft’s observations from independent conclusions and turns the post into a practical checklist. Teams should compare the claims with their own telemetry and incident history before changing priorities. Teams should check applicability against their own environment, record assumptions, and validate decisions with representative operational evidence.
A One-Week AI Agent Security Review Using the OWASP Risk Framework
The OWASP Gen AI Security Project published its 2026 Top 10 for Agentic Applications in December 2025 as a risk framework for systems that plan, call tools, or act across services. It is a starting point for security discussion, not a certification, compliance attestation, or guarantee of safety. This article turns it into a one-week baseline review: inventory agent identities and tools, narrow permissions, treat external content as untrusted input, record consequential actions, test shutdown and revocation, and define approval and recovery for high-impact operations. Teams should extend the checklist to match their architecture and threat model. Teams should check applicability against their own environment, record assumptions, and validate decisions with representative operational evidence.
Microsoft Announces Streaming Transcription: Voice Agents Can Respond Earlier
On October 1, Microsoft AI announced MAI-Transcribe-2-Streaming, MAI-Voice-2.1, and a Flash variant. Microsoft says the streaming transcription model supports 60 languages with continuous language detection, and that initial partials can appear just over 100 milliseconds after audio arrives. Its speech-generation models are described as supporting 23 languages and 26 locales. Rankings, accuracy, speed, and price claims come from Microsoft’s announcement and do not predict results for every workload. This article explains the interaction changes and suggests tests for latency, revisions, accents, privacy, and total cost. Teams should check applicability against their own environment, record assumptions, and validate decisions with representative operational evidence.