This article provides a detailed technical overview of the newly architected Amazon OpenSearch Serverless.
Amazon has announced a significant re-architecture of its OpenSearch Serverless, designed to enhance performance and cost-efficiency. The new system promises up to 20 times faster autoscaling, the ability to scale to zero, and cost reductions of up to 60% compared to traditional cluster provisioning for peak loads.
Modern applications often experience unpredictable workloads, such as e-commerce spikes during sales or AI-driven tasks that fluctuate in demand. The updated OpenSearch Serverless architecture addresses these needs by decoupling compute from storage, allowing for rapid provisioning of infrastructure and scaling down to zero when idle.
Key Improvements
The new architecture focuses on three main areas:
- Performance: OpenSearch Compute Units (OCUs) are provisioned in seconds, enabling quick resource allocation during traffic surges.
- Cost: Independent metering for indexing, search, and storage allows for optimized billing, potentially reducing costs significantly for idle workloads.
- User Experience: Simplified setup processes eliminate the need for extensive capacity planning, making it easier to deploy collections.
Decoupled Compute and Storage
With the new design, OCUs can scale independently of the data stored, thanks to a new shared storage layer. This means users can maintain multiple indices without incurring compute costs when not actively used, leading to substantial savings.
Scale to Zero Feature
When idle, the service can reduce compute resources to zero after a set timeout, quickly resuming capacity within approximately 10 seconds when traffic returns. This feature ensures that resources are not wasted during periods of inactivity.
Simplified Setup and Management
The launch of Express Create allows users to set up collections in seconds without the need for initial configuration of network or access policies. This feature is ideal for agentic workloads that require immediate response times.
Enhanced Endpoint Management
Amazon has introduced two endpoint formats for improved management. The per-collection endpoint continues to function as before, while the new per-account regional endpoint simplifies access for multiple collections.
Collection Groups
Collection groups enable sharing of compute resources across multiple collections, optimizing costs for smaller collections with similar traffic patterns. This feature also allows for distinct encryption keys for enhanced security.
Getting Started with OpenSearch Serverless
To create a collection, users can utilize the Express Create feature, enabling rapid deployment for various search types, including vector searches. The architecture supports independent scaling for search and indexing operations.
Conclusion
The new Amazon OpenSearch Serverless architecture is now available, offering enhanced performance and cost-saving features tailored for dynamic workloads. Users can create collections quickly and efficiently, ensuring that compute costs drop to zero during idle periods.
