Speed up container startup with SOCI
SOCI (Seekable Open Container Initiative) indexing enables lazy loading of custom container images in Amazon SageMaker Studio or Amazon SageMaker Unified Studio. SOCI significantly reduces startup times by roughly 30-70% for your custom Bring your own image (BYOI) containers. Latency improvement varies depending on the size of the image, hosting instance availability, and other application dependencies. SOCI creates an index that allows containers to launch with only necessary components, fetching additional files on-demand as needed.
SOCI addresses slow container startup times, that interrupt iterative machine learning (ML) development workflows, for custom images. As ML workloads become more complex, container images have grown larger, creating startup delays that hamper development cycles.
Topics
Key benefits
-
Faster iteration cycles: Reduce container startup, depending on image and instance types
-
Universal optimization: Extend performance benefits to all custom BYOI containers in Studio
How SOCI indexing works
SOCI creates a specialized metadata index that maps your container image's internal file structure. This index enables access to individual files without downloading the entire image. The SOCI index is stored as an OCI (Open Container Initiative) compliant artifact in Amazon ECR and linked to your original container image, preserving image digests and signature validity.
When you launch a container in Studio, the system uses the SOCI index to identify and download only essential files needed for startup. Additional components are fetched in parallel as your application requires them.
Architecture components
-
Original container image: Your base container stored in Amazon ECR
-
SOCI index artifact: Metadata mapping your image's file structure
-
OCI Image Index manifest: Links your original image and SOCI index
-
Finch container runtime: Enables lazy loading integration with Studio
Supported tools
| Tool | Integration |
|---|---|
| nerdctl | Requires containerd setup |
| Finch CLI | Native SOCI support |
| Docker + SOCI CLI | Additional tooling required |