Considerations Configuration options Use GPU time-slicing with the NVIDIA DRA driver Use system-mediated GPU time-slicing with Karpenter Use GPU time-slicing on Bottlerocket nodes with the NVIDIA device plugin Use GPU time-slicing on AL2023 nodes with the NVIDIA device plugin Verify that GPU time-slicing is active

Use time-slicing with NVIDIA GPUs on Amazon EKS

Time-slicing lets multiple Pods share a single physical NVIDIA GPU. The Kubernetes scheduler places multiple Pods on the same GPU, and the GPU’s CUDA scheduler time-multiplexes the work. Time-slicing is the simplest GPU-sharing strategy. It uses software only, requires no special hardware, and works on every NVIDIA GPU instance type on Amazon. Time-slicing does not offer memory or compute isolation between Pods that share a GPU. It best suits workloads with low GPU utilization, such as inference services that idle between requests or development environments where several users share a GPU. For workloads that require hardware-level memory and compute isolation, use Multi-Instance GPU (MIG) instead.

On Amazon EKS, you can manage time-slicing with either the NVIDIA DRA driver or the NVIDIA device plugin.

Time-slicing is supported with Karpenter using either dynamic or static capacity provisioning, as well as with EKS managed node groups and self-managed nodes. Dynamic provisioning requires Karpenter version 1.15 or later with the DRANVIDIAGPU feature gate, and Kubernetes version 1.36 or later for Karpenter to simulate time-sliced GPUs. For more information, see Using DRA with Karpenter.

Time-slicing is not available in EKS Auto Mode. Although EKS Auto Mode includes the NVIDIA DRA driver, it does not enable the alpha driver capabilities that GPU sharing depends on, and it does not expose the configuration of the NVIDIA device plugin it manages. EKS Auto Mode support follows the graduation of these capabilities to beta in the upstream NVIDIA DRA driver. To share a whole GPU across Pods on EKS Auto Mode without time-slicing, have several Pods reference the same ResourceClaim. For more information, see Using DRA with EKS Auto Mode.

Time-slicing is a good fit when:

GPU utilization is consistently low, for example latency-sensitive inference services that idle between requests.
Several developers share a single GPU node for notebook or development work.
Your nodes use a GPU instance type that does not support MIG, such as the g5, g6, or g6e families.
You can accept that a Pod is occasionally slower because another Pod on the same GPU is busy.

Consider a different approach when:

You need memory isolation. Pods on a time-sliced GPU share the same GPU memory, and one Pod can exhaust the memory that other Pods rely on. Use MIG for memory isolation.
You need predictable per-Pod latency or quality of service. The GPU scheduler shares compute across slots on a best-effort basis without guarantees.
You run training workloads. Time-slicing adds context-switching that reduces training efficiency. A checkpointed job that another Pod slows down must retry from its last checkpoint.

Considerations

Review the following considerations before you use time-slicing in production.

General considerations

No memory isolation: Pods that share a time-sliced GPU share its memory. One Pod can allocate memory that other Pods need, which can cause out-of-memory errors. Match the number of Pods that share each GPU to the memory footprint of your workloads. If your Pods regularly load large models, share fewer Pods per GPU or use MIG.
Best-effort compute sharing: The GPU scheduler shares compute across Pods on a best-effort basis and does not guarantee proportional compute to each Pod.
Time-slicing and MPS cannot share the same GPU: Time-slicing sets the GPU compute mode to DEFAULT, while NVIDIA Multi-Process Service (MPS) requires EXCLUSIVE_PROCESS. You can use both strategies in the same cluster, but not on the same physical GPU at the same time. Time-slicing also has no effect on a MIG instance. To share a single MIG instance across containers, use MPS instead.
Per-container metrics: NVIDIA Data Center GPU Manager (DCGM) cannot attribute metrics to individual containers when time-slicing is active. GPU-level metrics remain available, but you cannot identify which Pod consumed a given amount of GPU resources.

NVIDIA DRA driver considerations

Alpha feature: Time-slicing through the DRA driver requires the TimeSlicingSettings feature gate, which is an alpha feature disabled by default. For more information, see Use GPU time-slicing with the NVIDIA DRA driver.
Two sharing models: With user-mediated sharing, Pods share a GPU only by referencing the same ResourceClaim or ResourceClaimTemplate, which is namespace-scoped, so sharing cannot cross namespaces. With system-mediated sharing, the driver divides each GPU into a fixed number of shares and Kubernetes consumable capacity enforces the limit across independent claims and namespaces. Both models are alpha in the NVIDIA DRA driver. For more information, see Use GPU time-slicing with the NVIDIA DRA driver and Use system-mediated GPU time-slicing with Karpenter.
Disable the built-in device plugin on Bottlerocket: The DRA driver cannot run alongside the NVIDIA device plugin on the same node. On Bottlerocket, disable the built-in device plugin, which requires Bottlerocket version 1.63.0 or later. For more information, see Install the NVIDIA DRA driver.
Compute support: Time-slicing through the NVIDIA DRA driver is supported with Karpenter, EKS managed node groups, and self-managed nodes. It is not available in EKS Auto Mode, which does not enable the alpha driver capabilities that GPU sharing requires. Karpenter supports DRA with both dynamic and static capacity provisioning as of Karpenter version 1.15. Dynamic provisioning requires enabling Karpenter feature gates. For more information, see Using DRA with Karpenter.

NVIDIA device plugin considerations

Best-effort compute sharing with slots: The device plugin advertises a fixed number of slots per GPU. If a single Pod requests more than one slot, it does not receive additional compute. Enable the fail-requests-greater-than-one option to reject Pods that request more than one slot.
Provisioning with Karpenter: Karpenter counts each nvidia.com/gpu request as a physical GPU even when time-slicing is enabled on the GPU. For more information, see Karpenter issue #2140 on GitHub. To have Karpenter account for shared GPUs when it chooses an instance type, use system-mediated time-slicing through the DRA driver instead. For more information, see Use system-mediated GPU time-slicing with Karpenter.
No support on EKS Auto Mode: EKS Auto Mode manages the NVIDIA device plugin and does not expose its configuration. Because time-slicing requires device plugin configuration, you cannot apply a time-slicing configuration on EKS Auto Mode nodes.
Time-slicing configuration changes: The NVIDIA device plugin does not monitor the time-slicing ConfigMap for changes. Restart the device plugin after you update the configuration.

Configuration options

You can use time-slicing for NVIDIA GPUs with the EKS-optimized AL2023 and Bottlerocket NVIDIA AMIs. The time-slicing settings vary based on whether you are using the NVIDIA DRA driver or NVIDIA device plugin.

NVIDIA DRA driver

The EKS-optimized accelerated AMIs do not include the NVIDIA DRA driver. If you are using Bottlerocket, you must disable the built-in NVIDIA device plugin before you use the NVIDIA DRA driver by setting settings.kubelet-device-plugins.nvidia.enabled = false in the node user data, which requires Bottlerocket version 1.63.0 or later. For more information, see Install the NVIDIA DRA driver.

With the NVIDIA DRA driver, time-slicing is configured through ResourceClaimTemplates. The interval field controls the CUDA time-slice duration.

Interval	Description
Default	Uses the NVIDIA GPU driver’s built-in default interval
Short	Shorter interval; contexts switch more frequently
Medium	Intermediate interval
Long	each context runs longer per turn before being preempted

NVIDIA device plugin

Bottlerocket – The AMI includes a pre-installed NVIDIA device plugin. You configure time-slicing through Bottlerocket settings, with no separate device plugin installation, Helm chart, or ConfigMap.

AL2023 – You install the NVIDIA device plugin as described in Install the NVIDIA Kubernetes device plugin, and supply the time-slicing configuration through a ConfigMap.

The following options control how the NVIDIA device plugin advertises and manages time-sliced GPUs. The field names differ between the Bottlerocket settings and the AL2023 ConfigMap, as shown in the procedures that follow.

Option	Recommended value	Description
Replicas	2–8	The number of schedulable slots to advertise per physical GPU. Higher values allow more sharing but increase contention between Pods.
Rename by default	`false`	When `false`, Pods request `nvidia.com/gpu`, which keeps compatibility with existing manifests. When `true`, the device plugin advertises the resource as `nvidia.com/gpu.shared`, and Pods must request that name. Set this to `true` when you run both shared and dedicated GPU node pools and want workloads to select the shared resource explicitly.
Fail requests greater than one	`true`	Rejects Pods that request more than one time-sliced slot. A Pod that requests more than one slot does not receive proportional compute. Enable this setting to prevent a common misconfiguration.

For the complete list of GPU sharing options and their default values, see the NVIDIA Kubernetes device plugin documentation on GitHub.

Use GPU time-slicing with the NVIDIA DRA driver

With the NVIDIA DRA driver, Pods share a GPU by referencing a common ResourceClaim that requests the gpu.nvidia.com DeviceClass with a time-slicing GpuConfig.

The NVIDIA DRA driver currently implements user-mediated time-slicing: a GPU is shared only among the Pods and containers that you explicitly point at the same claim. Because a ResourceClaim and a ResourceClaimTemplate are namespace-scoped, the Pods that share a GPU this way must be in the same namespace. To share a GPU across containers in a single Pod, have each container reference the same request name in the claim. Containers that reference different request names each receive a separate GPU. Create a separate ResourceClaimTemplate for each time-slice interval you need, and one in each namespace where you want to share GPUs.

System-mediated time-slicing is when the driver shares a GPU across independent claims, including claims in different namespaces, rather than only among the Pods you point at one claim. The driver divides each GPU into a fixed number of shares, and Kubernetes consumable capacity tracks and enforces the limit. If you use Karpenter with dynamic capacity provisioning, system-mediated time-slicing also lets Karpenter account for shared GPUs when it chooses an instance type, so it launches a new node only when the existing GPUs run out of shares. For more information, see Use system-mediated GPU time-slicing with Karpenter.

Important

Time-slicing through the DRA driver requires the TimeSlicingSettings feature gate, which is an alpha feature disabled by default. If you request the TimeSlicing sharing strategy without enabling this feature gate, the driver fails to prepare the device and the Pod stays in ContainerCreating with a FailedPrepareDynamicResources event that reports error validating GPU config: unknown GPU sharing strategy: TimeSlicing. Turn on the feature gate only if you accept the risks of using an alpha capability.

Prerequisites

An Amazon EKS cluster running Kubernetes version 1.34 or later. The NVIDIA DRA driver is supported with Karpenter, EKS managed node groups, or self-managed nodes. If you use Karpenter dynamic capacity provisioning, you need Karpenter version 1.15 or later and Kubernetes version 1.36 or later.
Nodes with NVIDIA GPU instance types using the EKS-optimized AL2023 NVIDIA AMI.
The NVIDIA DRA driver installed as described in Install the NVIDIA DRA driver, with the TimeSlicingSettings feature gate enabled.

Procedure

Create a ResourceClaim that requests a GPU with the TimeSlicing sharing strategy. Multiple Pods that reference this claim share the same physical GPU.


cat <<EOF | kubectl apply -f -
apiVersion: resource.k8s.io/v1
kind: ResourceClaim
metadata:
  name: shared-timeslice-gpu
spec:
  devices:
    requests:
    - name: gpu
      exactly:
        deviceClassName: gpu.nvidia.com
        count: 1
    config:
    - requests: ["gpu"]
      opaque:
        driver: gpu.nvidia.com
        parameters:
          apiVersion: resource.nvidia.com/v1beta1
          kind: GpuConfig
          sharing:
            strategy: TimeSlicing
            timeSlicingConfig:
              interval: Long
EOF

Deploy two or more Pods that reference the shared ResourceClaim by name. Each Pod references the claim through resourceClaims and resources.claims.


cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: Pod
metadata:
  name: share-a
spec:
  tolerations:
    - key: nvidia.com/gpu
      operator: Exists
      effect: NoSchedule
  containers:
    - name: cuda
      image: nvidia/cuda:12.6.0-base-ubuntu22.04
      command: ["nvidia-smi", "-L"]
      resources:
        claims:
        - name: gpu
  resourceClaims:
    - name: gpu
      resourceClaimName: shared-timeslice-gpu
  restartPolicy: OnFailure
EOF

Verify that the Pods share the same physical GPU. Run nvidia-smi -L in each Pod and confirm they report the same GPU UUID.
```
kubectl logs share-a
```
An example output is as follows. A second Pod that references the same claim reports an identical GPU UUID, which confirms that both Pods share one physical GPU.
```
GPU 0: NVIDIA L4 (UUID: GPU-b41973ee-5d0a-cde8-6287-12b53f861f02)
```

Use system-mediated GPU time-slicing with Karpenter

System-mediated time-slicing divides each GPU into a fixed number of shares, or carves it up by memory, and lets independent ResourceClaim objects consume those shares. Unlike user-mediated time-slicing, Pods do not have to reference a common claim and do not have to be in the same namespace. Kubernetes consumable capacity tracks the shares and enforces the limit.

This model also tells Karpenter how many claims fit on each GPU. Karpenter packs claims onto the GPUs of the nodes it plans and launches a new node only when the existing GPUs run out of shares, rather than treating every claim as a whole GPU.

System-mediated time-slicing is alpha

System-mediated time-slicing is alpha. It requires the alpha ConsumableShares feature gate in the NVIDIA DRA driver and the alpha DRANVIDIAGPU feature gate in Karpenter. Turn these on only if you accept the risks of using alpha capabilities.

Prerequisites

An Amazon EKS cluster running Kubernetes version 1.36 or later, where DRA consumable capacity is enabled by default.
Karpenter version 1.15 or later with the DRANVIDIAGPU feature gate enabled. For more information, see Using DRA with Karpenter.
Nodes with NVIDIA GPU instance types using the EKS-optimized AL2023 NVIDIA AMI, or Bottlerocket with the built-in NVIDIA device plugin disabled.
kubectl configured to communicate with your cluster, see Install or update kubectl for more information.

Procedure

Install the NVIDIA DRA driver as described in Install the NVIDIA DRA driver, adding the ConsumableShares feature gate and the number of shares per GPU. This is a cluster-wide driver setting. The following example configures four shares per GPU.
```
--set featureGates.ConsumableShares=true \
--set consumableShares=4
```

Tell Karpenter the same sharing mode with the karpenter.k8s.aws/nvidia-consumable-capacity annotation on your EC2NodeClass. Karpenter cannot read the driver’s configuration, so this value must match the driver’s consumableShares setting.


apiVersion: karpenter.k8s.aws/v1
kind: EC2NodeClass
metadata:
  name: gpu-shared
  annotations:
    karpenter.k8s.aws/nvidia-consumable-capacity: "4"
spec:
  # ...

The annotation accepts the following values.

Value	How Karpenter models each GPU
Absent or `disabled`	Not shareable. Each GPU serves one claim.
A positive integer, such as `4`	Shareable with that many shares. A claim consumes one share by default, and its memory request defaults to `0`.
`memory`	Shareable by memory. A claim that does not request memory takes the GPU’s full memory.
`unlimited`	Shareable without a limit. A claim’s memory request defaults to `0`.

Create a ResourceClaimTemplate. A plain whole-GPU request consumes the default of one share. To request more than one share, or to request memory in memory mode, set the capacity explicitly.


cat <<EOF | kubectl apply -f -
apiVersion: resource.k8s.io/v1
kind: ResourceClaimTemplate
metadata:
  name: gpu-share
spec:
  spec:
    devices:
      requests:
      - name: gpu
        exactly:
          deviceClassName: gpu.nvidia.com
          capacity:
            requests:
              shares: "1"
EOF

Deploy a workload that references the template. Each Pod gets its own generated ResourceClaim. With four shares per GPU, the following eight replicas occupy two GPUs.


cat <<EOF | kubectl apply -f -
apiVersion: apps/v1
kind: Deployment
metadata:
  name: gpu-share-demo
spec:
  replicas: 8
  selector:
    matchLabels:
      app: gpu-share-demo
  template:
    metadata:
      labels:
        app: gpu-share-demo
    spec:
      tolerations:
        - key: nvidia.com/gpu
          operator: Exists
          effect: NoSchedule
      containers:
        - name: cuda
          image: nvidia/cuda:12.6.0-base-ubuntu22.04
          command: ["bash", "-c", "nvidia-smi --query-gpu=uuid --format=csv,noheader; sleep infinity"]
          resources:
            claims:
            - name: gpu
      resourceClaims:
        - name: gpu
          resourceClaimTemplateName: gpu-share
EOF

Delete the workload when you are finished, so that the GPU nodes it provisioned are released. The ResourceClaimTemplate can stay in place for future workloads.
```
kubectl delete deployment gpu-share-demo
kubectl delete resourceclaimtemplate gpu-share
```

Considerations

The annotation must match the driver: If the annotation claims more shares than the driver provides, Karpenter over-packs. Pods schedule onto a node that cannot serve them and stay Pending. Karpenter cannot detect this.
Validation errors: An invalid annotation value sets ValidationSucceeded=False on the EC2NodeClass with the reason NVIDIAConsumableCapacityInvalid. Enabling sharing while the DRANVIDIAGPU feature gate is off sets ValidationSucceeded=False with the reason DRADisabled. In both cases, Karpenter does not launch nodes for that EC2NodeClass. An absent or disabled value never blocks provisioning.
Sharing affects scheduling only: Shares and memory requests govern how many claims the scheduler places on a GPU. They do not cap what a container actually consumes on the GPU, so the general time-slicing considerations about memory and compute isolation still apply. For more information, see Considerations.
Cluster-wide driver setting: consumableShares applies to every GPU the driver manages in the cluster. To run different sharing modes side by side, you need separate clusters until the NVIDIA DRA driver supports per-node and per-GPU share configuration. For more information, see NVIDIA DRA driver issue #1284 on GitHub.
Not available in EKS Auto Mode: EKS Auto Mode does not enable the alpha NVIDIA DRA driver capabilities that system-mediated time-slicing requires.
Time-slice intervals are separate: The interval setting described in Configuration options belongs to user-mediated time-slicing and the TimeSlicingSettings feature gate. It does not apply to consumable shares.

For the current annotation values and supported instance types, see Utilizing GPUs and EFAs with Dynamic Resource Allocation in the Karpenter documentation.

Use GPU time-slicing on Bottlerocket nodes with the NVIDIA device plugin

On Bottlerocket, the EKS-optimized accelerated AMI includes the NVIDIA device plugin. You enable time-slicing through the settings.kubelet-device-plugins.nvidia settings, which Bottlerocket renders into the device plugin configuration when the node boots.

Prerequisites

An Amazon EKS cluster. The following procedure provisions NVIDIA GPU nodes with the EKS-optimized Bottlerocket NVIDIA AMI.
Karpenter installed and configured in your cluster, because the following procedure uses a Karpenter EC2NodeClass to supply the time-slicing settings in the Bottlerocket node user data. For more information, see the Getting Started with Karpenter on the Karpenter website.
kubectl configured to communicate with your cluster, see Install or update kubectl for more information.

Procedure

Add the time-slicing settings to the Bottlerocket user data for your GPU nodes. The following example configures four slots per GPU. The way you supply user data depends on how you provision nodes. The following example shows a Karpenter EC2NodeClass.


cat <<EOF | kubectl apply -f -
apiVersion: karpenter.k8s.aws/v1
kind: EC2NodeClass
metadata:
  name: gpu-bottlerocket-timeslicing
spec:
  amiFamily: Bottlerocket
  amiSelectorTerms:
    - alias: bottlerocket@latest
  role: eksctl-KarpenterNodeRole-<cluster-name>
  subnetSelectorTerms:
    - tags:
        karpenter.sh/discovery: <cluster-name>
  securityGroupSelectorTerms:
    - tags:
        karpenter.sh/discovery: <cluster-name>
  userData: |
    [settings.kubelet-device-plugins.nvidia]
    device-sharing-strategy = "time-slicing"

    [settings.kubelet-device-plugins.nvidia.time-slicing]
    replicas = 4
    rename-by-default = false
    fail-requests-greater-than-one = true
EOF

When nodes provisioned with these settings join the cluster, the device plugin advertises four nvidia.com/gpu slots for each physical GPU. You request them in your workload specification with container resources requests or limits for the nvidia.com/gpu extended resource.

Use GPU time-slicing on AL2023 nodes with the NVIDIA device plugin

On AL2023, you install the NVIDIA device plugin as described in Install the NVIDIA Kubernetes device plugin, and supply the time-slicing configuration in a ConfigMap.

Prerequisites

An Amazon EKS cluster with nodes that use NVIDIA GPU instance types and the EKS-optimized AL2023 NVIDIA AMI.
Helm installed in your command-line environment, see the Setup Helm instructions for more information.
kubectl configured to communicate with your cluster, see Install or update kubectl for more information.

Procedure

Create the time-slicing ConfigMap in the nvidia namespace, where you installed the device plugin in Install the NVIDIA Kubernetes device plugin. This example configures four slots per GPU.


cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: ConfigMap
metadata:
  name: nvidia-device-plugin-config
  namespace: nvidia
data:
  config.yaml: |
    version: v1
    sharing:
      timeSlicing:
        renameByDefault: false
        failRequestsGreaterThanOne: true
        resources:
          - name: nvidia.com/gpu
            replicas: 4
EOF

Update the existing NVIDIA device plugin release to reference the ConfigMap with the config.name value. The --reuse-values flag preserves the values you set when you installed the device plugin in Install the NVIDIA Kubernetes device plugin.
```
helm upgrade nvdp nvdp/nvidia-device-plugin \
    --namespace nvidia \
    --reuse-values \
    --set config.name=nvidia-device-plugin-config
```

Note

The device plugin does not reload automatically when you change the ConfigMap. After you update the time-slicing configuration, restart the device plugin Pods to apply the change.

Verify that GPU time-slicing is active

After your time-sliced nodes are Ready, confirm that the device plugin advertises the expected number of slots and that Pods share a physical GPU.

Note

The shared-UUID check in this procedure demonstrates time-slicing most clearly when the Pods land on the same physical GPU. On a node with a single GPU, the device plugin advertises four slots and all four Pods share that GPU, so they report the same UUID. On a node with multiple physical GPUs, the scheduler can place Pods on different GPUs. Those Pods report different UUIDs even though time-slicing is active. To demonstrate sharing on one GPU, schedule the workload onto a single-GPU instance type. For example, add a node selector such as node.kubernetes.io/instance-type: g6.2xlarge to the Pod specification.

Confirm that the node advertises the configured number of GPU slots. With four slots per GPU, a node with one physical GPU reports 4.


kubectl get nodes "-o=custom-columns=NAME:.metadata.name,GPU:.status.allocatable.nvidia\.com/gpu"

An example output is as follows.


NAME                                           GPU
ip-192-168-11-225.us-west-2.compute.internal   4

Create a deployment that runs four replicas, each requesting one GPU slot.


cat <<EOF | kubectl apply -f -
apiVersion: apps/v1
kind: Deployment
metadata:
  name: timeslicing-demo
spec:
  replicas: 4
  selector:
    matchLabels:
      app: timeslicing-demo
  template:
    metadata:
      labels:
        app: timeslicing-demo
    spec:
      tolerations:
        - key: nvidia.com/gpu
          operator: Exists
          effect: NoSchedule
      containers:
        - name: cuda
          image: nvidia/cuda:12.6.0-base-ubuntu22.04
          command: ["bash", "-c", "nvidia-smi --query-gpu=uuid --format=csv,noheader; sleep infinity"]
          resources:
            limits:
              nvidia.com/gpu: 1
EOF

Confirm that all four Pods are scheduled on the same node.
```
kubectl get pods -l app=timeslicing-demo -o wide
```

Confirm that all four Pods report the same GPU UUID. A single shared UUID across all four Pods confirms that one physical GPU is being time-multiplexed.


kubectl logs -l app=timeslicing-demo --prefix

An example output is as follows.


[pod/timeslicing-demo-xxxxxxxxxx-aaaaa/cuda] GPU-c0583cce-87c5-c736-db7f-6d3128c84d03
[pod/timeslicing-demo-xxxxxxxxxx-bbbbb/cuda] GPU-c0583cce-87c5-c736-db7f-6d3128c84d03
[pod/timeslicing-demo-xxxxxxxxxx-ccccc/cuda] GPU-c0583cce-87c5-c736-db7f-6d3128c84d03
[pod/timeslicing-demo-xxxxxxxxxx-ddddd/cuda] GPU-c0583cce-87c5-c736-db7f-6d3128c84d03

Javascript is disabled or is unavailable in your browser.

To use the Amazon Web Services Documentation, Javascript must be enabled. Please refer to your browser's Help pages for instructions.

Multi-instance GPUs (MIG)

Recipes