Use Jupyter AI in JupyterLab or Studio Classic - Amazon SageMaker
Use language models from the chat UIUse language models from a notebook
Services or capabilities described in Amazon Web Services documentation might vary by Region. To see the differences applicable to the China Regions, see Getting Started with Amazon Web Services in China (PDF).

Use Jupyter AI in JupyterLab or Studio Classic

Use language models from the chat UI

Compose your message in the chat UI text box to start interacting with your model. To clear the message history, use the /clear command.

Note

Clearing the message history does not erase the chat context with the model provider.

Use language models from notebook cells

Before using the %%ai and %ai commands to invoke a language model, load the IPython extension by running the following command in a JupyterLab or Studio Classic notebook cell.

%load_ext jupyter_ai_magics

  • For models hosted by Amazon:

    • To invoke a model deployed in SageMaker, pass the string sagemaker-endpoint:endpoint-name to the %%ai magic command with the required parameters below, then add your prompt in the following lines.

      The following table lists the required and optional parameters when invoking models hosted by SageMaker or Amazon Bedrock.

      Parameter Name Parameter Short Version Description
      Request schema --request-schema -q Required: The JSON object the endpoint expects, with the prompt being substituted into any value that matches the string literal <prompt>.
      Region name --region-name -n Required: The Amazon Web Services Region where the model is deployed.
      Response path --response-path -p Required: A JSONPath string used to extract the language model's output from the JSON response of the endpoint.
      Extra model parameters --model-parameters -m Optional: A JSON value specifying additional parameters to be passed to the model. The accepted value is parsed into a dictionary, unpacked, and directly passed to the provider class. This is useful when the endpoint or the model requires custom parameters. For example, in Llama 2 models when accepting the End User License Agreement (EULA) is necessary, you can pass the EULA acceptance to the endpoint using -m {"endpoint_kwargs":{"CustomAttributes":"accept_eula=true"}}. Alternatively, you can use the -m parameter to pass extra model parameters, such as setting the maximum number of tokens for a model's generated response. For example, when working with an AI21 Labs Jurassic model: -m {"model_kwargs":{"maxTokens":256}}.
      Output format --format -f Optional: The IPython display used to render the output. It can be any of the following values [code|html|image|json|markdown|math|md|text], provided that the invoked model supports the specified format.

      The following command invokes a Llama2-7b model hosted by SageMaker.

      %%ai sagemaker-endpoint:jumpstart-dft-meta-textgeneration-llama-2-7b -q {"inputs":"<prompt>","parameters":{"max_new_tokens":64,"top_p":0.9,"temperature":0.6,"return_full_text":false}} -n us-east-2 -p [0].generation -m {"endpoint_kwargs":{"CustomAttributes":"accept_eula=true"}} -f text
Translate English to French:
sea otter => loutre de mer
peppermint => menthe poivrée
plush girafe => girafe peluche
cheese =>

      The following example invokes a Flan-t5-small model hosted by SageMaker.

      %%ai sagemaker-endpoint:hf-text2text-flan-t5-small --request-schema={"inputs":"<prompt>","parameters":{"num_return_sequences":4}} --region-name=us-west-2 --response-path=[0]["generated_text"] -f text
What is the atomic number of Hydrogen?

    • To invoke a model deployed in Amazon Bedrock, pass the string bedrock:model-name to the %%ai magic command with any optional parameter defined in the list of parameters for invoking models hosted by JumpStart or Amazon Bedrock, then add your prompt in the following lines.

      The following example invokes an AI21 Labs Jurassic-2 model hosted by Amazon Bedrock.

      %%ai bedrock:ai21.j2-mid-v1 -m {"model_kwargs":{"maxTokens":256}} -f code
Write a function in python implementing a bubbble sort.

  • For models hosted by third-party providers

    To invoke a model hosted by third-party providers, pass the string provider-id:model-name to the %%ai magic command with an optional Output format, then add your prompt in the following lines. You can find the details of each provider, including their ID, in the Jupyter AI list of model providers.

    The following command asks an Anthropic Claude model to output an HTML file containing the image of a white square with black borders.

    %%ai anthropic:claude-v1.2 -f html
Create a square using SVG with a black border and white fill.