Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
74 changes: 74 additions & 0 deletions app/_data/ai-gateway/v2/providers.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -1561,3 +1561,77 @@ providers:
limitations:
provider_specific: []
statistics_logging: []
- name: Amazon SageMaker
url_patterns:
- 'https://runtime.sagemaker.{region}.amazonaws.com'
min_version: '2.0'
capabilities:
generate:
supported: true
streaming: true
upstream_path: '`/endpoints/{endpoint_name}/invocations` (or `/endpoints/{endpoint_name}/invocations-response-stream` for streaming)'
model_example: 'User-defined (the name of your SageMaker endpoint)'
min_version: '2.0'
completions:
supported: false
streaming: false
model_example: ''
min_version: ''
embeddings:
supported: false
streaming: false
model_example: ''
min_version: ''
files:
supported: false
streaming: false
model_example: ''
min_version: ''
batches:
supported: false
streaming: false
model_example: ''
min_version: ''
agentic:
supported: false
streaming: false
model_example: ''
min_version: ''
audio_speech:
supported: false
streaming: false
model_example: ''
min_version: ''
audio_transcription:
supported: false
streaming: false
model_example: ''
min_version: ''
audio_translation:
supported: false
streaming: false
model_example: ''
min_version: ''
image:
supported: false
streaming: false
model_example: ''
min_version: ''
video:
supported: false
streaming: false
model_example: ''
min_version: ''
realtime:
supported: false
streaming: false
model_example: ''
min_version: ''
rerank:
supported: false
streaming: false
model_example: ''
min_version: ''
limitations:
provider_specific: []
statistics_logging: []
11 changes: 11 additions & 0 deletions app/_includes/md/ai-gateway/v2/aws-auth.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
## Authentication with AWS

For {{ provider.name }}, you can also set `auth` to `aws`. Provide static IAM user credentials with `access_key_id` and `secret_access_key`, or omit them to fall back to the default AWS credentials provider chain (EC2 instance profiles, environment variables, and so on). For cross-account access, assume a role with `assume_role_arn` and `role_session_name`.

See [Outbound authentication](/ai-gateway/entities/ai-model-provider/#outbound-authentication) on the AI Model Provider entity page for the full list of `auth` fields, including `sts_endpoint_url` and the Bedrock-specific `batch_role_arn`.

{:.info}
> **AWS Session tokens**
>
> There's no dedicated field for an AWS session token.
> For temporary credentials, set `config.auth.assume_role_arn` and `config.auth.role_session_name` to assume a role through AWS STS, or leave `config.auth.access_key_id` and `config.auth.secret_access_key` unset to fall back to environment variables or an instance or task IAM role.
7 changes: 7 additions & 0 deletions app/_landing_pages/ai-gateway/ai-providers.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,13 @@ rows:
icon: /assets/icons/bedrock.svg
cta:
url: /ai-gateway/ai-providers/bedrock/
- blocks:
- type: icon_card
config:
title: Amazon SageMaker
icon: /assets/icons/sagemaker.svg
cta:
url: /ai-gateway/ai-providers/sagemaker/
- blocks:
- type: icon_card
config:
Expand Down
26 changes: 6 additions & 20 deletions app/ai-gateway/ai-providers/bedrock.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ products:

tools:
- konnect-api
- kongctl

tags:
- ai
Expand Down Expand Up @@ -64,14 +65,9 @@ To use {{ provider.name }} with {{site.ai_gateway}}, configure a new [AI Model P

Here's a minimal configuration for chat completions:

<!--vale off-->
{% konnect_api_request %}
url: /v1/ai-gateways/$AI_GATEWAY_ID/model-providers
status_code: 201
method: POST
headers:
- 'Content-Type: application/json'
body:
{% entity_example %}
type: model-provider
data:
display_name: AWS Production
name: my-aws-account
type: bedrock
Expand All @@ -80,16 +76,6 @@ body:
type: aws
access_key_id: $AWS_ACCESS_KEY_ID
secret_access_key: $AWS_SECRET_ACCESS_KEY
{% endkonnect_api_request %}
<!--vale on-->
{% endentity_example %}

## Authentication with AWS

You can also use {{ provider.name }} with AWS credentials by setting `auth` to `aws` and specifying:

* **`access_key_id`** (optional): AWS access key ID for static IAM user credentials. If omitted, the default AWS credentials provider chain is used (EC2 instance profiles, environment variables, etc.).
* **`secret_access_key`** (optional): AWS secret access key paired with `access_key_id`. Required if `access_key_id` is set.
* **`assume_role_arn`** (optional): IAM role ARN to assume for temporary credentials. Useful for cross-account access.
* **`role_session_name`** (optional): Session name for the assumed role. Required if `assume_role_arn` is set.
* **`sts_endpoint_url`** (optional): Custom STS endpoint for role assumption. Defaults to `https://sts.amazonaws.com`.
* **`batch_role_arn`** (optional): Separate role ARN for Bedrock batch API calls.
{% include md/ai-gateway/v2/aws-auth.md %}
89 changes: 89 additions & 0 deletions app/ai-gateway/ai-providers/sagemaker.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,89 @@
---
title: "Amazon SageMaker provider"
layout: reference
content_type: reference
description: Reference for supported capabilities for Amazon SageMaker provider
breadcrumbs:
- /ai-gateway/
- /ai-gateway/ai-providers/

permalink: /ai-gateway/ai-providers/sagemaker/

works_on:
- konnect

products:
- ai-gateway

tools:
- konnect-api
- kongctl

tags:
- ai

min_version:
ai-gateway: '2.0'

related_resources:
- text: "{{site.ai_gateway}}"
url: /ai-gateway/
- text: "{{site.ai_gateway}} Policies"
url: /ai-gateway/policies/
- text: AI Providers
url: /ai-gateway/ai-providers/
- text: AI Model Provider entity
url: /ai-gateway/entities/ai-model-provider/
- text: AI Model entity
url: /ai-gateway/entities/ai-model/

---


{% include md/ai-gateway/v2/providers.md providers=site.data.ai-gateway.v2.providers provider_name="Amazon SageMaker" %}

{% include md/ai-gateway/v2/native-routes.md providers=site.data.ai-gateway.v2.providers provider_name="Amazon SageMaker" %}

## Configure {{ provider.name }}

To use {{ provider.name }} with {{site.ai_gateway}}, configure a new [AI Model Provider](/ai-gateway/entities/ai-model-provider/). You can then access supported [AI Models](/ai-gateway/entities/ai-model/) from {{ provider.name }}.

Here's a minimal configuration for chat completions:

{% entity_example %}
type: model-provider
data:
display_name: SageMaker Production
name: my-sagemaker-account
type: sagemaker
config:
auth:
type: aws
access_key_id: $AWS_ACCESS_KEY_ID
secret_access_key: $AWS_SECRET_ACCESS_KEY
{% endentity_example %}

{% include md/ai-gateway/v2/aws-auth.md %}

## Configure a model target for {{ provider.name }}

Only the `llm/v1/chat` route type is supported for {{ provider.name }} targets.

A [target](/ai-gateway/entities/ai-model/#targets) is an entry in the `targets` array on the AI Model entity, not the AI Model Provider. The target `name` is the name of your SageMaker endpoint. Beyond the common target options (`name`, `provider`, `weight`), a target routing to {{ provider.name }} supports these `config` fields for multi-model, multi-variant, and multi-container endpoints:

* **`region`**: The AWS region hosting the SageMaker endpoint.
* **`target_model`** (optional): The model artifact to invoke on a multi-model endpoint.
* **`target_variant`** (optional): The production variant to invoke on a multi-variant endpoint.
* **`target_container_hostname`** (optional): The container hostname to invoke on a multi-container endpoint.

```yaml
targets:
- name: my-sagemaker-endpoint
provider: my-sagemaker-account
config:
type: sagemaker
region: us-east-1
target_model: my-model.tar.gz
target_variant: production-variant-1
target_container_hostname: container-1
```
18 changes: 18 additions & 0 deletions app/assets/icons/sagemaker.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading