diff --git a/app/_data/ai-gateway/v2/providers.yaml b/app/_data/ai-gateway/v2/providers.yaml index 21862b37f70..332ab394d7f 100644 --- a/app/_data/ai-gateway/v2/providers.yaml +++ b/app/_data/ai-gateway/v2/providers.yaml @@ -1561,3 +1561,77 @@ providers: limitations: provider_specific: [] statistics_logging: [] + - name: Amazon SageMaker + url_patterns: + - 'https://runtime.sagemaker.{region}.amazonaws.com' + min_version: '2.0' + capabilities: + generate: + supported: true + streaming: true + upstream_path: '`/endpoints/{endpoint_name}/invocations` (or `/endpoints/{endpoint_name}/invocations-response-stream` for streaming)' + model_example: 'User-defined (the name of your SageMaker endpoint)' + min_version: '2.0' + completions: + supported: false + streaming: false + model_example: '' + min_version: '' + embeddings: + supported: false + streaming: false + model_example: '' + min_version: '' + files: + supported: false + streaming: false + model_example: '' + min_version: '' + batches: + supported: false + streaming: false + model_example: '' + min_version: '' + agentic: + supported: false + streaming: false + model_example: '' + min_version: '' + audio_speech: + supported: false + streaming: false + model_example: '' + min_version: '' + audio_transcription: + supported: false + streaming: false + model_example: '' + min_version: '' + audio_translation: + supported: false + streaming: false + model_example: '' + min_version: '' + image: + supported: false + streaming: false + model_example: '' + min_version: '' + video: + supported: false + streaming: false + model_example: '' + min_version: '' + realtime: + supported: false + streaming: false + model_example: '' + min_version: '' + rerank: + supported: false + streaming: false + model_example: '' + min_version: '' + limitations: + provider_specific: [] + statistics_logging: [] diff --git a/app/_includes/md/ai-gateway/v2/aws-auth.md b/app/_includes/md/ai-gateway/v2/aws-auth.md new file mode 100644 index 00000000000..7dff554c642 --- /dev/null +++ b/app/_includes/md/ai-gateway/v2/aws-auth.md @@ -0,0 +1,11 @@ +## Authentication with AWS + +For {{ provider.name }}, you can also set `auth` to `aws`. Provide static IAM user credentials with `access_key_id` and `secret_access_key`, or omit them to fall back to the default AWS credentials provider chain (EC2 instance profiles, environment variables, and so on). For cross-account access, assume a role with `assume_role_arn` and `role_session_name`. + +See [Outbound authentication](/ai-gateway/entities/ai-model-provider/#outbound-authentication) on the AI Model Provider entity page for the full list of `auth` fields, including `sts_endpoint_url` and the Bedrock-specific `batch_role_arn`. + +{:.info} +> **AWS Session tokens** +> +> There's no dedicated field for an AWS session token. +> For temporary credentials, set `config.auth.assume_role_arn` and `config.auth.role_session_name` to assume a role through AWS STS, or leave `config.auth.access_key_id` and `config.auth.secret_access_key` unset to fall back to environment variables or an instance or task IAM role. diff --git a/app/_landing_pages/ai-gateway/ai-providers.yaml b/app/_landing_pages/ai-gateway/ai-providers.yaml index 070a95eace4..5d2de881bd7 100644 --- a/app/_landing_pages/ai-gateway/ai-providers.yaml +++ b/app/_landing_pages/ai-gateway/ai-providers.yaml @@ -54,6 +54,13 @@ rows: icon: /assets/icons/bedrock.svg cta: url: /ai-gateway/ai-providers/bedrock/ + - blocks: + - type: icon_card + config: + title: Amazon SageMaker + icon: /assets/icons/sagemaker.svg + cta: + url: /ai-gateway/ai-providers/sagemaker/ - blocks: - type: icon_card config: diff --git a/app/ai-gateway/ai-providers/bedrock.md b/app/ai-gateway/ai-providers/bedrock.md index 83091f8aa03..5923f8445b7 100644 --- a/app/ai-gateway/ai-providers/bedrock.md +++ b/app/ai-gateway/ai-providers/bedrock.md @@ -17,6 +17,7 @@ products: tools: - konnect-api + - kongctl tags: - ai @@ -64,14 +65,9 @@ To use {{ provider.name }} with {{site.ai_gateway}}, configure a new [AI Model P Here's a minimal configuration for chat completions: - -{% konnect_api_request %} -url: /v1/ai-gateways/$AI_GATEWAY_ID/model-providers -status_code: 201 -method: POST -headers: - - 'Content-Type: application/json' -body: +{% entity_example %} +type: model-provider +data: display_name: AWS Production name: my-aws-account type: bedrock @@ -80,16 +76,6 @@ body: type: aws access_key_id: $AWS_ACCESS_KEY_ID secret_access_key: $AWS_SECRET_ACCESS_KEY -{% endkonnect_api_request %} - +{% endentity_example %} -## Authentication with AWS - -You can also use {{ provider.name }} with AWS credentials by setting `auth` to `aws` and specifying: - -* **`access_key_id`** (optional): AWS access key ID for static IAM user credentials. If omitted, the default AWS credentials provider chain is used (EC2 instance profiles, environment variables, etc.). -* **`secret_access_key`** (optional): AWS secret access key paired with `access_key_id`. Required if `access_key_id` is set. -* **`assume_role_arn`** (optional): IAM role ARN to assume for temporary credentials. Useful for cross-account access. -* **`role_session_name`** (optional): Session name for the assumed role. Required if `assume_role_arn` is set. -* **`sts_endpoint_url`** (optional): Custom STS endpoint for role assumption. Defaults to `https://sts.amazonaws.com`. -* **`batch_role_arn`** (optional): Separate role ARN for Bedrock batch API calls. \ No newline at end of file +{% include md/ai-gateway/v2/aws-auth.md %} \ No newline at end of file diff --git a/app/ai-gateway/ai-providers/sagemaker.md b/app/ai-gateway/ai-providers/sagemaker.md new file mode 100644 index 00000000000..968e1c86441 --- /dev/null +++ b/app/ai-gateway/ai-providers/sagemaker.md @@ -0,0 +1,89 @@ +--- +title: "Amazon SageMaker provider" +layout: reference +content_type: reference +description: Reference for supported capabilities for Amazon SageMaker provider +breadcrumbs: + - /ai-gateway/ + - /ai-gateway/ai-providers/ + +permalink: /ai-gateway/ai-providers/sagemaker/ + +works_on: + - konnect + +products: + - ai-gateway + +tools: + - konnect-api + - kongctl + +tags: + - ai + +min_version: + ai-gateway: '2.0' + +related_resources: + - text: "{{site.ai_gateway}}" + url: /ai-gateway/ + - text: "{{site.ai_gateway}} Policies" + url: /ai-gateway/policies/ + - text: AI Providers + url: /ai-gateway/ai-providers/ + - text: AI Model Provider entity + url: /ai-gateway/entities/ai-model-provider/ + - text: AI Model entity + url: /ai-gateway/entities/ai-model/ + +--- + + +{% include md/ai-gateway/v2/providers.md providers=site.data.ai-gateway.v2.providers provider_name="Amazon SageMaker" %} + +{% include md/ai-gateway/v2/native-routes.md providers=site.data.ai-gateway.v2.providers provider_name="Amazon SageMaker" %} + +## Configure {{ provider.name }} + +To use {{ provider.name }} with {{site.ai_gateway}}, configure a new [AI Model Provider](/ai-gateway/entities/ai-model-provider/). You can then access supported [AI Models](/ai-gateway/entities/ai-model/) from {{ provider.name }}. + +Here's a minimal configuration for chat completions: + +{% entity_example %} +type: model-provider +data: + display_name: SageMaker Production + name: my-sagemaker-account + type: sagemaker + config: + auth: + type: aws + access_key_id: $AWS_ACCESS_KEY_ID + secret_access_key: $AWS_SECRET_ACCESS_KEY +{% endentity_example %} + +{% include md/ai-gateway/v2/aws-auth.md %} + +## Configure a model target for {{ provider.name }} + +Only the `llm/v1/chat` route type is supported for {{ provider.name }} targets. + +A [target](/ai-gateway/entities/ai-model/#targets) is an entry in the `targets` array on the AI Model entity, not the AI Model Provider. The target `name` is the name of your SageMaker endpoint. Beyond the common target options (`name`, `provider`, `weight`), a target routing to {{ provider.name }} supports these `config` fields for multi-model, multi-variant, and multi-container endpoints: + +* **`region`**: The AWS region hosting the SageMaker endpoint. +* **`target_model`** (optional): The model artifact to invoke on a multi-model endpoint. +* **`target_variant`** (optional): The production variant to invoke on a multi-variant endpoint. +* **`target_container_hostname`** (optional): The container hostname to invoke on a multi-container endpoint. + +```yaml +targets: + - name: my-sagemaker-endpoint + provider: my-sagemaker-account + config: + type: sagemaker + region: us-east-1 + target_model: my-model.tar.gz + target_variant: production-variant-1 + target_container_hostname: container-1 +``` diff --git a/app/assets/icons/sagemaker.svg b/app/assets/icons/sagemaker.svg new file mode 100644 index 00000000000..542ea19dd3c --- /dev/null +++ b/app/assets/icons/sagemaker.svg @@ -0,0 +1,18 @@ + + + + Icon-Architecture/64/Arch_AWS-SageMaker_64 + Created with Sketch. + + + + + + + + + + + + + \ No newline at end of file