As the error says, you must provide the ID of an inference profile and not the model for this particular model. The easiest way to do this is to provide the ID of a system-defined inference profile for this model. You can find it by invoking this awscli command with the correct credentials defined in the environment (or set via standard flags):
aws bedrock list-inference-profiles
You will see this one in the JSON list:
{
"inferenceProfileName": "US Anthropic Claude 3.5 Sonnet v2",
"description": "Routes requests to Anthropic Claude 3.5 Sonnet v2 in us-west-2, us-east-1 and us-east-2.",
"inferenceProfileArn": "arn:aws:bedrock:us-east-1:381492273274:inference-profile/us.anthropic.claude-3-5-sonnet-20241022-v2:0",
"models": [
{
"modelArn": "arn:aws:bedrock:us-west-2::foundation-model/anthropic.claude-3-5-sonnet-20241022-v2:0"
},
{
"modelArn": "arn:aws:bedrock:us-east-1::foundation-model/anthropic.claude-3-5-sonnet-20241022-v2:0"
},
{
"modelArn": "arn:aws:bedrock:us-east-2::foundation-model/anthropic.claude-3-5-sonnet-20241022-v2:0"
}
],
"inferenceProfileId": "us.anthropic.claude-3-5-sonnet-20241022-v2:0",
"status": "ACTIVE",
"type": "SYSTEM_DEFINED"
}
Modify the invoke_model line in your code to specify the ID or ARN of the inference profile instead:
response = bedrock_runtime.invoke_model(
body=body,
modelId="us.anthropic.claude-3-5-sonnet-20241022-v2:0",
)
Videos
As the error says, you must provide the ID of an inference profile and not the model for this particular model. The easiest way to do this is to provide the ID of a system-defined inference profile for this model. You can find it by invoking this awscli command with the correct credentials defined in the environment (or set via standard flags):
aws bedrock list-inference-profiles
You will see this one in the JSON list:
{
"inferenceProfileName": "US Anthropic Claude 3.5 Sonnet v2",
"description": "Routes requests to Anthropic Claude 3.5 Sonnet v2 in us-west-2, us-east-1 and us-east-2.",
"inferenceProfileArn": "arn:aws:bedrock:us-east-1:381492273274:inference-profile/us.anthropic.claude-3-5-sonnet-20241022-v2:0",
"models": [
{
"modelArn": "arn:aws:bedrock:us-west-2::foundation-model/anthropic.claude-3-5-sonnet-20241022-v2:0"
},
{
"modelArn": "arn:aws:bedrock:us-east-1::foundation-model/anthropic.claude-3-5-sonnet-20241022-v2:0"
},
{
"modelArn": "arn:aws:bedrock:us-east-2::foundation-model/anthropic.claude-3-5-sonnet-20241022-v2:0"
}
],
"inferenceProfileId": "us.anthropic.claude-3-5-sonnet-20241022-v2:0",
"status": "ACTIVE",
"type": "SYSTEM_DEFINED"
}
Modify the invoke_model line in your code to specify the ID or ARN of the inference profile instead:
response = bedrock_runtime.invoke_model(
body=body,
modelId="us.anthropic.claude-3-5-sonnet-20241022-v2:0",
)
You can add the ARN of an Inference profile to your modelID itself while invoking the model.
response = bedrock_client.invoke_model(
modelId="arn:aws:bedrock:us-east-1::model/your-bedrock-model-arn"
prompt="Your prompt here"
)
Hey, i have been using `http://anthropic.claude-3-5-sonnet-20240620-v1:0/` with great success from us-east-1. Today, I tried to pivot to v2 which has an inference profile ID of `us.anthropic.claude-3-5-sonnet-20241022-v2:0`. The model seems to be available in both us-east-1 and in us-west-2.
I am getting denied with a error code of 403 (not authorised) to access the model. Which doesn't make sense as I have access to the model in us-east-1. However, I noticed that the API tries to access the model from us-west-2, `arn:aws:bedrock:us-west-2::foundation-model/anthropic.claude-3-5-sonnet-20241022-v2:0` even though I am manually selecting aws_region of "us-east-1".
The same problem when I am using the Inference profile ARN (arn:aws:bedrock:us-east-1:422697756335:inference-profile/us.anthropic.claude-3-5-sonnet-20241022-v2:0). Any ideas why the API seems to force a us-west-2 access?
By the way, I don't have access to us-west-2
Here's more info on how to use Anthropic’s Claude 3.5 Sonnet on Amazon Bedrock with the console, the AWS CLI, and AWS SDKs (Python/Boto3):
https://aws.amazon.com/blogs/aws/anthropics-claude-3-5-sonnet-model-now-available-in-amazon-bedrock-the-most-intelligent-claude-model-yet/
Hi,
I have been utilizing AWS Bedrock hosted version of Claude 3.5 extensively for my clients and the convenience and ease of use is unmatched.
However, there seems to be an issue with Amazon or Anthropic updating the model to the latest version meaning last month (July) Anthropic updated Sonnet to have an output token length of 8192 tokens, but it requires extra headers to be passed (when using their native API).
I tried it with Bedrock, but to my surprise I realized Bedrock models are hardly being updated from the first time they are launched and it still doesn't have the latest update of the models which is a but frustrating as I already have a use case for the 8000 context window and the client doesn't want to use LLM outside of Bedrock.
Does anyone know if AWS or Anthropic will update the model to July version?
X discussion (for native Anthropic API): https://x.com/alexalbert__/status/1812921642143900036?t=NH_I5sEF2Z7UjQecleEaug&s=19