AWS Machine Learning

JANUARY 10, 2023

This post describes the best practices for load testing a SageMaker endpoint to find the right configuration for the number of instances and size. Note that the model container also includes any custom inference code or scripts that you have passed for inference.

Best practices 91

Best practices Scripts APIs Metrics 91

AWS Machine Learning

OCTOBER 2, 2024

This two-part series explores best practices for building generative AI applications using Amazon Bedrock Agents. This data provides a benchmark for expected agent behavior, including the interaction with existing APIs, knowledge bases, and guardrails connected with the agent.

Best practices 98

Best practices APIs Metrics Accountability 98

AWS Machine Learning

MARCH 15, 2023

The best practice for migration is to refactor these legacy codes using the Amazon SageMaker API or the SageMaker Python SDK. SageMaker runs the legacy script inside a processing container. SageMaker takes your script, copies your data from Amazon Simple Storage Service (Amazon S3), and then pulls a processing container.

Scripts 140

Scripts APIs Engineering Construction 140

AWS Machine Learning

MARCH 27, 2025

Refer to Getting started with the API to set up your environment to make Amazon Bedrock requests through the AWS API. Test the code using the native inference API for Anthropics Claude The following code uses the native inference API to send a text message to Anthropics Claude. client = boto3.client("bedrock-runtime",

Education 95

Education Engineering APIs Enterprise 95

AWS Machine Learning

FEBRUARY 8, 2024

At the forefront of this evolution sits Amazon Bedrock , a fully managed service that makes high-performing foundation models (FMs) from Amazon and other leading AI companies available through an API. System integration – Agents make API calls to integrated company systems to run specific actions.

APIs 140

APIs Scripts Enterprise Management 140

AWS Machine Learning

FEBRUARY 5, 2025

Some links for security best practices are shared below but we strongly recommend reaching out to your account team for detailed guidance and to discuss the appropriate security architecture needed for a secure and compliant deployment. model API exposed by SageMaker JumpStart properly. define bot express greeting "Hey there!"

Chatbots 98

Chatbots Construction Best practices APIs 98

AWS Machine Learning

JULY 11, 2024

This solution uses Retrieval Augmented Generation (RAG) to ensure the generated scripts adhere to organizational needs and industry standards. In this blog post, we explore how Agents for Amazon Bedrock can be used to generate customized, organization standards-compliant IaC scripts directly from uploaded architecture diagrams.

Scripts 134

Scripts APIs Engineering industry standards 134

AWS Machine Learning

JULY 17, 2024

This post shows how to use AWS generative artificial intelligence (AI) services , like Amazon Q Business , with AWS Support cases, AWS Trusted Advisor , and AWS Health data to derive actionable insights based on common patterns, issues, and resolutions while using the AWS recommendations and best practices enabled by support data.

Scripts 120

Scripts APIs Enterprise Accountability 120

AWS Machine Learning

APRIL 23, 2024

The AWS Well-Architected Framework provides best practices and guidelines for designing and operating reliable, secure, efficient, and cost-effective systems in the cloud. It calls the CreateDataSource and DeleteDataSource APIs. Minimally, you must specify the following properties: Name – Specify a name for the knowledge base.

APIs 127

APIs Enterprise Scripts Accountability 127

AWS Machine Learning

JANUARY 8, 2024

The first allows you to run a Python script from any server or instance including a Jupyter notebook; this is the quickest way to get started. In the following sections, we first describe the script solution, followed by the AWS CDK construct solution. The following diagram illustrates the sequence of events within the script.

Scripts 110

Scripts Construction APIs Healthcare 110

AWS Machine Learning

AUGUST 1, 2024

Amazon Bedrock is a fully managed service that offers a choice of high-performing foundation models (FMs) from leading artificial intelligence (AI) companies like AI21 Labs, Anthropic, Cohere, Meta, Mistral AI, Stability AI, and Amazon through a single API. The scripts for fine-tuning and evaluation are available on the GitHub repository.

APIs 124

APIs Scripts Real estate Accountability 124

AWS Machine Learning

FEBRUARY 25, 2025

We recommend running similar scripts only on your own data sources after consulting with the team who manages them, or be sure to follow the terms of service for the sources that youre trying to fetch data from. As a security best practice, storing the client application data in Secrets Manager is recommended.

Enterprise 91

Enterprise Engineering APIs Accountability 91

AWS Machine Learning

NOVEMBER 14, 2022

We provide a step-by-step guide to deploy your SageMaker trained model to Graviton-based instances, cover best practices when working with Graviton, discuss the price-performance benefits, and demo how to deploy a TensorFlow model on a SageMaker Graviton instance. The inference script URI is needed in the INFERENCE_SCRIPT_S3_LOCATION.

Scripts 96

Scripts APIs Best practices Engineering 96

AWS Machine Learning

MAY 30, 2024

AWS Prototyping successfully delivered a scalable prototype, which solved CBRE’s business problem with a high accuracy rate (over 95%) and supported reuse of embeddings for similar NLQs, and an API gateway for integration into CBRE’s dashboards. The following diagram illustrates the web interface and API management layer.

Real estate 129

Real estate APIs Metrics Construction 129

AWS Machine Learning

SEPTEMBER 8, 2023

Amazon Rekognition makes it easy to add image analysis capability to your applications without any machine learning (ML) expertise and comes with various APIs to fulfil use cases such as object detection, content moderation, face detection and analysis, and text and celebrity recognition, which we use in this example.

APIs 122

APIs Scripts Entertainment Personalization 122

AWS Machine Learning

JULY 2, 2024

Image 2: Hugging Face NLP model inference performance improvement with torch.compile on AWS Graviton3-based c7g instance using Hugging Face example scripts. This section shows how to run inference in eager and torch.compile modes using torch Python wheels and benchmarking scripts from Hugging Face and TorchBench repos.

Benchmark 118

Benchmark Scripts Metrics APIs 118

AWS Machine Learning

APRIL 12, 2023

The code to invoke the pipeline script is available in the Studio notebooks, and we can change the hyperparameters and input/output when invoking the pipeline. This is quite different from our earlier method where we had all the parameters hard coded within the scripts and all the processes were inextricably linked.

Scripts 103

Scripts APIs Metrics Best practices 103

AWS Machine Learning

SEPTEMBER 11, 2023

IaC ensures that customer infrastructure and services are consistent, scalable, and reproducible while following best practices in the area of development operations (DevOps). This is required to communicate with the SageMaker API. SageMaker runtime: com.amazonaws.region.sagemaker.runtime.

Scripts 121

Scripts APIs Management Best practices 121

AWS Machine Learning

SEPTEMBER 14, 2023

Integrating security in our workflow Following the best practices of the Security Pillar of the Well-Architected Framework , Amazon Cognito is used for authentication. Amazon API Gateway hosts a REST API with various endpoints to handle user requests that are authenticated using Amazon Cognito.

APIs 120

APIs Healthcare Scripts Engineering 120

AWS Machine Learning

MARCH 23, 2023

Applications and services can call the deployed endpoint directly or through a deployed serverless Amazon API Gateway architecture. To learn more about real-time endpoint architectural best practices, refer to Creating a machine learning-powered REST API with Amazon API Gateway mapping templates and Amazon SageMaker.

Scripts 118

Scripts APIs Government Analytics 118

AWS Machine Learning

JULY 24, 2023

If the model changes on the server side, the client has to know and change its API call to the new endpoint accordingly. In this post, we share best practices to deploy deep learning models with FastAPI on AWS Inferentia NeuronCores. script in the fastapi and trace-model folders use this to create Docker images.

Scripts 90

Scripts APIs Best practices Engineering 90

TechSee

APRIL 22, 2024

Furthermore, TechSee’s technology can be integrated anywhere through APIs or SDKs. To ensure that new technologies are embraced and utilized to their fullest, they must be integrated smoothly into the existing agent dashboards and scripts, requiring minimal shifts from established routines.

APIs 109

APIs Scripts Best practices Technology 109

TechSee

APRIL 22, 2024

Furthermore, TechSee’s technology can be integrated anywhere through APIs or SDKs. To ensure that new technologies are embraced and utilized to their fullest, they must be integrated smoothly into the existing agent dashboards and scripts, requiring minimal shifts from established routines.

APIs 109

APIs Scripts Best practices Technology 109

AWS Machine Learning

APRIL 10, 2023

In this post, we present a comprehensive guide on deploying and running inference using the Stable Diffusion inpainting model in two methods: through JumpStart’s user interface (UI) in Amazon SageMaker Studio , and programmatically through JumpStart APIs available in the SageMaker Python SDK.

Scripts 95

Scripts APIs Construction Finance 95

AWS Machine Learning

APRIL 29, 2024

This often means the method of using a third-party LLM API won’t do for security, control, and scale reasons. It provides an approachable, robust Python API for the full infrastructure stack of ML/AI, from data and compute to workflows and observability. The following figure illustrates this workflow.

APIs 124

APIs Engineering Scripts Management 124

AWS Machine Learning

OCTOBER 2, 2023

For more information about best practices, refer to the AWS re:Invent 2019 talk, Build accurate training datasets with Amazon SageMaker Ground Truth. For this we use AWS Step Functions , a serverless workflow service that provides us with API integrations to quickly orchestrate and visualize the steps in our workflow.

Scripts 119

Scripts Construction Engineering APIs 119

AWS Machine Learning

MAY 8, 2023

To use TensorRT as a backend for Triton Inference Server, you need to create a TensorRT engine from your trained model using the TensorRT API. Inference requests arrive at the server via either HTTP/REST or by the C API , and are then routed to the appropriate per-model scheduler. script from the following cell.

Engineering 99

Engineering APIs Best practices Scripts 99

AWS Machine Learning

JANUARY 6, 2023

Creates an API Gateway that adds an additional layer of security between the web app user interface and Lambda. Wait until the script provisions all the required resources and finishes running. Copy the API Gateway URL that the AWS CDK script prints out and save it. (We The S3 path to the movie node file.

Entertainment 99

Entertainment APIs Scripts Analytics 99

AWS Machine Learning

MAY 31, 2023

Triton with PyTorch backend The PyTorch backend is designed to run TorchScript models using the PyTorch C++ API. Alternatively, you can use ensemble models or business logic scripting. file in the workspace directory contains scripts to load and save a PyTorch model. client(service_name="sagemaker") runtime_sm_client = boto3.client("sagemaker-runtime")

APIs 109

APIs Scripts Engineering Accountability 109

AWS Machine Learning

SEPTEMBER 1, 2023

Solution overview To get responses streamed back from SageMaker, you can use our new InvokeEndpointWithResponseStream API. To take advantage of the new streaming API, you need to make sure the model container returns the streamed response as chunked encoded data. We use Streamlit for the sample demo application UI.

APIs 115

APIs Chatbots Engineering Best practices 115

AWS Machine Learning

JUNE 27, 2023

The ML components for data ingestion, preprocessing, and model training were available as disjointed Python scripts and notebooks, which required a lot of manual heavy lifting on the part of engineers. The initial solution also required the support of a technical third party, to release new models swiftly and efficiently.

Engineering 116

Engineering APIs Scripts Entertainment 116

AWS Machine Learning

JUNE 4, 2024

The workflow includes the following steps: The user runs the terraform apply The Terraform local-exec provisioner is used to run a Python script that downloads the public dataset DialogSum from the Hugging Face Hub. file you have been working in and add the terraform_data resource type, uses a local provisioner to invoke your Python script.

Scripts 131

Scripts Accountability Engineering Best practices 131

AWS Machine Learning

SEPTEMBER 18, 2023

Each stage in the ML workflow is broken into discrete steps, with its own script that takes input and output parameters. In the following code, the desired number of actors is passed in as an input argument to the script. Let’s look at sections of the scripts that perform this data preprocessing. get("OfflineStoreConfig").get("S3StorageConfig").get("ResolvedOutputS3Uri")

Scripts 126

Scripts APIs Government Construction 126

AWS Machine Learning

MAY 8, 2023

In this post, we address these limitations by implementing the access control outside of the MLflow server and offloading authentication and authorization tasks to Amazon API Gateway , where we implement fine-grained access control mechanisms at the resource level using Identity and Access Management (IAM). Adds an IAM authorizer.

APIs 88

APIs Government Accountability Scripts 88

AWS Machine Learning

MAY 30, 2023

With SageMaker Processing, you can bring your own custom processing scripts and choose to build a custom container or use a SageMaker managed container with common frameworks like scikit-learn, Lime, Spark and more. Alternatively, you can use the list_processing_jobs API. You can choose from two methods to do this.

Scripts 80

Scripts Best practices Metrics APIs 80

AWS Machine Learning

AUGUST 10, 2023

As recommended by AWS as a best practice , customers have used separate accounts to simplify policy management for users and isolate resources by workloads and account. You can deploy the management account resources by running the following command: /scripts/organization-deployment/deploy-management-account.sh aws/config.

Scripts 94

Scripts Accountability Metrics APIs 94

AWS Machine Learning

JULY 5, 2022

Lifecycle configurations (LCCs) are shell scripts to automate customization for your Studio environments, such as installing JupyterLab extensions, preloading datasets, and setting up source code repositories. LCC scripts are triggered by Studio lifecycle events, such as starting a new Studio notebook. Apply the script (see below).

Scripts 95

Scripts Engineering APIs Consulting 95