AWS Machine Learning

OCTOBER 29, 2024

Observability refers to the ability to understand the internal state and behavior of a system by analyzing its outputs, logs, and metrics. Evaluation, on the other hand, involves assessing the quality and relevance of the generated outputs, enabling continual improvement.

Best practices 111

Best practices Feedback Metrics APIs 111

AWS Machine Learning

MARCH 6, 2025

Current RAG pipelines frequently employ similarity-based metrics such as ROUGE , BLEU , and BERTScore to assess the quality of the generated responses, which is essential for refining and enhancing the models capabilities. More sophisticated metrics are needed to evaluate factual alignment and accuracy.

Metrics 86

Metrics Enterprise APIs Engineering 86

AWS Machine Learning

FEBRUARY 12, 2025

This approach allows organizations to assess their AI models effectiveness using pre-defined metrics, making sure that the technology aligns with their specific needs and objectives. The introduction of an LLM-as-a-judge framework represents a significant step forward in simplifying and streamlining the model evaluation process.

Metrics 94

Metrics Engineering APIs Benchmark 94

AWS Machine Learning

NOVEMBER 13, 2024

The Amazon Bedrock single API access, regardless of the models you choose, gives you the flexibility to use different FMs and upgrade to the latest model versions with minimal code changes. Amazon Titan FMs provide customers with a breadth of high-performing image, multimodal, and text model choices, through a fully managed API.

Engineering 115

Engineering Management APIs Healthcare 115

AWS Machine Learning

FEBRUARY 1, 2023

Amazon Rekognition has two sets of APIs that help you moderate images or videos to keep digital communities safe and engaged. Some customers have asked if they could use this approach to moderate videos by sampling image frames and sending them to the Amazon Rekognition image moderation API.

APIs 86

APIs Scripts Metrics Surveys 86

AWS Machine Learning

MARCH 25, 2025

Finally, we explore how to set up rolling updates in different scenarios. Automated safety guards Integrated Amazon CloudWatch alarms monitor metrics on an inference component. AlarmName This CloudWatch alarm is configured to monitor metrics on an InferenceComponent.

APIs 86

APIs Engineering Accountability Metrics 86

AWS Machine Learning

FEBRUARY 18, 2025

During these live events, F1 IT engineers must triage critical issues across its services, such as network degradation to one of its APIs. This impacts downstream services that consume data from the API, including products such as F1 TV, which offer live and on-demand coverage of every race as well as real-time telemetry.

APIs 70

APIs Engineering Metrics Big data 70

AWS Machine Learning

SEPTEMBER 20, 2024

Amazon Bedrock agents use LLMs to break down tasks, interact dynamically with users, run actions through API calls, and augment knowledge using Amazon Bedrock Knowledge Bases. In this post, we demonstrate how to use Amazon Bedrock Agents with a web search API to integrate dynamic web content in your generative AI application.

APIs 106

APIs Chatbots Construction Engineering 106

AWS Machine Learning

FEBRUARY 21, 2025

The solution uses the FMs tool use capabilities, accessed through the Amazon Bedrock Converse API. This enables the FMs to not just process text, but to actively engage with various external tools and APIs to perform complex document analysis tasks. For more details on how tool use works, refer to The complete tool use workflow.

APIs 86

APIs Healthcare Consulting Consulting 86

AWS Machine Learning

OCTOBER 9, 2024

Amazon Lookout for Metrics is a fully managed service that uses machine learning (ML) to detect anomalies in virtually any time-series business or operational metrics—such as revenue performance, purchase transactions, and customer acquisition and retention rates—with no ML experience required.

Metrics 91

Metrics APIs Engineering Accountability 91

AWS Machine Learning

NOVEMBER 20, 2024

Amazon Bedrock is a fully managed service that offers a choice of high-performing foundation models (FMs) from leading AI companies like AI21 Labs, Anthropic, Cohere, Meta, Mistral AI, Stability AI, and Amazon through a single API, along with a broad set of capabilities to build generative AI applications with security, privacy, and responsible AI.

Construction 116

Construction APIs Metrics Accountability 116

AWS Machine Learning

MARCH 27, 2025

We dive deep into this process on how to use XML tags to structure the prompt and guide Amazon Bedrock in generating a balanced label dataset with high accuracy. Where discrete outcomes with labeled data exist, standard ML methods such as precision, recall, or other classic ML metrics can be used. client = boto3.client("bedrock-runtime",

Education 92

Education Engineering APIs Enterprise 92

AWS Machine Learning

MARCH 14, 2025

Organizations building and deploying AI applications, particularly those using large language models (LLMs) with Retrieval Augmented Generation (RAG) systems, face a significant challenge: how to evaluate AI outputs effectively throughout the application lifecycle. Human evaluation, although thorough, is time-consuming and expensive at scale.

Metrics 82

Metrics Best practices Engineering APIs 82

AWS Machine Learning

JANUARY 28, 2025

As businesses increasingly use large language models (LLMs) for these critical tasks and processes, they face a fundamental challenge: how to maintain the quick, responsive performance users expect while delivering the high-quality outputs these sophisticated models promise. These metrics are shown in the following diagram.

Benchmark 72

Benchmark APIs Engineering Metrics 72

Calabrio

MARCH 4, 2025

Workforce Management 2025 Guide to the Omnichannel Contact Center: How to Drive Success with the Right Software, Strategy, and Solutions Share Calling, email, texting, instant messaging, social mediathe communication channels available to us today can seem almost endless. What Are the Benefits of Having an Omnichannel Contact Center?

Contact Center 52

Contact Center Contact center software Multichannel APIs 52

AWS Machine Learning

MARCH 3, 2025

In this post, we walk through how to discover, deploy, and use the Pixtral 12B model for a variety of real-world vision use cases. Performance metrics and benchmarks Pixtral 12B is trained to understand both natural images and documents, achieving 52.5% To begin using Pixtral 12B, choose Deploy.

Benchmark 89

Benchmark APIs Enterprise Construction 89

AWS Machine Learning

OCTOBER 24, 2024

In this first post, we focus on the basics of RAG architecture and how to optimize text-only RAG. The second post outlines how to work with multiple data formats such as structured data (tables, databases) and images. In part 2 of this post, we will discuss how to extend this capability to images and structured data.

Chatbots 89

Chatbots Metrics Scripts APIs 89

AWS Machine Learning

AUGUST 7, 2024

We then retrieve answers using standard RAG and a two-stage RAG, which involves a reranking API. Retrieve answers using the knowledge base retrieve API Evaluate the response using the RAGAS Retrieve answers again by running a two-stage RAG, using the knowledge base retrieve API and then applying reranking on the context.

APIs 133

APIs Chatbots Metrics Education 133

AWS Machine Learning

OCTOBER 5, 2022

In this post, we discuss the key elements needed to evaluate the performance aspect of a content moderation service in terms of various accuracy metrics, and a provide an example using Amazon Rekognition Content Moderation API’s. Understanding such distribution can help you define your actual metric goals. What to evaluate.

Metrics 96

Metrics APIs Advertising Enterprise 96

AWS Machine Learning

FEBRUARY 25, 2025

This post shows how to configure an Amazon Q Business custom connector and derive insights by creating a generative AI-powered conversation experience on AWS using Amazon Q Business while using access control lists (ACLs) to restrict access to documents based on user permissions. secrets_manager_client = boto3.client('secretsmanager')

Enterprise 93

Enterprise Engineering APIs Accountability 93

AWS Machine Learning

MARCH 3, 2025

They enable applications requiring very low latency or local data processing using familiar APIs and tool sets. This guide demonstrates how to deploy an open source FM from Hugging Face on Amazon Elastic Compute Cloud (Amazon EC2) instances across three locations: a commercial AWS Region and two AWS Local Zones.

APIs 72

APIs Benchmark Metrics Healthcare 72

AWS Machine Learning

MAY 7, 2024

In this post, we demonstrate how to use enhanced video search capabilities by enabling semantic retrieval of videos based on text queries. Amazon Transcribe The transcription for the entire video is generated using the StartTranscriptionJob API. The metadata generated for each video by the APIs is processed and stored with timestamps.

APIs 132

APIs Advertising Entertainment Metrics 132

AWS Machine Learning

NOVEMBER 14, 2024

Have you ever stumbled upon a breathtaking travel photo and instantly wondered where it was and how to get there? Each one of these millions of travelers need to plan where they’ll stay, what they’ll see, and how they’ll get from place to place. It will then return the place name with the highest similarity score.

APIs 110

APIs Engineering Personalization Transportation 110

AWS Machine Learning

JANUARY 25, 2023

This post shows you how to use an integrated solution with Amazon Lookout for Metrics to break these barriers by quickly and easily detecting anomalies in the key performance indicators (KPIs) of your interest. Lookout for Metrics automatically detects and diagnoses anomalies (outliers from the norm) in business and operational data.

Metrics 94

Metrics APIs Enterprise Analytics 94

AWS Machine Learning

NOVEMBER 15, 2024

Amazon Bedrock is a fully managed service that offers a choice of high-performing foundation models (FMs) from leading AI companies like AI21 Labs, Anthropic, Cohere, Meta, Mistral AI, Stability AI, and Amazon through a single API, along with a broad set of capabilities to build generative AI applications with security, privacy, and responsible AI.

Chatbots 114

Chatbots Engineering Enterprise Government 114

AWS Machine Learning

APRIL 24, 2024

Amazon Bedrock is a fully managed service that offers a choice of high-performing foundation models (FMs) from leading AI companies like AI21 Labs, Anthropic, Cohere, Meta, Stability AI, and Amazon using a single API, along with a broad set of capabilities you need to build generative AI applications with security, privacy, and responsible AI.

APIs 135

APIs Engineering Metrics Management 135

AWS Machine Learning

FEBRUARY 24, 2025

In this post, we walk through how to discover, deploy, and use Mistral-Small-24B-Instruct-2501. At the time of writing this post, you can use the InvokeModel API to invoke the model. It doesnt support Converse APIs or other Amazon Bedrock tooling. In this section, we go over how to discover the models in SageMaker Studio.

APIs 84

APIs Enterprise Benchmark Feedback 84

AWS Machine Learning

AUGUST 11, 2022

This post shows you how to use an integrated solution with Amazon Lookout for Metrics and Amazon Kinesis Data Firehose to break these barriers by quickly and easily ingesting streaming data, and subsequently detecting anomalies in the key performance indicators of your interest. You don’t need ML experience to use Lookout for Metrics.

Metrics 90

Metrics APIs Scripts Wireless 90

AWS Machine Learning

AUGUST 26, 2024

The GenASL web app invokes the backend services by sending the S3 object key in the payload to an API hosted on Amazon API Gateway. API Gateway instantiates an AWS Step Functions The state machine orchestrates the AI/ML services Amazon Transcribe and Amazon Bedrock and the NoSQL data store Amazon DynamoDB using AWS Lambda functions.

APIs 121

APIs Calibration Best practices Metrics 121

AWS Machine Learning

JANUARY 10, 2024

The solution uses AWS Lambda , Amazon API Gateway , Amazon EventBridge , and SageMaker to automate the workflow with human approval intervention in the middle. The approver approves the model by following the link in the email to an API Gateway endpoint. API Gateway invokes a Lambda function to initiate model updates.

APIs 127

APIs Construction Engineering Analytics 127

Hodusoft

JANUARY 11, 2022

2022 Checklist on How to Choose the Best Auto Dialer for Your Business. This metric is particularly important when outbound contact centers are also concerned with sales and conversions and not just customer service. These metrics can be monitored and tracked by administrators through the live dashboard. . Call Scheduling.

Outbound sales 119

Outbound sales Call flow Contact Center APIs 119

AWS Machine Learning

OCTOBER 2, 2024

In this post, we discuss how to address these challenges holistically. By using AWS services such as Amazon CloudWatch , AWS CloudTrail , and Amazon OpenSearch Service , enterprises can gain visibility into model metrics, usage patterns, and potential issues, enabling proactive management and optimization.

Government 124

Government APIs Enterprise Best practices 124

AWS Machine Learning

JUNE 5, 2024

In this post, we show how to use Amazon Comprehend Custom to train and host an ML model to classify if the input email is an phishing attempt or not. For details on how to build a classification pipeline with Amazon Comprehend, see Build a classification pipeline with Amazon Comprehend custom classification.

APIs 114

APIs Metrics Accountability Enterprise 114

AWS Machine Learning

JANUARY 24, 2025

Analyze results through metrics and evaluation. Under Output data , for S3 location , enter the S3 path for the bucket storing fine-tuning metrics. Model customization in Amazon Bedrock involves the following actions: Create training and validation datasets. Set up IAM permissions for data access. Configure a KMS key and VPC.

Best practices 113

Best practices Accountability industry standards Metrics 113

AWS Machine Learning

JULY 31, 2023

It also enables you to evaluate the models using advanced metrics as if you were a data scientist. In this post, we show how a business analyst can evaluate and understand a classification churn model created with SageMaker Canvas using the Advanced metrics tab.

Metrics 97

Metrics Engineering Accountability Telecommunications 97

AWS Machine Learning

FEBRUARY 7, 2025

Our commitment to innovation led us to a pivotal challenge: how to harness the power of machine learning (ML) to further enhance our competitive edge while balancing this technological advancement with strict data security requirements and the need to streamline access to our existing internal resources.

Entertainment 78

Entertainment APIs Technology Feedback 78

AWS Machine Learning

JULY 10, 2024

This process enhances task-specific model performance, allowing the model to handle custom use cases with task-specific performance metrics that meet or surpass more powerful models like Anthropic Claude 3 Sonnet or Anthropic Claude 3 Opus. Under Output data , for S3 location , enter the S3 path for the bucket storing fine-tuning metrics.

APIs 131

APIs Airlines Metrics Engineering 131