AWS Machine Learning

MARCH 27, 2025

Lets say the task at hand is to predict the root cause categories (Customer Education, Feature Request, Software Defect, Documentation Improvement, Security Awareness, and Billing Inquiry) for customer support cases. These metrics provide high precision but are limited to specific use cases due to limited ground truth data.

Education 91

Education Engineering APIs Enterprise 91

AWS Machine Learning

NOVEMBER 15, 2024

Principal wanted to use existing internal FAQs, documentation, and unstructured data and build an intelligent chatbot that could provide quick access to the right information for different roles. Model monitoring of key NLP metrics was incorporated and controls were implemented to prevent unsafe, unethical, or off-topic responses.

Chatbots 113

Chatbots Engineering Enterprise Government 113

AWS Machine Learning

APRIL 11, 2024

Organizations across industries want to categorize and extract insights from high volumes of documents of different formats. Manually processing these documents to classify and extract information remains expensive, error prone, and difficult to scale. Categorizing documents is an important first step in IDP systems.

APIs 128

APIs Banking Enterprise Engineering 128

AWS Machine Learning

MARCH 6, 2025

Current RAG pipelines frequently employ similarity-based metrics such as ROUGE , BLEU , and BERTScore to assess the quality of the generated responses, which is essential for refining and enhancing the models capabilities. More sophisticated metrics are needed to evaluate factual alignment and accuracy.

Metrics 84

Metrics Enterprise APIs Engineering 84

AWS Machine Learning

NOVEMBER 15, 2024

Amazon Bedrock is a fully managed service that offers a choice of high-performing foundation models (FMs) from leading AI companies like AI21 Labs, Anthropic, Cohere, Meta, Mistral AI, Stability AI, and Amazon through a single API, along with a broad set of capabilities to build generative AI applications with security, privacy, and responsible AI.

APIs 109

APIs Government Best practices Metrics 109

AWS Machine Learning

OCTOBER 29, 2024

All of this data is centralized and can be used to improve metrics in scenarios such as sales or call centers. For integration between services, we use API Gateway as an event trigger for our Lambda function, and DynamoDB as a highly scalable database to store our customer details.

Engineering 109

Engineering APIs Sales Enterprise 109

AWS Machine Learning

JANUARY 8, 2024

AWS customers in healthcare, financial services, the public sector, and other industries store billions of documents as images or PDFs in Amazon Simple Storage Service (Amazon S3). In this post, we focus on processing a large collection of documents into raw text files and storing them in Amazon S3.

Scripts 111

Scripts Construction APIs Healthcare 111

AWS Machine Learning

FEBRUARY 12, 2025

This approach allows organizations to assess their AI models effectiveness using pre-defined metrics, making sure that the technology aligns with their specific needs and objectives. The introduction of an LLM-as-a-judge framework represents a significant step forward in simplifying and streamlining the model evaluation process.

Metrics 93

Metrics Engineering APIs Benchmark 93

AWS Machine Learning

AUGUST 2, 2024

adds new APIs to customize GraphStorm pipelines: you now only need 12 lines of code to implement a custom node classification training loop. For more details about how to run graph multi-task learning with GraphStorm, refer to Multi-task Learning in GraphStorm in our documentation. introduces refactored graph ML pipeline APIs.

APIs 115

APIs Benchmark Construction Enterprise 115

AWS Machine Learning

MARCH 3, 2025

Designed for both image and document comprehension, Pixtral demonstrates advanced capabilities in vision-related tasks, including chart and figure interpretation, document question answering, multimodal reasoning, and instruction followingseveral of which are illustrated with examples later in this post.

Benchmark 88

Benchmark APIs Enterprise Construction 88

AWS Machine Learning

SEPTEMBER 8, 2023

For modern companies that deal with enormous volumes of documents such as contracts, invoices, resumes, and reports, efficiently processing and retrieving pertinent data is critical to maintaining a competitive edge. What if there was a way to process documents intelligently and make them searchable in with high accuracy?

Construction 129

Construction Metrics Best practices APIs 129

AWS Machine Learning

NOVEMBER 21, 2023

Amazon Textract is a machine learning (ML) service that automatically extracts text, handwriting, and data from any document or image. AnalyzeDocument Layout is a new feature that allows customers to automatically extract layout elements such as paragraphs, titles, subtitles, headers, footers, and more from documents.

APIs 112

APIs Construction Metrics Education 112

AWS Machine Learning

JANUARY 7, 2025

The solution offers two TM retrieval modes for users to choose from: vector and document search. When using the Amazon OpenSearch Service adapter (document search), translation unit groupings are parsed and stored into an index dedicated to the uploaded file. This is covered in detail later in the post.

Engineering 107

Engineering Metrics industry standards Finance 107

AWS Machine Learning

OCTOBER 24, 2024

Broadly speaking, a retriever is a module that takes a query as input and outputs relevant documents from one or more knowledge sources relevant to that query. Document ingestion In a RAG architecture, documents are often stored in a vector store. You must use the same embedding model at ingestion time and at search time.

Chatbots 88

Chatbots Metrics Scripts APIs 88

AWS Machine Learning

DECEMBER 6, 2023

Such data often lacks the specialized knowledge contained in internal documents available in modern businesses, which is typically needed to get accurate answers in domains such as pharmaceutical research, financial investigation, and customer support. For example, imagine that you are planning next year’s strategy of an investment company.

Analytics 125

Analytics APIs Engineering Construction 125

AWS Machine Learning

OCTOBER 9, 2024

Amazon Lookout for Metrics is a fully managed service that uses machine learning (ML) to detect anomalies in virtually any time-series business or operational metrics—such as revenue performance, purchase transactions, and customer acquisition and retention rates—with no ML experience required. To learn more, see the documentation.

Metrics 90

Metrics APIs Engineering Accountability 90

AWS Machine Learning

MARCH 25, 2025

Automated safety guards Integrated Amazon CloudWatch alarms monitor metrics on an inference component. AlarmName This CloudWatch alarm is configured to monitor metrics on an InferenceComponent. For more information about the SageMaker AI API, refer to the SageMaker AI API Reference.

APIs 85

APIs Engineering Accountability Metrics 85

AWS Machine Learning

APRIL 19, 2023

The ability to effectively handle and process enormous amounts of documents has become essential for enterprises in the modern world. Due to the continuous influx of information that all enterprises deal with, manually classifying documents is no longer a viable option.

APIs 98

APIs Metrics Enterprise Best practices 98

AWS Machine Learning

FEBRUARY 18, 2025

During these live events, F1 IT engineers must triage critical issues across its services, such as network degradation to one of its APIs. This impacts downstream services that consume data from the API, including products such as F1 TV, which offer live and on-demand coverage of every race as well as real-time telemetry.

APIs 69

APIs Engineering Metrics Big data 69

AWS Machine Learning

AUGUST 7, 2024

To find an answer, RAG takes an approach that uses vector search across the documents. Rather than scanning every single document to find the answer, with the RAG approach, you turn the texts (knowledge base) into embeddings and store these embeddings in the database. Generate questions from the document using an Amazon Bedrock LLM.

APIs 132

APIs Chatbots Metrics Education 132

AWS Machine Learning

FEBRUARY 25, 2025

This post shows how to configure an Amazon Q Business custom connector and derive insights by creating a generative AI-powered conversation experience on AWS using Amazon Q Business while using access control lists (ACLs) to restrict access to documents based on user permissions. Who are the data stewards for my proprietary database sources?

Enterprise 92

Enterprise Engineering APIs Accountability 92

AWS Machine Learning

NOVEMBER 13, 2024

The Amazon Bedrock single API access, regardless of the models you choose, gives you the flexibility to use different FMs and upgrade to the latest model versions with minimal code changes. Amazon Titan FMs provide customers with a breadth of high-performing image, multimodal, and text model choices, through a fully managed API.

Engineering 114

Engineering Management APIs Healthcare 114

AWS Machine Learning

NOVEMBER 3, 2022

In Part 1 of this series, we discussed intelligent document processing (IDP), and how IDP can accelerate claims processing use cases in the insurance industry. We discussed how we can use AWS AI services to accurately categorize claims documents along with supporting documents. Part 1: Classification and extraction of documents.

Analytics 94

Analytics APIs Healthcare Big data 94

AWS Machine Learning

OCTOBER 12, 2022

A massive amount of business documents are processed daily across industries. Many of these documents are paper-based, scanned into your system as images, or in an unstructured format like PDF. Each company may apply unique rules associated with its business background while processing these documents.

APIs 97

APIs Construction Metrics Engineering 97

AWS Machine Learning

JANUARY 28, 2025

To effectively optimize AI applications for responsiveness, we need to understand the key metrics that define latency and how they impact user experience. These metrics differ between streaming and nonstreaming modes and understanding them is crucial for building responsive AI applications.

Benchmark 71

Benchmark APIs Engineering Metrics 71

AWS Machine Learning

NOVEMBER 21, 2022

Organizations across industries such as retail, banking, finance, healthcare, manufacturing, and lending often have to deal with vast amounts of unstructured text documents coming from various sources, such as news, blogs, product reviews, customer support channels, and social media. Extract and analyze data from documents.

Scripts 91

Scripts Metrics APIs Healthcare 91

AWS Machine Learning

JUNE 5, 2024

You can use Amazon Comprehend to identify the language of the text; extract key phrases, places, people, brands, or events; understand sentiment about products or services; and identify the main topics from a library of documents. For minimum training requirements, see General quotas for document classification.

APIs 113

APIs Metrics Accountability Enterprise 113

AWS Machine Learning

MARCH 14, 2025

Although automated metrics are fast and cost-effective, they can only evaluate the correctness of an AI response, without capturing other evaluation dimensions or providing explanations of why an answer is problematic. Human evaluation, although thorough, is time-consuming and expensive at scale.

Metrics 81

Metrics Best practices Engineering APIs 81

AWS Machine Learning

DECEMBER 29, 2022

Documents are a primary tool for record keeping, communication, collaboration, and transactions across many industries, including financial, medical, legal, and real estate. The millions of mortgage applications and hundreds of millions of W2 tax forms processed each year are just a few examples of such documents.

APIs 114

APIs Banking Real estate Healthcare 114

AWS Machine Learning

MARCH 3, 2025

They enable applications requiring very low latency or local data processing using familiar APIs and tool sets. This tool launches multiple requests from the test users client to the FM endpoint and measures various performance metrics, including TTFT. Each request contains a random prompt with a mean token count of 250 tokens.

APIs 71

APIs Benchmark Metrics Healthcare 71

AWS Machine Learning

APRIL 24, 2024

Amazon Bedrock is a fully managed service that offers a choice of high-performing foundation models (FMs) from leading AI companies like AI21 Labs, Anthropic, Cohere, Meta, Stability AI, and Amazon using a single API, along with a broad set of capabilities you need to build generative AI applications with security, privacy, and responsible AI.

APIs 134

APIs Engineering Metrics Management 134

AWS Machine Learning

JULY 3, 2024

This new functionality offers industry-leading safety measures that filter harmful content and protect sensitive information in your documents, improving user experience and aligning with organizational standards. The query is then augmented to have the retrieved document chunks, prompt, and guardrails configuration.

APIs 130

APIs Banking Metrics Management 130

AWS Machine Learning

NOVEMBER 22, 2023

When a customer has a production-ready intelligent document processing (IDP) workload, we often receive requests for a Well-Architected review. To follow along with this post, you should be familiar with the previous posts in this series ( Part 1 and Part 2 ) and the guidelines in Guidance for Intelligent Document Processing on AWS.

APIs 116

APIs Metrics Benchmark Enterprise 116

AWS Machine Learning

JANUARY 29, 2025

Intelligent document processing , translation and summarization, flexible and insightful responses for customer support agents, personalized marketing content, and image and code generation are a few use cases using generative AI that organizations are rolling out in production. There are two types of inference profiles.

Enterprise 87

Enterprise Government Accountability Finance 87

AWS Machine Learning

MAY 9, 2024

It is designed to be deeply integrated into the FAST platform and use all of Verisk’s documentation, training materials, and collective expertise. Having that transparency helped Verisk identify areas of the system where their documents were lacking and needed some restructuring.

Engineering 129

Engineering Customer Support APIs Government 129

AWS Machine Learning

JUNE 20, 2024

In addition, RAG architecture can lead to potential issues like retrieval collapse , where the retrieval component learns to retrieve the same documents regardless of the input. This makes it difficult to apply standard evaluation metrics like BERTScore ( Zhang et al.

Metrics 134

Metrics Engineering Accountability Benchmark 134

AWS Machine Learning

MARCH 27, 2024

This requires carefully combining applications and metrics to provide complete awareness, accuracy, and control. The zAdviser uses Amazon Bedrock to provide summarization, analysis, and recommendations for improvement based on the DORA metrics data. It’s also vital to avoid focusing on irrelevant metrics or excessively tracking data.

Enterprise 115

Enterprise APIs Metrics Engineering 115