AWS Machine Learning

OCTOBER 29, 2024

Observability refers to the ability to understand the internal state and behavior of a system by analyzing its outputs, logs, and metrics. Security – The solution uses AWS services and adheres to AWS Cloud Security best practices so your data remains within your AWS account.

Best practices 112

Best practices Feedback Metrics APIs 112

AWS Machine Learning

DECEMBER 9, 2024

However, keeping track of numerous experiments, their parameters, metrics, and results can be difficult, especially when working on complex projects simultaneously. Prerequisites You need an AWS account with an AWS Identity and Access Management (IAM) role with permissions to manage resources created as part of the solution.

Metrics 104

Metrics APIs Engineering Accountability 104

AWS Machine Learning

FEBRUARY 12, 2025

This approach allows organizations to assess their AI models effectiveness using pre-defined metrics, making sure that the technology aligns with their specific needs and objectives. The introduction of an LLM-as-a-judge framework represents a significant step forward in simplifying and streamlining the model evaluation process.

Metrics 103

Metrics Engineering Benchmark APIs 103

AWS Machine Learning

MARCH 25, 2025

Automated safety guards Integrated Amazon CloudWatch alarms monitor metrics on an inference component. AlarmName This CloudWatch alarm is configured to monitor metrics on an InferenceComponent. For more information about the SageMaker AI API, refer to the SageMaker AI API Reference.

APIs 109

APIs Engineering Accountability Metrics 109

AWS Machine Learning

MARCH 6, 2025

Current RAG pipelines frequently employ similarity-based metrics such as ROUGE , BLEU , and BERTScore to assess the quality of the generated responses, which is essential for refining and enhancing the models capabilities. More sophisticated metrics are needed to evaluate factual alignment and accuracy.

Metrics 117

Metrics Enterprise APIs Engineering 117

AWS Machine Learning

AUGUST 2, 2024

With GraphStorm, you can build solutions that directly take into account the structure of relationships or interactions between billions of entities, which are inherently embedded in most real-world data, including fraud detection scenarios, recommendations, community detection, and search/retrieval problems. Specifically, GraphStorm 0.3

APIs 117

APIs Benchmark Construction Enterprise 117

AWS Machine Learning

APRIL 10, 2025

Fine-tune an Amazon Nova model using the Amazon Bedrock API In this section, we provide detailed walkthroughs on fine-tuning and hosting customized Amazon Nova models using Amazon Bedrock. We first provided a detailed walkthrough on how to fine-tune, host, and conduct inference with customized Amazon Nova through the Amazon Bedrock API.

APIs 123

APIs Metrics Best practices Construction 123

AWS Machine Learning

NOVEMBER 20, 2024

Amazon Bedrock is a fully managed service that offers a choice of high-performing foundation models (FMs) from leading AI companies like AI21 Labs, Anthropic, Cohere, Meta, Mistral AI, Stability AI, and Amazon through a single API, along with a broad set of capabilities to build generative AI applications with security, privacy, and responsible AI.

Construction 117

Construction APIs Metrics Accountability 117

AWS Machine Learning

OCTOBER 9, 2024

Amazon Lookout for Metrics is a fully managed service that uses machine learning (ML) to detect anomalies in virtually any time-series business or operational metrics—such as revenue performance, purchase transactions, and customer acquisition and retention rates—with no ML experience required.

Metrics 93

Metrics APIs Engineering Accountability 93

AWS Machine Learning

OCTOBER 5, 2022

So much exposure naturally brings added risks like account takeover (ATO). Each year, bad actors compromise billions of accounts through stolen credentials, phishing, social engineering, and multiple forms of ATO. To put it into perspective: account takeover fraud increased by 90% to an estimated $11.4 Overview of solution.

Accountability 99

Accountability APIs Metrics Engineering 99

AWS Machine Learning

JANUARY 24, 2025

Analyze results through metrics and evaluation. The workflow steps are as follows: The user submits an Amazon Bedrock fine-tuning job within their AWS account, using IAM for resource access. The fine-tuning job initiates a training job in the model deployment accounts. Provide your account, bucket name, and VPC settings.

Best practices 112

Best practices Accountability industry standards Metrics 112

AWS Machine Learning

MARCH 27, 2025

Where discrete outcomes with labeled data exist, standard ML methods such as precision, recall, or other classic ML metrics can be used. These metrics provide high precision but are limited to specific use cases due to limited ground truth data. If the use case doesnt yield discrete outputs, task-specific metrics are more appropriate.

Education 110

Education Engineering APIs Enterprise 110

AWS Machine Learning

SEPTEMBER 20, 2024

Amazon Bedrock agents use LLMs to break down tasks, interact dynamically with users, run actions through API calls, and augment knowledge using Amazon Bedrock Knowledge Bases. In this post, we demonstrate how to use Amazon Bedrock Agents with a web search API to integrate dynamic web content in your generative AI application.

APIs 105

APIs Chatbots Construction Engineering 105

AWS Machine Learning

FEBRUARY 1, 2023

Amazon Rekognition has two sets of APIs that help you moderate images or videos to keep digital communities safe and engaged. Some customers have asked if they could use this approach to moderate videos by sampling image frames and sending them to the Amazon Rekognition image moderation API.

APIs 95

APIs Scripts Metrics Surveys 95

AWS Machine Learning

FEBRUARY 21, 2025

The solution uses the FMs tool use capabilities, accessed through the Amazon Bedrock Converse API. This enables the FMs to not just process text, but to actively engage with various external tools and APIs to perform complex document analysis tasks. For more details on how tool use works, refer to The complete tool use workflow.

APIs 92

APIs Healthcare Consulting Consulting 92

AWS Machine Learning

MARCH 3, 2025

Performance metrics and benchmarks Pixtral 12B is trained to understand both natural images and documents, achieving 52.5% You can review the Mistral published benchmarks Prerequisites To try out Pixtral 12B in Amazon Bedrock Marketplace, you will need the following prerequisites: An AWS account that will contain all your AWS resources.

Benchmark 109

Benchmark APIs Construction Enterprise 109

AWS Machine Learning

MARCH 14, 2025

Although automated metrics are fast and cost-effective, they can only evaluate the correctness of an AI response, without capturing other evaluation dimensions or providing explanations of why an answer is problematic. Human evaluation, although thorough, is time-consuming and expensive at scale.

Metrics 103

Metrics Best practices Engineering APIs 103

AWS Machine Learning

JANUARY 29, 2025

Large organizations often have many business units with multiple lines of business (LOBs), with a central governing entity, and typically use AWS Organizations with an Amazon Web Services (AWS) multi-account strategy. LOBs have autonomy over their AI workflows, models, and data within their respective AWS accounts.

Enterprise 98

Enterprise Government Accountability Finance 98

AWS Machine Learning

FEBRUARY 25, 2025

Amazon Q Business only provides metric information that you can use to monitor your data source sync jobs. Prerequisites For this walkthrough, you should have the following prerequisites: An AWS account Access to the Alation service with the ability to create new policies and access tokens. secrets_manager_client = boto3.client('secretsmanager')

Enterprise 104

Enterprise Engineering APIs Accountability 104

AWS Machine Learning

APRIL 24, 2025

A seamless search journey not only enhances the overall user experience, but also directly impacts key business metrics such as conversion rates, average order value, and customer loyalty. Send the text, images, and metadata to Amazon Bedrock using its API to generate embeddings using the Amazon Titan Multimodal Embeddings G1 model.

Engineering 106

Engineering APIs Enterprise Accountability 106

AWS Machine Learning

NOVEMBER 19, 2024

Gain insights into training strategies, productivity metrics, and real-world use cases to empower your developers to harness the full potential of this game-changing technology. Learn how they created specialized agents for different tasks like account management, repos, pipeline management, and more to help their developers go faster.

APIs 106

APIs Enterprise Best practices Government 106

AWS Machine Learning

NOVEMBER 13, 2024

The Amazon Bedrock single API access, regardless of the models you choose, gives you the flexibility to use different FMs and upgrade to the latest model versions with minimal code changes. Amazon Titan FMs provide customers with a breadth of high-performing image, multimodal, and text model choices, through a fully managed API.

Engineering 115

Engineering Management APIs Healthcare 115

AWS Machine Learning

FEBRUARY 24, 2025

Performance metrics and benchmarks According to Mistral, the instruction-tuned version of the model achieves over 81% accuracy on Massive Multitask Language Understanding (MMLU) with 150 tokens per second latency, making it currently the most efficient model in its category. It doesnt support Converse APIs or other Amazon Bedrock tooling.

APIs 93

APIs Enterprise Benchmark Feedback 93

AWS Machine Learning

AUGUST 11, 2022

This post shows you how to use an integrated solution with Amazon Lookout for Metrics and Amazon Kinesis Data Firehose to break these barriers by quickly and easily ingesting streaming data, and subsequently detecting anomalies in the key performance indicators of your interest. You don’t need ML experience to use Lookout for Metrics.

Metrics 97

Metrics APIs Scripts Wireless 97

AWS Machine Learning

NOVEMBER 6, 2023

You can use the adapter for inference by passing the adapter identifier as an additional parameter to the Analyze Document Queries API request. Adapters can be created via the console or programmatically via the API. What is the account#? What is the account name/payer/drawer name? MICR line format). Who is the payee?

APIs 127

APIs Best practices Banking Accountability 127

AWS Machine Learning

NOVEMBER 16, 2022

When designing production CI/CD pipelines, AWS recommends leveraging multiple accounts to isolate resources, contain security threats and simplify billing-and data science pipelines are no different. Some things to note in the preceding architecture: Accounts follow a principle of least privilege to follow security best practices.

Accountability 98

Accountability Best practices Scripts Engineering 98

AWS Machine Learning

JULY 10, 2024

This process enhances task-specific model performance, allowing the model to handle custom use cases with task-specific performance metrics that meet or surpass more powerful models like Anthropic Claude 3 Sonnet or Anthropic Claude 3 Opus. Under Output data , for S3 location , enter the S3 path for the bucket storing fine-tuning metrics.

APIs 130

APIs Airlines Metrics Engineering 130

AWS Machine Learning

MAY 30, 2024

AWS Prototyping successfully delivered a scalable prototype, which solved CBRE’s business problem with a high accuracy rate (over 95%) and supported reuse of embeddings for similar NLQs, and an API gateway for integration into CBRE’s dashboards. The following diagram illustrates the web interface and API management layer.

Real estate 128

Real estate APIs Metrics Construction 128

AWS Machine Learning

AUGUST 7, 2024

We then retrieve answers using standard RAG and a two-stage RAG, which involves a reranking API. Retrieve answers using the knowledge base retrieve API Evaluate the response using the RAGAS Retrieve answers again by running a two-stage RAG, using the knowledge base retrieve API and then applying reranking on the context.

APIs 133

APIs Chatbots Metrics Education 133

AWS Machine Learning

FEBRUARY 7, 2025

The implementation uses Slacks event subscription API to process incoming messages and Slacks Web API to send responses. The incoming event from Slack is sent to an endpoint in API Gateway, and Slack expects a response in less than 3 seconds, otherwise the request fails. He has been helping customers at AWS for the past 4.5

Entertainment 85

Entertainment APIs Technology Feedback 85

AWS Machine Learning

MARCH 3, 2025

They enable applications requiring very low latency or local data processing using familiar APIs and tool sets. Prerequisites To run this demo, complete the following prerequisites: Create an AWS account , if you dont already have one. Enable the Local Zones in Los Angeles and Honolulu in the parent Region US West (Oregon).

APIs 82

APIs Benchmark Metrics Healthcare 82

AWS Machine Learning

JULY 18, 2024

Small business proprietors tend to prioritize the operational aspects of their enterprises over administrative tasks, such as maintaining financial records and accounting. While hiring a professional accountant can provide valuable guidance and expertise, it can be cost-prohibitive for many small businesses.

Chatbots 133

Chatbots APIs Accountability Finance 133

AWS Machine Learning

OCTOBER 2, 2024

It’s a fully managed service that offers a choice of high-performing foundation models (FMs) from leading AI companies like Anthropic, Cohere, Meta, Mistral AI, and Amazon through a single API, along with a broad set of capabilities to build generative AI applications with security, privacy, and responsible AI.

Government 128

Government APIs Enterprise Best practices 128

AWS Machine Learning

JANUARY 7, 2025

The translation playground could be adapted into a scalable serverless solution as represented by the following diagram using AWS Lambda , Amazon Simple Storage Service (Amazon S3), and Amazon API Gateway. The project also requires that the AWS account is bootstrapped to allow the deployment of the AWS CDK stack.

Engineering 112

Engineering Metrics industry standards Analytics 112

AWS Machine Learning

OCTOBER 29, 2024

A multi-account strategy is essential not only for improving governance but also for enhancing security and control over the resources that support your organization’s business. In this post, we dive into setting up observability in a multi-account environment with Amazon SageMaker.

Government 109

Government Metrics Accountability APIs 109

AWS Machine Learning

JULY 31, 2023

It also enables you to evaluate the models using advanced metrics as if you were a data scientist. In this post, we show how a business analyst can evaluate and understand a classification churn model created with SageMaker Canvas using the Advanced metrics tab. The F1 score provides a balanced evaluation of the model’s performance.

Metrics 98

Metrics Engineering Accountability Telecommunications 98

AWS Machine Learning

JUNE 5, 2024

Prerequisites Before diving into this use case, complete the following prerequisites: Set up an AWS account. You can train a custom classifier using either the Amazon Comprehend console or API. This confusion matrix provides metrics on how well the model performed in training. Test the model. Create an S3 bucket.

APIs 120

APIs Metrics Accountability Enterprise 120