AWS Machine Learning

OCTOBER 29, 2024

Observability refers to the ability to understand the internal state and behavior of a system by analyzing its outputs, logs, and metrics. Evaluation, on the other hand, involves assessing the quality and relevance of the generated outputs, enabling continual improvement.

Best practices 115

Best practices Feedback Metrics APIs 115

AWS Machine Learning

DECEMBER 9, 2024

However, keeping track of numerous experiments, their parameters, metrics, and results can be difficult, especially when working on complex projects simultaneously. Note that MLflow tracking starts from the mlflow.start_run() API. The mlflow.autolog() API can automatically log information such as metrics, parameters, and artifacts.

Metrics 105

Metrics APIs Engineering Accountability 105

AWS Machine Learning

FEBRUARY 18, 2025

During these live events, F1 IT engineers must triage critical issues across its services, such as network degradation to one of its APIs. This impacts downstream services that consume data from the API, including products such as F1 TV, which offer live and on-demand coverage of every race as well as real-time telemetry.

APIs 72

APIs Engineering Metrics Big data 72

AWS Machine Learning

OCTOBER 9, 2024

Amazon Lookout for Metrics is a fully managed service that uses machine learning (ML) to detect anomalies in virtually any time-series business or operational metrics—such as revenue performance, purchase transactions, and customer acquisition and retention rates—with no ML experience required.

Metrics 96

Metrics APIs Engineering Accountability 96

AWS Machine Learning

FEBRUARY 21, 2025

The solution uses the FMs tool use capabilities, accessed through the Amazon Bedrock Converse API. This enables the FMs to not just process text, but to actively engage with various external tools and APIs to perform complex document analysis tasks. For more details on how tool use works, refer to The complete tool use workflow.

APIs 95

APIs Healthcare Consulting Consulting 95

AWS Machine Learning

FEBRUARY 1, 2023

Amazon Rekognition has two sets of APIs that help you moderate images or videos to keep digital communities safe and engaged. Some customers have asked if they could use this approach to moderate videos by sampling image frames and sending them to the Amazon Rekognition image moderation API.

APIs 95

APIs Scripts Metrics Surveys 95

AWS Machine Learning

MARCH 6, 2025

Current RAG pipelines frequently employ similarity-based metrics such as ROUGE , BLEU , and BERTScore to assess the quality of the generated responses, which is essential for refining and enhancing the models capabilities. More sophisticated metrics are needed to evaluate factual alignment and accuracy.

Metrics 79

Metrics Enterprise APIs Engineering 79

AWS Machine Learning

JANUARY 28, 2025

To effectively optimize AI applications for responsiveness, we need to understand the key metrics that define latency and how they impact user experience. These metrics differ between streaming and nonstreaming modes and understanding them is crucial for building responsive AI applications.

Benchmark 78

Benchmark APIs Engineering Metrics 78

AWS Machine Learning

SEPTEMBER 20, 2024

Amazon Bedrock agents use LLMs to break down tasks, interact dynamically with users, run actions through API calls, and augment knowledge using Amazon Bedrock Knowledge Bases. In this post, we demonstrate how to use Amazon Bedrock Agents with a web search API to integrate dynamic web content in your generative AI application.

APIs 106

APIs Chatbots Construction Engineering 106

Cisco - Contact Center

MAY 24, 2024

Meraki APIs allow businesses to automate repetitive and time-consuming tasks, and configure and deploy networks quickly at a scale. Developrs can leverage API operations to retrieve performance metrics, monitor network health, analyze traffic data and create custom reports to gain insights into the network usage.

APIs 68

APIs Metrics 68

AWS Machine Learning

NOVEMBER 20, 2024

Amazon Bedrock is a fully managed service that offers a choice of high-performing foundation models (FMs) from leading AI companies like AI21 Labs, Anthropic, Cohere, Meta, Mistral AI, Stability AI, and Amazon through a single API, along with a broad set of capabilities to build generative AI applications with security, privacy, and responsible AI.

Construction 119

Construction APIs Metrics Accountability 119

AWS Machine Learning

MARCH 3, 2025

Performance metrics and benchmarks Pixtral 12B is trained to understand both natural images and documents, achieving 52.5% You can find detailed usage instructions, including sample API calls and code snippets for integration. To begin using Pixtral 12B, choose Deploy.

Benchmark 101

Benchmark APIs Enterprise Construction 101

AWS Machine Learning

FEBRUARY 24, 2025

Performance metrics and benchmarks According to Mistral, the instruction-tuned version of the model achieves over 81% accuracy on Massive Multitask Language Understanding (MMLU) with 150 tokens per second latency, making it currently the most efficient model in its category. It doesnt support Converse APIs or other Amazon Bedrock tooling.

APIs 96

APIs Enterprise Benchmark Feedback 96

AWS Machine Learning

OCTOBER 5, 2022

In this post, we discuss the key elements needed to evaluate the performance aspect of a content moderation service in terms of various accuracy metrics, and a provide an example using Amazon Rekognition Content Moderation API’s. Understanding such distribution can help you define your actual metric goals. What to evaluate.

Metrics 101

Metrics APIs Advertising Enterprise 101

AWS Machine Learning

NOVEMBER 13, 2024

The Amazon Bedrock single API access, regardless of the models you choose, gives you the flexibility to use different FMs and upgrade to the latest model versions with minimal code changes. Amazon Titan FMs provide customers with a breadth of high-performing image, multimodal, and text model choices, through a fully managed API.

Engineering 117

Engineering Management APIs Healthcare 117

AWS Machine Learning

JANUARY 25, 2023

This post shows you how to use an integrated solution with Amazon Lookout for Metrics to break these barriers by quickly and easily detecting anomalies in the key performance indicators (KPIs) of your interest. Lookout for Metrics automatically detects and diagnoses anomalies (outliers from the norm) in business and operational data.

Metrics 98

Metrics APIs Enterprise Analytics 98

AWS Machine Learning

MARCH 27, 2024

This requires carefully combining applications and metrics to provide complete awareness, accuracy, and control. The zAdviser uses Amazon Bedrock to provide summarization, analysis, and recommendations for improvement based on the DORA metrics data. It’s also vital to avoid focusing on irrelevant metrics or excessively tracking data.

Enterprise 123

Enterprise APIs Metrics Engineering 123

AWS Machine Learning

NOVEMBER 14, 2024

The user’s request is sent to AWS API Gateway , which triggers a Lambda function to interact with Amazon Bedrock using Anthropic’s Claude Instant V1 FM to process the user’s request and generate a natural language response of the place location. It will then return the place name with the highest similarity score.

APIs 113

APIs Engineering Personalization Transportation 113

AWS Machine Learning

NOVEMBER 15, 2024

Amazon Bedrock is a fully managed service that offers a choice of high-performing foundation models (FMs) from leading AI companies like AI21 Labs, Anthropic, Cohere, Meta, Mistral AI, Stability AI, and Amazon through a single API, along with a broad set of capabilities to build generative AI applications with security, privacy, and responsible AI.

Chatbots 115

Chatbots Engineering Enterprise Government 115

AWS Machine Learning

AUGUST 7, 2024

We then retrieve answers using standard RAG and a two-stage RAG, which involves a reranking API. Retrieve answers using the knowledge base retrieve API Evaluate the response using the RAGAS Retrieve answers again by running a two-stage RAG, using the knowledge base retrieve API and then applying reranking on the context.

APIs 136

APIs Chatbots Metrics Education 136

AWS Machine Learning

OCTOBER 2, 2024

It’s a fully managed service that offers a choice of high-performing foundation models (FMs) from leading AI companies like Anthropic, Cohere, Meta, Mistral AI, and Amazon through a single API, along with a broad set of capabilities to build generative AI applications with security, privacy, and responsible AI.

Government 131

Government APIs Enterprise Best practices 131

AWS Machine Learning

MAY 7, 2024

Amazon Transcribe The transcription for the entire video is generated using the StartTranscriptionJob API. The solution runs Amazon Rekognition APIs for label detection , text detection, celebrity detection , and face detection on videos. The metadata generated for each video by the APIs is processed and stored with timestamps.

APIs 138

APIs Advertising Entertainment Metrics 138

AWS Machine Learning

FEBRUARY 25, 2025

Amazon Q Business only provides metric information that you can use to monitor your data source sync jobs. With the connector ready, move over to the SageMaker Studio notebook and perform data synchronization operations by invoking Amazon Q Business APIs. secrets_manager_client = boto3.client('secretsmanager')

Enterprise 106

Enterprise Engineering APIs Accountability 106

AWS Machine Learning

APRIL 24, 2024

Amazon Bedrock is a fully managed service that offers a choice of high-performing foundation models (FMs) from leading AI companies like AI21 Labs, Anthropic, Cohere, Meta, Stability AI, and Amazon using a single API, along with a broad set of capabilities you need to build generative AI applications with security, privacy, and responsible AI.

APIs 138

APIs Engineering Metrics Management 138

AWS Machine Learning

FEBRUARY 7, 2025

The implementation uses Slacks event subscription API to process incoming messages and Slacks Web API to send responses. The incoming event from Slack is sent to an endpoint in API Gateway, and Slack expects a response in less than 3 seconds, otherwise the request fails.

Entertainment 87

Entertainment APIs Technology Feedback 87

AWS Machine Learning

MAY 30, 2024

AWS Prototyping successfully delivered a scalable prototype, which solved CBRE’s business problem with a high accuracy rate (over 95%) and supported reuse of embeddings for similar NLQs, and an API gateway for integration into CBRE’s dashboards. The following diagram illustrates the web interface and API management layer.

Real estate 131

Real estate APIs Metrics Construction 131

AWS Machine Learning

JANUARY 10, 2024

The solution uses AWS Lambda , Amazon API Gateway , Amazon EventBridge , and SageMaker to automate the workflow with human approval intervention in the middle. The approver approves the model by following the link in the email to an API Gateway endpoint. API Gateway invokes a Lambda function to initiate model updates.

APIs 131

APIs Construction Engineering Analytics 131

AWS Machine Learning

AUGUST 26, 2024

The GenASL web app invokes the backend services by sending the S3 object key in the payload to an API hosted on Amazon API Gateway. API Gateway instantiates an AWS Step Functions The state machine orchestrates the AI/ML services Amazon Transcribe and Amazon Bedrock and the NoSQL data store Amazon DynamoDB using AWS Lambda functions.

APIs 123

APIs Calibration Best practices Metrics 123

CSM Magazine

FEBRUARY 19, 2025

Challenge 2: Integration with Wearables and Third-Party APIs Many people use smartwatches and heart rate monitors to measure sleep, stress, and physical activity, which may affect mental health. Third-party APIs may link apps to healthcare and meditation services. However, integrating these diverse sources is not straightforward.

APIs 52

APIs Healthcare Gamification Consulting 52

AWS Machine Learning

OCTOBER 24, 2024

The retrieve_and_generate API does both the retrieval and a call to an FM (Amazon Titan or Anthropic’s Claude family of models on Amazon Bedrock ), for a fully managed solution. Mean Reciprocal Rank (MRR) – This metric considers the ranking of the retrieved documents. More advanced models such as Anthropic’s Claude Sonnet 3.5

Chatbots 94

Chatbots Metrics Scripts APIs 94

AWS Machine Learning

JULY 10, 2024

This process enhances task-specific model performance, allowing the model to handle custom use cases with task-specific performance metrics that meet or surpass more powerful models like Anthropic Claude 3 Sonnet or Anthropic Claude 3 Opus. Under Output data , for S3 location , enter the S3 path for the bucket storing fine-tuning metrics.

APIs 132

APIs Airlines Metrics Engineering 132

AWS Machine Learning

DECEMBER 5, 2022

Then we dive into the two key metrics used to evaluate a biometric system’s accuracy: the false match rate (also known as false acceptance rate) and false non-match rate (also known as false rejection rate). We use FMR and FNMR as our two key metrics to evaluate facial biometric systems. False non-match rate. Conclusion.

Metrics 97

Metrics Best practices Banking APIs 97

AWS Machine Learning

JANUARY 7, 2025

The translation playground could be adapted into a scalable serverless solution as represented by the following diagram using AWS Lambda , Amazon Simple Storage Service (Amazon S3), and Amazon API Gateway. Also note the completion metrics on the left pane, displaying latency, input/output tokens, and quality scores.

Engineering 115

Engineering Metrics industry standards Finance 115

AWS Machine Learning

NOVEMBER 6, 2023

You can use the adapter for inference by passing the adapter identifier as an additional parameter to the Analyze Document Queries API request. Adapters can be created via the console or programmatically via the API. You can analyze these metrics either collectively or on a per-document basis. MICR line format).

APIs 130

APIs Best practices Banking Accountability 130

AWS Machine Learning

JUNE 5, 2024

You train the model using semi-structured documents, which includes the following document types such as digital and scanned PDF documents and Word documents; Images sunch as JPG files, PNG files, and single-page TIFF files and Amazon Textract API output JSON files. The model detects the input email text is a non-phishing email.

APIs 123

APIs Metrics Accountability Enterprise 123

AWS Machine Learning

JULY 31, 2023

It also enables you to evaluate the models using advanced metrics as if you were a data scientist. In this post, we show how a business analyst can evaluate and understand a classification churn model created with SageMaker Canvas using the Advanced metrics tab. The F1 score provides a balanced evaluation of the model’s performance.

Metrics 98

Metrics Engineering Accountability Telecommunications 98

AWS Machine Learning

MARCH 3, 2025

They enable applications requiring very low latency or local data processing using familiar APIs and tool sets. This tool launches multiple requests from the test users client to the FM endpoint and measures various performance metrics, including TTFT. Each request contains a random prompt with a mean token count of 250 tokens.

APIs 74

APIs Benchmark Metrics Healthcare 74