APIs, Benchmark and Examples - Customer Contact Central

Benchmarking Amazon Nova and GPT-4o models with FloTorch

AWS Machine Learning

MARCH 11, 2025

Using its enterprise software, FloTorch conducted an extensive comparison between Amazon Nova models and OpenAIs GPT-4o models with the Comprehensive Retrieval Augmented Generation (CRAG) benchmark dataset. The following table provides example questions with their domain and question type.

Benchmark

Benchmark APIs Enterprise Scripts

Pixtral-12B-2409 is now available on Amazon Bedrock Marketplace

AWS Machine Learning

MARCH 3, 2025

Overview of Pixtral 12B Pixtral 12B, Mistrals inaugural VLM, delivers robust performance across a range of benchmarks, surpassing other open models and rivaling larger counterparts, according to Mistrals evaluation. Performance metrics and benchmarks Pixtral 12B is trained to understand both natural images and documents, achieving 52.5%

Benchmark

Benchmark APIs Construction Enterprise

GraphStorm 0.3: Scalable, multi-task learning on graphs with user-friendly APIs

AWS Machine Learning

AUGUST 2, 2024

adds new APIs to customize GraphStorm pipelines: you now only need 12 lines of code to implement a custom node classification training loop. To help you get started with the new API, we have published two Jupyter notebook examples: one for node classification, and one for a link prediction task. Specifically, GraphStorm 0.3

APIs

APIs Benchmark Construction Enterprise

LLM-as-a-judge on Amazon Bedrock Model Evaluation

AWS Machine Learning

FEBRUARY 12, 2025

Amazon Bedrock , a fully managed service offering high-performing foundation models from leading AI companies through a single API, has recently introduced two significant evaluation capabilities: LLM-as-a-judge under Amazon Bedrock Model Evaluation and RAG evaluation for Amazon Bedrock Knowledge Bases. 0]}-{evaluator_model.split('.')[0]}-{datetime.now().strftime('%Y-%m-%d-%H-%M-%S')}"

Metrics

Metrics Engineering Benchmark APIs

Optimizing AI responsiveness: A practical guide to Amazon Bedrock latency-optimized inference

AWS Machine Learning

JANUARY 28, 2025

For example, a prompt that generates 100 tokens in one model might generate 150 tokens in another. Consider benchmarking your user experience to find the best latency for your use case, considering that most humans cant read faster than 225 words per minute and therefore extremely fast response can hinder user experience.

Benchmark

Benchmark APIs Engineering Metrics

Anthropic Claude 3.5 Sonnet ranks number 1 for business and finance in S&P AI Benchmarks by Kensho

AWS Machine Learning

JULY 9, 2024

Sonnet currently ranks at the top of S&P AI Benchmarks by Kensho , which assesses large language models (LLMs) for finance and business. For example, there could be leakage of benchmark datasets’ questions and answers into training data. Anthropic Claude 3.5 Kensho is the AI Innovation Hub for S&P Global.

Finance

Finance Benchmark industry standards Accountability

Benchmark and optimize endpoint deployment in Amazon SageMaker JumpStart

AWS Machine Learning

JANUARY 29, 2024

This post explores these relationships via a comprehensive benchmarking of LLMs available in Amazon SageMaker JumpStart, including Llama 2, Falcon, and Mistral variants. We provide theoretical principles on how accelerator specifications impact LLM benchmarking. Additionally, models are fully sharded on the supported instance.

Benchmark

Benchmark APIs Enterprise Accountability

Speed up your time series forecasting by up to 50 percent with Amazon SageMaker Canvas UI and AutoML APIs

AWS Machine Learning

SEPTEMBER 28, 2023

As an example, time-series forecasting allows retailers to predict future sales demand and plan for inventory levels, logistics, and marketing campaigns. In this post, we describe the enhancements to the forecasting capabilities of SageMaker Canvas and guide you on using its user interface (UI) and AutoML APIs for time-series forecasting.

APIs

APIs Construction Finance Enterprise

Mistral-Small-24B-Instruct-2501 is now available on SageMaker Jumpstart and Amazon Bedrock Marketplace

AWS Machine Learning

FEBRUARY 24, 2025

Performance metrics and benchmarks According to Mistral, the instruction-tuned version of the model achieves over 81% accuracy on Massive Multitask Language Understanding (MMLU) with 150 tokens per second latency, making it currently the most efficient model in its category. It doesnt support Converse APIs or other Amazon Bedrock tooling.

APIs

APIs Enterprise Benchmark Feedback

Generate training data and cost-effectively train categorical models with Amazon Bedrock

AWS Machine Learning

MARCH 27, 2025

We also showcase a real-world example for predicting the root cause category for support cases. For the use case of labeling the support root cause categories, its often harder to source examples for categories such as Software Defect, Feature Request, and Documentation Improvement for labeling than it is for Customer Education.

Education

Education Engineering APIs Enterprise

Evaluate RAG responses with Amazon Bedrock, LlamaIndex and RAGAS

AWS Machine Learning

MARCH 6, 2025

Here are some examples of these metrics: Retrieval component Context precision Evaluates whether all of the ground-truth relevant items present in the contexts are ranked higher or not. For example, metrics like Answer Relevancy and Faithfulness are typically scored on a scale from 0 to 1.

Metrics

Metrics Enterprise APIs Engineering

From RAG to fabric: Lessons learned from building real-world RAGs at GenAIIC – Part 2

AWS Machine Learning

NOVEMBER 15, 2024

For example, a technician could query the system about a specific machine part, receiving both textual maintenance history and annotated images showing wear patterns or common failure points, enhancing their ability to diagnose and resolve issues efficiently. has 92% accuracy on the HumanEval code benchmark.

APIs

APIs Engineering Chatbots Construction

Fine-tune LLMs with synthetic data for context-based Q&A using Amazon Bedrock

AWS Machine Learning

FEBRUARY 12, 2025

Amazon Bedrock is a fully managed service that makes FMs from leading AI startups and Amazon available through an API, so you can choose from a wide range of FMs to find the model that is best suited for your use case. Solution overview The solution comprises two main steps: Generate synthetic data using the Amazon Bedrock InvokeModel API.

APIs

APIs Management Benchmark Scripts

Speed up your AI inference workloads with new NVIDIA-powered capabilities in Amazon SageMaker

AWS Machine Learning

DECEMBER 2, 2024

This integration provides a powerful multilingual model that excels in reasoning benchmarks. Use the notebook examples to start your deployment using SageMaker to create the model, configure the endpoint, and deploy the model, and AWS will handle the orchestration of resources, networking, and scaling as needed. or Mixtral.

Enterprise

Enterprise Benchmark Technology APIs

Your guide to generative AI and ML at AWS re:Invent 2024

AWS Machine Learning

NOVEMBER 19, 2024

Hear from customer speakers with real-world examples of how they’ve used data to support a variety of use cases, including generative AI, to create unique customer experiences. Explore examples to estimate the severity and likelihood of potential events that could be harmful.

APIs

APIs Enterprise Best practices Government

Get started with Amazon Titan Text Embeddings V2: A new state-of-the-art embeddings model on Amazon Bedrock

AWS Machine Learning

MAY 2, 2024

We published a follow-up post on January 31, 2024, and provided code examples using AWS SDKs and LangChain, showcasing a Streamlit semantic search app. For example, in a recommendation system for a large ecommerce platform, a modest increase in recommendation accuracy could translate into significant additional revenue.

Benchmark

Benchmark Metrics Enterprise APIs

Reduce conversational AI response time through inference at the edge with AWS Local Zones

AWS Machine Learning

MARCH 3, 2025

They enable applications requiring very low latency or local data processing using familiar APIs and tool sets. Through comparative benchmarking tests, we illustrate how deploying FMs in Local Zones closer to end users can significantly reduce latencya critical factor for real-time applications such as conversational AI assistants.

APIs

APIs Benchmark Metrics Healthcare

AI21 Labs Jamba-Instruct model is now available in Amazon Bedrock

AWS Machine Learning

JUNE 25, 2024

Example use cases for Jamba-Instruct Jamba-Instruct’s long context length is particularly well-suited for complex Retrieval Augmented Generation (RAG) workloads, or potentially complex document analysis. For example: You are a curious and novel researcher, who is highly interested in getting all the relevant information on a specific topic.

Benchmark

Benchmark APIs Enterprise Technology

Transition your Amazon Forecast usage to Amazon SageMaker Canvas

AWS Machine Learning

JULY 29, 2024

You can also either use the SageMaker Canvas UI, which provides a visual interface for building and deploying models without needing to write any code or have any ML expertise, or use its automated machine learning (AutoML) APIs for programmatic interactions.

APIs

APIs Scripts Benchmark Metrics

Build a secure enterprise application with Generative AI and RAG using Amazon SageMaker JumpStart

AWS Machine Learning

SEPTEMBER 6, 2023

These SageMaker endpoints are consumed in the Amplify React application through Amazon API Gateway and AWS Lambda functions. To protect the application and APIs from inadvertent access, Amazon Cognito is integrated into Amplify React, API Gateway, and Lambda functions. For this example, we use train.cc_casebooks.jsonl.xz

Enterprise

Enterprise APIs Real estate Construction

Learn how Amazon Ads created a generative AI-powered image generation capability using Amazon SageMaker

AWS Machine Learning

MAY 15, 2024

For example, an advertiser might have static images of their product against a white background. An example from the image generation solution showing a hydro flask with various backgrounds. Acting as a model hub, JumpStart provided a large selection of foundation models and the team quickly ran their benchmarks on candidate models.

Advertising

Advertising APIs Engineering Benchmark

Accelerated PyTorch inference with torch.compile on AWS Graviton processors

AWS Machine Learning

JULY 2, 2024

You can see that for the 45 models we benchmarked, there is a 1.35x latency improvement (geomean for the 45 models). You can see that for the 33 models we benchmarked, there is around 2x performance improvement (geomean for the 33 models). We benchmarked 45 models using the scripts from the TorchBench repo.

Benchmark

Benchmark Scripts Metrics APIs

Package and deploy classical ML and LLMs easily with Amazon SageMaker, part 2: Interactive User Experiences in SageMaker Studio

AWS Machine Learning

NOVEMBER 30, 2023

SageMaker makes it easy to deploy models into production directly through API calls to the service. It’s a low-level API available for Java, C++, Go, JavaScript, Node.js, PHP, Ruby, and Python. It’s a low-level API available for Java, C++, Go, JavaScript, Node.js, PHP, Ruby, and Python.

Benchmark

Benchmark APIs Scripts Engineering

A review of purpose-built accelerators for financial services

AWS Machine Learning

SEPTEMBER 11, 2024

In this example figure, features are extracted from raw historical data, which are then are fed into a neural network (NN). In terms of resulting speedups, the approximate order is programming hardware, then programming against PBA APIs, then programming in an unmanaged language such as C++, then a managed language such as Python.

Benchmark

Benchmark Banking Analytics Big data

Optimize your machine learning deployments with auto scaling on Amazon SageMaker

AWS Machine Learning

FEBRUARY 8, 2023

Although you can integrate the model directly into an application, the approach that works well for production-grade applications is to deploy the model behind an endpoint and then invoke the endpoint via a RESTful API call to obtain the inference. However, you can use any other benchmarking tool. This value range is 0–100%.

Benchmark

Benchmark Metrics APIs Engineering

Build a multilingual automatic translation pipeline with Amazon Translate Active Custom Translation

AWS Machine Learning

JUNE 15, 2023

It features interactive Jupyter notebooks with self-contained code in PyTorch, JAX, TensorFlow, and MXNet, as well as real-world examples, exposition figures, and math. ACT allows you to customize translation output on the fly by providing tailored translation examples in the form of parallel data.

APIs

APIs Benchmark Best practices Engineering

Secure AccountantAI Chatbot: Lili’s journey with Amazon Bedrock

AWS Machine Learning

JULY 18, 2024

The ingestion workflow transforms these curated questions into vector embeddings using Amazon Titan Text Embeddings model API. Example prompt template Provides context about the agent's role as Lili's AI assistant for financial questions and outlines the general guidelines applied to all queries. Provides details on Lili platform.

Chatbots

Chatbots APIs Accountability Finance

Evaluation of generative AI techniques for clinical report summarization

AWS Machine Learning

MAY 13, 2024

This is a fully managed service that offers a choice of high-performing foundation models (FMs) from leading artificial intelligence (AI) companies like AI21 Labs, Anthropic, Cohere, Meta, Stability AI, and Amazon through a single API. A prompt has three main types of content: input, context, and examples.

Healthcare

Healthcare Engineering APIs Benchmark

Enhance performance of generative language models with self-consistency prompting on Amazon Bedrock

AWS Machine Learning

MARCH 19, 2024

For example, chain-of-thought (CoT) is known to improve a model’s capacity for complex multi-step problems. With the batch inference API, you can use Amazon Bedrock to run inference with foundation models in batches and get responses more efficiently. The service is in preview as of this writing and only available through the API.

APIs

APIs Benchmark SaaS Engineering

Optimize pet profiles for Purina’s Petfinder application using Amazon Rekognition Custom Labels and AWS Step Functions

AWS Machine Learning

OCTOBER 18, 2023

The solution uses the following services: Amazon API Gateway is a fully managed service that makes it easy for developers to publish, maintain, monitor, and secure APIs at any scale. Purina’s solution is deployed as an API Gateway HTTP endpoint, which routes the requests to obtain pet attributes.

APIs

APIs Metrics Consulting Consulting

Image classification model selection using Amazon SageMaker JumpStart

AWS Machine Learning

FEBRUARY 6, 2023

The former question addresses model selection across model architectures, while the latter question concerns benchmarking trained models against a test dataset. This post provides details on how to implement large-scale Amazon SageMaker benchmarking and model selection tasks. swin-large-patch4-window7-224 195.4M efficientnet-b5 29.0M

APIs

APIs Scripts Metrics Benchmark

Cepsa Química improves the efficiency and accuracy of product stewardship using Amazon Bedrock

AWS Machine Learning

AUGUST 2, 2024

Example questions might be “What are the restrictions for CMR substances?”, “How long do I need to keep the documents related to a toluene sale?”, The following screenshot shows an example of the conversational interface. or “What is the reach characterization ratio and how do I calculate it?”

APIs

APIs Analytics Chatbots Engineering

A progress update on our commitment to safe, responsible generative AI

AWS Machine Learning

JULY 10, 2024

Red-teaming engages human testers to probe an AI system for flaws in an adversarial style, and complements our other testing techniques, which include automated benchmarking against publicly available and proprietary datasets, human evaluation of completions against proprietary datasets, and more.

Government

Government Education Best practices Benchmark

How to use public reviews to improve customer experience

Lumoa

DECEMBER 8, 2022

How to add your competitors’ public reviews to the mix, so you can get a reliable benchmark and win in the market. There are many ways to use the integration function, and this is just one example. Lumoa has very clear open APIs and based on your goals, integrations can serve various purposes.

Customer Experience

Customer Experience Benchmark APIs Marketing

How Forethought saves over 66% in costs for generative AI models using Amazon SageMaker

AWS Machine Learning

JUNE 13, 2023

For example, in order to share expensive GPU resources between multiple models, we were responsible for allocating rigid memory fractions to models that were specified during deployment. In addition, deployments are now as simple as calling Boto3 SageMaker APIs and attaching the proper auto scaling policies.

APIs

APIs Benchmark Engineering Management

Exciting new developments from Spearline

Spearline

FEBRUARY 3, 2020

With such a rise in popularity of mobile usage around the world, we are delighted to announce that from February 2020, our customers will be able to test the sending of an SMS message to a destination specified by them, via the Spearline API. Access real-time reporting and analytics via Spearline API polling.

Telecommunications

Telecommunications APIs Benchmark Enterprise

Run PyTorch Lightning and native PyTorch DDP on Amazon SageMaker Training, featuring Amazon Search

AWS Machine Learning

AUGUST 18, 2022

The team’s early benchmarking results show 7.3 The baseline model used in these benchmarking is a multi-layer perceptron neural network with seven dense fully connected layers and over 200 parameters. The following table summarizes the benchmarking result on ml.p3.16xlarge SageMaker training instances. Number of Instances.

Scripts

Scripts APIs Benchmark Engineering

Improved ML model deployment using Amazon SageMaker Inference Recommender

AWS Machine Learning

APRIL 20, 2023

The majority of data is non-fraudulent (284,315 samples), with only 492 samples corresponding to fraudulent examples. An advanced job is a custom load test job that allows you to perform extensive benchmarks based on your ML application SLA requirements, such as latency, concurrency, and traffic pattern. Running Advanced job.

APIs

APIs Metrics Benchmark Engineering

The executive’s guide to generative AI for sustainability

AWS Machine Learning

APRIL 22, 2024

It provides examples of use cases and best practices for using generative AI’s potential to accelerate sustainability and ESG initiatives, as well as insights into the main operational challenges of generative AI for sustainability. Figure 1 illustrates selected examples of use cases of generative AI for sustainability across the value chain.

Best practices

Best practices Benchmark Transportation Engineering

Building an efficient MLOps platform with OSS tools on Amazon ECS with AWS Fargate

AWS Machine Learning

SEPTEMBER 18, 2024

Together, these AI-driven tools and technologies aren’t just reshaping how brands perform marketing tasks; they’re setting new benchmarks for what’s possible in customer engagement. From our experience, artifact server has some limitations, such as limits on artifact size (because of sending it using REST API).

APIs

APIs Engineering Analytics Marketing

Build a RAG-based QnA application using Llama3 models from SageMaker JumpStart

AWS Machine Learning

SEPTEMBER 12, 2024

On Hugging Face, the Massive Text Embedding Benchmark (MTEB) is provided as a leaderboard for diverse text embedding tasks. It currently provides 129 benchmarking datasets across 8 different tasks on 113 languages. medium instance to demonstrate deploying the model as an API endpoint using an SDK through SageMaker JumpStart.

APIs

APIs Benchmark Enterprise Construction

Snowflake Arctic models are now available in Amazon SageMaker JumpStart

AWS Machine Learning

AUGUST 22, 2024

Snowflake Arctic is a family of enterprise-grade large language models (LLMs) built by Snowflake to cater to the needs of enterprise users, exhibiting exceptional capabilities (as shown in the following benchmarks ) in SQL querying, coding, and accurately following instructions. To learn more, refer to API documentation.

Enterprise

Enterprise APIs Benchmark Scripts

Train gigantic models with near-linear scaling using sharded data parallelism on Amazon SageMaker

AWS Machine Learning

OCTOBER 31, 2022

For example, 8 V100 GPUs (32GB each) are sufficient to hold the model states replica of a 10B-parameter model which needs about 200GB of memory when training with Adam optimizer using mixed-precision. To get started, follow Modify a PyTorch Training Script to adapt SMPs’ APIs in your training script. Benchmarking performance.

Scripts

Scripts Benchmark APIs Engineering

Best practices for load testing Amazon SageMaker real-time inference endpoints

AWS Machine Learning

JANUARY 10, 2023

We first benchmark the performance of our model on a single instance to identify the TPS it can handle per our acceptable latency requirements. The entire set of code for the example is available in the following GitHub repository. Overview of solution. For CPUUtilization , you may see percentages above 100% at first in CloudWatch.

Best practices

Best practices Scripts APIs Metrics

Benchmarking Amazon Nova and GPT-4o models with FloTorch

Pixtral-12B-2409 is now available on Amazon Bedrock Marketplace

Trending Sources

GraphStorm 0.3: Scalable, multi-task learning on graphs with user-friendly APIs

LLM-as-a-judge on Amazon Bedrock Model Evaluation

Optimizing AI responsiveness: A practical guide to Amazon Bedrock latency-optimized inference

Anthropic Claude 3.5 Sonnet ranks number 1 for business and finance in S&P AI Benchmarks by Kensho

Benchmark and optimize endpoint deployment in Amazon SageMaker JumpStart

Speed up your time series forecasting by up to 50 percent with Amazon SageMaker Canvas UI and AutoML APIs

Mistral-Small-24B-Instruct-2501 is now available on SageMaker Jumpstart and Amazon Bedrock Marketplace

Generate training data and cost-effectively train categorical models with Amazon Bedrock

Evaluate RAG responses with Amazon Bedrock, LlamaIndex and RAGAS

From RAG to fabric: Lessons learned from building real-world RAGs at GenAIIC – Part 2

Fine-tune LLMs with synthetic data for context-based Q&A using Amazon Bedrock

Speed up your AI inference workloads with new NVIDIA-powered capabilities in Amazon SageMaker

Your guide to generative AI and ML at AWS re:Invent 2024

Get started with Amazon Titan Text Embeddings V2: A new state-of-the-art embeddings model on Amazon Bedrock

Reduce conversational AI response time through inference at the edge with AWS Local Zones

AI21 Labs Jamba-Instruct model is now available in Amazon Bedrock

Transition your Amazon Forecast usage to Amazon SageMaker Canvas

Build a secure enterprise application with Generative AI and RAG using Amazon SageMaker JumpStart

Learn how Amazon Ads created a generative AI-powered image generation capability using Amazon SageMaker

Accelerated PyTorch inference with torch.compile on AWS Graviton processors

Package and deploy classical ML and LLMs easily with Amazon SageMaker, part 2: Interactive User Experiences in SageMaker Studio

A review of purpose-built accelerators for financial services

Optimize your machine learning deployments with auto scaling on Amazon SageMaker

Build a multilingual automatic translation pipeline with Amazon Translate Active Custom Translation

Secure AccountantAI Chatbot: Lili’s journey with Amazon Bedrock

Evaluation of generative AI techniques for clinical report summarization

Enhance performance of generative language models with self-consistency prompting on Amazon Bedrock

Optimize pet profiles for Purina’s Petfinder application using Amazon Rekognition Custom Labels and AWS Step Functions

Image classification model selection using Amazon SageMaker JumpStart

Cepsa Química improves the efficiency and accuracy of product stewardship using Amazon Bedrock

A progress update on our commitment to safe, responsible generative AI

How to use public reviews to improve customer experience

How Forethought saves over 66% in costs for generative AI models using Amazon SageMaker

Exciting new developments from Spearline

Run PyTorch Lightning and native PyTorch DDP on Amazon SageMaker Training, featuring Amazon Search

Improved ML model deployment using Amazon SageMaker Inference Recommender

The executive’s guide to generative AI for sustainability

Building an efficient MLOps platform with OSS tools on Amazon ECS with AWS Fargate

Build a RAG-based QnA application using Llama3 models from SageMaker JumpStart

Snowflake Arctic models are now available in Amazon SageMaker JumpStart

Train gigantic models with near-linear scaling using sharded data parallelism on Amazon SageMaker

Best practices for load testing Amazon SageMaker real-time inference endpoints

Stay Connected