AWS Machine Learning

OCTOBER 29, 2024

Recently, we’ve been witnessing the rapid development and evolution of generative AI applications, with observability and evaluation emerging as critical aspects for developers, data scientists, and stakeholders. This feature allows you to separate data into logical partitions, making it easier to analyze and process data later.

Best practices 110

Best practices Feedback Metrics APIs 110

AWS Machine Learning

FEBRUARY 12, 2025

This approach allows organizations to assess their AI models effectiveness using pre-defined metrics, making sure that the technology aligns with their specific needs and objectives. This format promotes proper processing of evaluation data. Both features use the LLM-as-a-judge technique behind the scenes but evaluate different things.

Metrics 93

Metrics Engineering APIs Benchmark 93

AWS Machine Learning

DECEMBER 9, 2024

In the initial stages of an ML project, data scientists collaborate closely, sharing experimental results to address business challenges. However, keeping track of numerous experiments, their parameters, metrics, and results can be difficult, especially when working on complex projects simultaneously.

Metrics 95

Metrics APIs Engineering Accountability 95

AWS Machine Learning

AUGUST 2, 2024

With GraphStorm, you can build solutions that directly take into account the structure of relationships or interactions between billions of entities, which are inherently embedded in most real-world data, including fraud detection scenarios, recommendations, community detection, and search/retrieval problems. Specifically, GraphStorm 0.3

APIs 115

APIs Benchmark Construction Enterprise 115

AWS Machine Learning

MARCH 6, 2025

In the rapidly evolving landscape of artificial intelligence, Retrieval Augmented Generation (RAG) has emerged as a game-changer, revolutionizing how Foundation Models (FMs) interact with organization-specific data. More sophisticated metrics are needed to evaluate factual alignment and accuracy.

Metrics 84

Metrics Enterprise APIs Engineering 84

AWS Machine Learning

NOVEMBER 20, 2024

Amazon Bedrock is a fully managed service that offers a choice of high-performing foundation models (FMs) from leading AI companies like AI21 Labs, Anthropic, Cohere, Meta, Mistral AI, Stability AI, and Amazon through a single API, along with a broad set of capabilities to build generative AI applications with security, privacy, and responsible AI.

Construction 115

Construction APIs Metrics Accountability 115

AWS Machine Learning

MAY 30, 2024

The opportunities to unlock value using AI in the commercial real estate lifecycle starts with data at scale. Although CBRE provides customers their curated best-in-class dashboards, CBRE wanted to provide a solution for their customers to quickly make custom queries of their data using only natural language prompts.

Real estate 128

Real estate APIs Metrics Construction 128

AWS Machine Learning

NOVEMBER 15, 2024

Principal wanted to use existing internal FAQs, documentation, and unstructured data and build an intelligent chatbot that could provide quick access to the right information for different roles. The chatbot improved access to enterprise data and increased productivity across the organization.

Chatbots 113

Chatbots Engineering Enterprise Government 113

AWS Machine Learning

MARCH 25, 2025

Automated safety guards Integrated Amazon CloudWatch alarms monitor metrics on an inference component. AlarmName This CloudWatch alarm is configured to monitor metrics on an InferenceComponent. For more information about the SageMaker AI API, refer to the SageMaker AI API Reference.

APIs 85

APIs Engineering Accountability Metrics 85

AWS Machine Learning

FEBRUARY 18, 2025

During these live events, F1 IT engineers must triage critical issues across its services, such as network degradation to one of its APIs. This impacts downstream services that consume data from the API, including products such as F1 TV, which offer live and on-demand coverage of every race as well as real-time telemetry.

APIs 69

APIs Engineering Metrics Big data 69

AWS Machine Learning

JANUARY 5, 2024

Many organizations have been using a combination of on-premises and open source data science solutions to create and manage machine learning (ML) models. Data science and DevOps teams may face challenges managing these isolated tool stacks and systems.

Management 132

Management APIs Engineering Government 132

AWS Machine Learning

OCTOBER 9, 2024

Amazon Lookout for Metrics is a fully managed service that uses machine learning (ML) to detect anomalies in virtually any time-series business or operational metrics—such as revenue performance, purchase transactions, and customer acquisition and retention rates—with no ML experience required.

Metrics 90

Metrics APIs Engineering Accountability 90

AWS Machine Learning

FEBRUARY 21, 2025

This serves as an example of how generative AI can streamline operations that involve diverse data types and formats. Rule-based systems or specialized machine learning (ML) models often struggle with the variability of real-world documents, especially when dealing with semi-structured and unstructured data.

APIs 85

APIs Healthcare Consulting Consulting 85

AWS Machine Learning

JANUARY 28, 2025

To effectively optimize AI applications for responsiveness, we need to understand the key metrics that define latency and how they impact user experience. These metrics differ between streaming and nonstreaming modes and understanding them is crucial for building responsive AI applications.

Benchmark 71

Benchmark APIs Engineering Metrics 71

AWS Machine Learning

NOVEMBER 13, 2024

In the context of generative AI , significant progress has been made in developing multimodal embedding models that can embed various data modalities—such as text, image, video, and audio data—into a shared vector space. Each data point in the database is associated with a vector that encapsulates its attributes or features.

Engineering 114

Engineering Management APIs Healthcare 114

AWS Machine Learning

SEPTEMBER 20, 2024

These agents help users complete actions based on organizational data and user input, orchestrating interactions between foundation models (FMs), data sources, software applications, and user conversations. Amazon Bedrock Agents offers developers the ability to build and configure autonomous agents in their applications.

APIs 105

APIs Chatbots Construction Engineering 105

AWS Machine Learning

NOVEMBER 14, 2024

It enables you to privately customize the FM of your choice with your data using techniques such as fine-tuning, prompt engineering, and retrieval augmented generation (RAG) and build agents that run tasks using your enterprise systems and data sources while adhering to security and privacy requirements.

APIs 109

APIs Engineering Personalization Transportation 109

Cisco - Contact Center

MAY 24, 2024

Meraki APIs allow businesses to automate repetitive and time-consuming tasks, and configure and deploy networks quickly at a scale. Developrs can leverage API operations to retrieve performance metrics, monitor network health, analyze traffic data and create custom reports to gain insights into the network usage.

APIs 68

APIs Metrics 68

AWS Machine Learning

JANUARY 12, 2023

At The Very Group , which operates digital retailer Very, security is a top priority in handling data for millions of customers. However, this can mean processing customer data in the form of personally identifiable information (PII) in relation to activities such as purchases, returns, use of flexible payment options, and account management.

Engineering 126

Engineering APIs Accountability Metrics 126

AWS Machine Learning

MARCH 14, 2025

Although automated metrics are fast and cost-effective, they can only evaluate the correctness of an AI response, without capturing other evaluation dimensions or providing explanations of why an answer is problematic. Human evaluation, although thorough, is time-consuming and expensive at scale.

Metrics 81

Metrics Best practices Engineering APIs 81

AWS Machine Learning

MARCH 3, 2025

Performance metrics and benchmarks Pixtral 12B is trained to understand both natural images and documents, achieving 52.5% You can find detailed usage instructions, including sample API calls and code snippets for integration. In this particular use case, we use Pixtral 12B to analyze an intricate image containing GDP data.

Benchmark 88

Benchmark APIs Enterprise Construction 88

AWS Machine Learning

JANUARY 4, 2024

One such area that is evolving is using natural language processing (NLP) to unlock new opportunities for accessing data through intuitive SQL queries. Instead of dealing with complex technical code, business users and data analysts can ask questions related to data and insights in plain language.

Best practices 139

Best practices Enterprise Engineering APIs 139

AWS Machine Learning

OCTOBER 5, 2022

In this post, we discuss the key elements needed to evaluate the performance aspect of a content moderation service in terms of various accuracy metrics, and a provide an example using Amazon Rekognition Content Moderation API’s. To annotate your image data, you can use Amazon SageMaker Ground Truth (GT)to manage image annotation.

Metrics 95

Metrics APIs Advertising Enterprise 95

AWS Machine Learning

FEBRUARY 15, 2024

With the use of cloud computing, big data and machine learning (ML) tools like Amazon Athena or Amazon SageMaker have become available and useable by anyone without much effort in creation and maintenance. This dilemma hampers the creation of efficient models that use data to generate business-relevant insights.

Metrics 118

Metrics Analytics APIs Big data 118

AWS Machine Learning

FEBRUARY 25, 2025

To build a generative AI -based conversational application integrated with relevant data sources, an enterprise needs to invest time, money, and people. First, you would need build connectors to the data sources. For a full list of supported data source connectors, see Amazon Q Business connectors.

Enterprise 92

Enterprise Engineering APIs Accountability 92

AWS Machine Learning

JANUARY 24, 2025

Fine-tuning pre-trained language models allows organizations to customize and optimize the models for their specific use cases, providing better performance and more accurate outputs tailored to their unique data and requirements. Amazon Bedrock prioritizes security through a comprehensive approach to protect customer data and AI workloads.

Best practices 112

Best practices Accountability industry standards Metrics 112

AWS Machine Learning

JANUARY 7, 2025

It can help collect more data on the value of LLMs for your content translation use cases. Translation Memory eXchange (TMX) is a widely used open standard for representing and exchanging TM data. Also note the completion metrics on the left pane, displaying latency, input/output tokens, and quality scores.

Engineering 107

Engineering Metrics industry standards Finance 107

AWS Machine Learning

JANUARY 25, 2023

This post shows you how to use an integrated solution with Amazon Lookout for Metrics to break these barriers by quickly and easily detecting anomalies in the key performance indicators (KPIs) of your interest. Lookout for Metrics automatically detects and diagnoses anomalies (outliers from the norm) in business and operational data.

Metrics 94

Metrics APIs Enterprise Analytics 94

AWS Machine Learning

OCTOBER 2, 2024

However, scaling up generative AI and making adoption easier for different lines of businesses (LOBs) comes with challenges around making sure data privacy and security, legal, compliance, and operational complexities are governed on an organizational level. In this post, we discuss how to address these challenges holistically.

Government 123

Government APIs Enterprise Best practices 123

AWS Machine Learning

AUGUST 7, 2024

RAG is the process of optimizing the output of an LLM so it references an authoritative knowledge base outside of its training data sources before generating a response. RAG also scales better with more data compared to pure generative models, and it doesn’t require fine-tuning of the model when new data is added to the knowledge base.

APIs 132

APIs Chatbots Metrics Education 132

AWS Machine Learning

FEBRUARY 7, 2025

Our commitment to innovation led us to a pivotal challenge: how to harness the power of machine learning (ML) to further enhance our competitive edge while balancing this technological advancement with strict data security requirements and the need to streamline access to our existing internal resources.

Entertainment 77

Entertainment APIs Technology Feedback 77

AWS Machine Learning

SEPTEMBER 29, 2023

This guest post is co-written by Lydia Lihui Zhang, Business Development Specialist, and Mansi Shah, Software Engineer/Data Scientist, at Planet Labs. In late 2023, Planet announced a partnership with AWS to make its geospatial data available through Amazon SageMaker.

Engineering 134

Engineering Metrics Surveys APIs 134

AWS Machine Learning

APRIL 24, 2024

Amazon Bedrock is a fully managed service that offers a choice of high-performing foundation models (FMs) from leading AI companies like AI21 Labs, Anthropic, Cohere, Meta, Stability AI, and Amazon using a single API, along with a broad set of capabilities you need to build generative AI applications with security, privacy, and responsible AI.

APIs 134

APIs Engineering Metrics Management 134

AWS Machine Learning

MAY 7, 2024

We match the most relevant videos to text-based search queries by incorporating new multimodal embedding models like Amazon Titan Multimodal Embeddings to encode all visual, visual-meta, and transcription data. Amazon Transcribe The transcription for the entire video is generated using the StartTranscriptionJob API.

APIs 132

APIs Advertising Entertainment Metrics 132

AWS Machine Learning

JULY 10, 2024

Frontier large language models (LLMs) like Anthropic Claude on Amazon Bedrock are trained on vast amounts of data, allowing Anthropic Claude to understand and generate human-like text. The fine-tuning as a deep level of customization represents a key differentiating factor by using your own unique data.

APIs 130

APIs Airlines Metrics Engineering 130

AWS Machine Learning

OCTOBER 24, 2024

Such use cases, which augment a large language model’s (LLM) knowledge with external data sources, are known as Retrieval-Augmented Generation (RAG). The second post outlines how to work with multiple data formats such as structured data (tables, databases) and images. doc,pdf, or.txt).

Chatbots 88

Chatbots Metrics Scripts APIs 88

AWS Machine Learning

FEBRUARY 24, 2025

Performance metrics and benchmarks According to Mistral, the instruction-tuned version of the model achieves over 81% accuracy on Massive Multitask Language Understanding (MMLU) with 150 tokens per second latency, making it currently the most efficient model in its category. It doesnt support Converse APIs or other Amazon Bedrock tooling.

APIs 83

APIs Enterprise Benchmark Feedback 83