How to capture user behavior signals with the UBI schema, ingest them into Amazon OpenSearch Service, and convert those signals into relevance judgments with Search Relevance Workbench so you can measure and validate search-result quality.
Collect query and event data using the UBI open schema so every search and user action is linkable via query_id.
Record zero-result queries and absolute positions for paginated results to capture gaps like vocabulary mismatches or long-tail failures.
It's the lack of deliberate signal collection, measurement, and a feedback loop to act on it. Together, UBI and SRW give you a repeatable framework: collect signals, turn them into relevance judgments, and validate every change before it ships. In this post, we show you how to capture UBI data on an Amazon OpenSearch Service domain and use those signals to evaluate search quality.
The application creates the ID when it runs a search and stamps it on every subsequent event the user produces, until the user issues a new search or the session ends. opensearch: hosts: ["https:// "] aws: serverless: false region: sts_role_arn: index_type: custom index: ubi_events. Optionally, both pipelines archive records to Amazon Simple Storage Service (Amazon S3).
When search feels broken, the culprit is rarely the engine. You can close this gap on Amazon OpenSearch Service using User Behavior Insights (UBI), an open schema standard for capturing search behavior, and Search Relevance Workbench (SRW), a toolkit for measuring and evaluating search quality. Your application generates the UBI-formatted records.
Learn how to observe and evaluate production AI agents by combining an agent running on AWS with OpenSearch Agent Health. This post walks through deploying an agent and its observability pipeline to AWS, then using Agent Health to explore traces and run evaluations that measure and improve agent quality over time.
Learn how to migrate an Amazon CloudSearch domain to Amazon OpenSearch Serverless: assess your configuration, create a collection with explicit index mappings, convert your documents and queries to the OpenSearch query DSL, configure security, load data with Amazon OpenSearch Ingestion, and validate before cutover.
NarrateAI delivers production-ready LLM quality assurance on Amazon Bedrock. This post details five techniques—adaptive pipeline orchestration, cross-account multi-model failover, real-time streaming evaluation, composite evaluation, and data accuracy verification—that reach about 99% numerical accuracy while streaming
Comparing models on dollars per million tokens misses what production workloads actually pay for: outcomes. This post shares an open-source benchmarking harness that measures cost per correct answer, agent trajectory cost, and rubric-graded deliverable quality across OpenAI models on Amazon Bedrock.
The 2026 Open Data Infrastructure Report finds 62% of current users already run OpenSearch for AI workloads.
We’re pleased to confirm the successful completion of our annual Amazon Web Services (AWS) Outsourced Service Provider’s Audit Report (OSPAR) assessment on July 29, 2026, in line with the OSPAR version 2.0 framework. The Association of Banks in Singapore (ABS) established the Guidelines on Control Objectives and Proced
Open the app view to save this story, compare related coverage, and continue from the same source.
