Adaptive behavioral analytics for production AI
Discover, investigate, and track insights to improve AI products
Platform
Enterprise platform that integrates with your production AI stack
- Open distribution: Deploy our free platform in a scalable Kubernetes cluster in your VPC
- Enterprise controls: Keep data secure with enterprise authentication and authorization using OpenID Connect SSO and role-based access controls
- Flexible integrations: Ingest logs via OTEL traces, SQL, or SDK, bring your own evals, use your current LLM providers, and leverage existing notifications channels
Pipeline
Automated pipeline for scalable data analysis on production AI logs
- Enrich production AI logs with metrics, statistics, attributes, evals, and LLM as judge metrics to capture the status of AI product behavior
- Analyze enriched production logs to uncover behavioral signals with unsupervised clustering, topic modeling, anomaly detection, and data drift assessment
- Publish signals from analysis, report on them via dashboard or notification channels, and select which signals to track in a shared dashboard
Workflow
Intuitive workflow to discover and act on behavioral signals
- Discover behavioral signals that correspond to daily shifts, clusters, or outliers in production AI logs
- Investigate these signals with contextual evidence from relevant production data to quickly triage and take action
- Track signals to codify preferences and optimize future analysis on behaviors you care about most
Distributional is built for AI product team’s scaling AI agents
Be confident in AI product behavior
Continuously analyze your product to understand behavior, and evolve your product based on this understanding
Discover
Discover signals on AI product behavior – the interplay and correlations between users, context, tools, models, and metrics – that provide guidance on how to improve, fix, optimize, or evolve your product over time.
Distributional’s analytics workflow starts with daily insights on your AI products and agents. The Distributional Data Pipeline enriches your production AI Logs, analyzes these enriched Logs to produce behavioral signals, and publishes these signals as human readable Insights in your Dashboards and Notifications Connections. These Insights are based on outliers, shifts, changes, topics, clusters, or other interesting usage patterns that represent opportunities to improve or fix your AI product.
Investigate
Investigate and triage these behavioral signals with evidence in the form of specific logs that are driving the signal and visualizations of temporal or comparative analysis of these signals to prioritize and take action on these insights.
Each new Insight is linked to supporting evidence to facilitate rapid root cause analysis. This evidence includes a curated set of Logs that are most correlated with the insight. It also includes single segment analysis, segment comparisons, and temporal comparisons in the Explorer. And it includes recommendations for new Segments or Metrics.
Track
Track these signals as new segments, metrics, or filters to guide future analysis of production AI logs and customize the dashboard the team views to make daily decisions on the AI product.
The purpose of these Insights is to guide prioritization of fixes and improvements to your AI product or agent. As you investigate these Insights and decide which are most relevant, you can track these Insights as Metrics or Segments. When tracked, these Metrics and Segments are automatically analyzed in the Distributional Data Pipeline and become a part of Distributional Dashboards.
Improve
Improve your AI product by leveraging insights from these signals to fix issues, identify new segments of users to target, or boost model performance with reinforcement learning, fine tuning, or context engineering.
As you use DBNL over time, these Segments, Metrics, and Insights become tailored to your specific AI product or agent, improving the analysis and increasing the capacity for your team to make and track product decisions. In this sense, the workflow, as well as the analytics supporting it, adapt to your preferences over time.
Understand your AI products
Improve or fix your AI product using behavioral signals buried in your AI production data