The announcement
Distributional's next chapter: Talaria Scientific
Distributional is now Talaria Scientific: same company, same investors, hard pivot. The honest story of what worked, what didn't, and the new mission one level deeper.
Blog
Company news and the Distributional technical archive, preserved at its original addresses.
Selected archive
Technical writing from the Distributional era, restored at its original addresses with original bylines and dates.
Beyond averages: how DBNL's distribution comparison reveals what summary metrics hide
Why P95s and averages hide bimodal regressions: four use cases for distribution comparison and the design decisions behind them. From the Distributional archive. The product described has been sunset: read the pivot post, then the Talaria Scientific manifesto.
Examples of issues that break agents in production
Concrete failure modes analytics catches in production agents: evals that pass but do not generalize, topic shifts, and complexity-correlated issues. From the Distributional archive. The product described has been sunset: read the pivot post, then the Talaria Scientific manifesto.
The four stages of AI observability maturity
The four stages of AI observability maturity: the aggregate trap, the manual-investigation plateau, the scaling threshold, and behavioral signals. From the Distributional archive. The product described has been sunset: read the pivot post, then the Talaria Scientific manifesto.
Enrichments versus evals
Evals as narrow known-known criteria versus enrichments as many weak signals for unknown-unknown discovery in production agent logs. From the Distributional archive. The product described has been sunset: read the pivot post, then the Talaria Scientific manifesto.
The agent observability hierarchy
The agent observability hierarchy: logging, monitoring, analytics; why scale and agent complexity broke trace-reading, and how the layers complement. From the Distributional archive. The product described has been sunset: read the pivot post, then the Talaria Scientific manifesto.
Demo: Financial analyst
A financial research agent under behavioral analytics: baseline understanding, proactive insights, and active governance for regulated industries. From the Distributional archive. The product described has been sunset: read the pivot post, then the Talaria Scientific manifesto.
Demo: Improving agents with production data analysis
The open NVIDIA stack (NeMo Agent Toolkit, NIM, Optimizer) on a production agent: finding improvement signals in production data. From the Distributional archive. The product described has been sunset: read the pivot post, then the Talaria Scientific manifesto.
Demo: Agent hyperparameter optimization with behavioral analytics
Closing the loop from analytics to optimization: diagnosing a seeded bug and driving hyperparameter optimization from behavioral signals. From the Distributional archive. The product described has been sunset: read the pivot post, then the Talaria Scientific manifesto.
Demo: Fixing agent issues in production with tab complete analytics
Google ADK calculator tutorial: multiple ingestion pathways, circumstance-and-pathway analysis, and insight-driven fixes. From the Distributional archive. The product described has been sunset: read the pivot post, then the Talaria Scientific manifesto.
Lesson notes from the "Hidden Signals in Production AI Logs" session
Q&A notes on production AI log analytics: analytics versus monitoring versus logging, how behavioral analytics works, and deployment. From the Distributional archive. The product described has been sunset: read the pivot post, then the Talaria Scientific manifesto.
Analytics-driven agent A/B testing
End-to-end agent A/B testing: discover an issue from behavioral signals, root-cause it, fix it, and verify with distribution comparison. From the Distributional archive. The product described has been sunset: read the pivot post, then the Talaria Scientific manifesto.
DBNL fits your agent data
Agent data ingestion: OTel traces versus SDK logs versus SQL pulls, the DBNL Semantic Convention, and concrete adapters. From the Distributional archive. The product described has been sunset: read the pivot post, then the Talaria Scientific manifesto.
Integrating DBNL with NVIDIA NeMo Agent Toolkit
Seven-step tutorial for integrating DBNL with the NVIDIA NeMo Agent Toolkit, from install to analyzing traces. From the Distributional archive. The product described has been sunset: read the pivot post, then the Talaria Scientific manifesto.
Adapting product analytics to the AI era
Scott Clark's pivot-to-analytics thesis: production AI as a black box, behavior as the unit of analysis, and production-led development. From the Distributional archive. The product described has been sunset: read the pivot post, then the Talaria Scientific manifesto.
The Distributional Workflow: Define, Detect, Understand, and Improve
The Define, Detect, Understand, Improve workflow walked end to end on a customer-support agent. From the Distributional archive. The product described has been sunset: read the pivot post, then the Talaria Scientific manifesto.
Scale, Metrics, Tests, and Results in Distributional
How scale, metrics, tests, and results compose Distributional's analysis model for translating unstructured LLM data into testing. From the Distributional archive. The product described has been sunset: read the pivot post, then the Talaria Scientific manifesto.
How Distributional tests for consistent app behavior in production
Prompts and responses as random variables: the hypothesis-testing framing behind testing AI apps for consistent production behavior. From the Distributional archive. The product described has been sunset: read the pivot post, then the Talaria Scientific manifesto.
How Distributional’s Similarity Index works
How the Similarity Index works: computing a 0-100 score per column and aggregating to app level to test behavioral consistency. From the Distributional archive. The product described has been sunset: read the pivot post, then the Talaria Scientific manifesto.
How Distributional balances security with insights
Security architecture for AI testing: in-environment deployment, no call-home, namespace access control, and secure integration. From the Distributional archive. The product described has been sunset: read the pivot post, then the Talaria Scientific manifesto.
Model Evaluation: From ML to GenAI
Why GenAI evaluation differs from classical ML: multi-component coupling, non-stationarity, non-determinism, and where distributional testing fits. From the Distributional archive. The product described has been sunset: read the pivot post, then the Talaria Scientific manifesto.
Designing for speed and scale: Integrating Distributional within an existing environment
How the platform integrates within existing environments: cloud and VPC deployment, storage integration, and data-format agnosticism. From the Distributional archive. The product described has been sunset: read the pivot post, then the Talaria Scientific manifesto.
Understanding Distributional’s Platform Architecture
Distributional's platform architecture: integrate with existing storage, scheduled batch ingestion, and analytic-style processing of AI logs. From the Distributional archive. The product described has been sunset: read the pivot post, then the Talaria Scientific manifesto.
Adaptive testing for AI confidence
A three-step methodology for adaptive AI testing: define desired behavior, understand change, continuously adapt as the state evolves. From the Distributional archive. The product described has been sunset: read the pivot post, then the Talaria Scientific manifesto.
Distributional Simplifies Adaptive Testing with Similarity Index and Key Insights
Why static golden-dataset testing breaks, and how Similarity Index and Key Insights pinpoint and explain behavioral change. From the Distributional archive. The product described has been sunset: read the pivot post, then the Talaria Scientific manifesto.
Driving confidence in AI: Understanding deployment testing through model version updates
Deployment testing through LLM version updates: metric choices, vibe checks versus statistical perspective, and automating the comparison. From the Distributional archive. The product described has been sunset: read the pivot post, then the Talaria Scientific manifesto.
Bridging the AI Confidence Gap with adaptive behavioral testing
Why AI confidence erodes between development benchmarks and production, and adaptive behavioral testing over distributions as the fix. From the Distributional archive. The product described has been sunset: read the pivot post, then the Talaria Scientific manifesto.
What is Distributional? Explaining AI behavior through the language of statistical distributions
Distributions as the native language of GenAI behavior: the AI Confidence Gap and statistical testing over behavior distributions. From the Distributional archive. The product described has been sunset: read the pivot post, then the Talaria Scientific manifesto.
The rise of AI platforms
Why internal AI platforms echo the 2018 ML platform wave: the emerging component stack and how platform teams sequence adoption. From the Distributional archive. The product described has been sunset: read the pivot post, then the Talaria Scientific manifesto.
Driving confidence in AI: Foundational distributional testing explained through a RAG example
Foundational distributional testing walked through a RAG example: baselines, distributional comparison, experiments, and production data. From the Distributional archive. The product described has been sunset: read the pivot post, then the Talaria Scientific manifesto.
The AI Software Development Lifecycle: A practical framework for modern AI systems
The AI SDLC framework: how the development lifecycle morphs across traditional software, ML, and GenAI, with risk rising as determinism falls. From the Distributional archive. The product described has been sunset: read the pivot post, then the Talaria Scientific manifesto.
Why testing is an enterprise problem that requires an enterprise solution
Part 3 of the enterprise AI testing series: why production-grade AI testing needs an enterprise platform, not developer-tool latitude. From the Distributional archive. The product described has been sunset: read the pivot post, then the Talaria Scientific manifesto.
Why all ML and AI use cases need standardized testing
Part 2 of the enterprise AI testing series: the testing gaps in classical ML workflows, from process visibility to false-alarm suppression. From the Distributional archive. The product described has been sunset: read the pivot post, then the Talaria Scientific manifesto.
Why generative AI creates unique testing challenges
Field notes from 1,000+ hours with Fortune 500 AI leaders on why GenAI breaks traditional testing: non-determinism, non-stationarity, and behavior definition. From the Distributional archive. The product described has been sunset: read the pivot post, then the Talaria Scientific manifesto.
We raised $11M for better AI testing
Scott Clark's 2023 founding essay: a decade of AI testing problems through Yelp, SigOpt, and Intel, and the thesis behind Distributional. From the Distributional archive. The product described has been sunset: read the pivot post, then the Talaria Scientific manifesto.