# Blog: Distributional

> Company news and the Distributional technical archive, preserved at its original addresses.

Each entry links the post's plain-markdown twin; HTML versions live at `https://distributional.com/blog/<slug>`. Full-content RSS: https://distributional.com/rss.xml. Entries marked "archive" are restored Distributional-era posts (original bylines and dates) — the product they describe has been sunset. For what the company is now, read the pivot post first (https://distributional.com/blog/distributional-is-now-talaria), then the Talaria manifesto (https://talariasci.com/blog/why-im-building-talaria).

- [Distributional's next chapter: Talaria Scientific](https://distributional.com/blog/distributional-is-now-talaria.md) (July 24, 2026): Distributional is now Talaria Scientific: same company, same investors, hard pivot. The honest story of what worked, what didn't, and the new mission one level deeper.
- [Beyond averages: how DBNL's distribution comparison reveals what summary metrics hide](https://distributional.com/blog/beyond-averages-how-dbnls-distribution-comparison-reveals-what-summary-metrics-hide.md) (May 27, 2026, by Kat Crisostomo, archive): Why P95s and averages hide bimodal regressions: four use cases for distribution comparison and the design decisions behind them. From the Distributional archive. The product described has been sunset: read the pivot post, then the Talaria Scientific manifesto.
- [Examples of issues that break agents in production](https://distributional.com/blog/examples-of-issues-that-break-agents-in-production.md) (May 19, 2026, by Nick Payton, archive): Concrete failure modes analytics catches in production agents: evals that pass but do not generalize, topic shifts, and complexity-correlated issues. From the Distributional archive. The product described has been sunset: read the pivot post, then the Talaria Scientific manifesto.
- [The four stages of AI observability maturity](https://distributional.com/blog/the-four-stages-of-ai-observability-maturity.md) (May 5, 2026, by Nick Payton, archive): The four stages of AI observability maturity: the aggregate trap, the manual-investigation plateau, the scaling threshold, and behavioral signals. From the Distributional archive. The product described has been sunset: read the pivot post, then the Talaria Scientific manifesto.
- [Enrichments versus evals](https://distributional.com/blog/enrichments-versus-evals.md) (April 28, 2026, by Nick Payton, archive): Evals as narrow known-known criteria versus enrichments as many weak signals for unknown-unknown discovery in production agent logs. From the Distributional archive. The product described has been sunset: read the pivot post, then the Talaria Scientific manifesto.
- [The agent observability hierarchy](https://distributional.com/blog/the-agent-observability-hierarchy.md) (April 21, 2026, by Nick Payton, archive): The agent observability hierarchy: logging, monitoring, analytics; why scale and agent complexity broke trace-reading, and how the layers complement. From the Distributional archive. The product described has been sunset: read the pivot post, then the Talaria Scientific manifesto.
- [Demo: Financial analyst](https://distributional.com/blog/demo-financial-analyst.md) (April 17, 2026, by Erin LeDell, archive): A financial research agent under behavioral analytics: baseline understanding, proactive insights, and active governance for regulated industries. From the Distributional archive. The product described has been sunset: read the pivot post, then the Talaria Scientific manifesto.
- [Demo: Improving agents with production data analysis](https://distributional.com/blog/demo-improving-agents-with-production-data-analysis.md) (April 8, 2026, by Nick Payton, archive): The open NVIDIA stack (NeMo Agent Toolkit, NIM, Optimizer) on a production agent: finding improvement signals in production data. From the Distributional archive. The product described has been sunset: read the pivot post, then the Talaria Scientific manifesto.
- [Demo: Agent hyperparameter optimization with behavioral analytics](https://distributional.com/blog/demo-agent-hyperparameter-optimization-with-behavioral-analytics.md) (April 6, 2026, by Nick Payton, archive): Closing the loop from analytics to optimization: diagnosing a seeded bug and driving hyperparameter optimization from behavioral signals. From the Distributional archive. The product described has been sunset: read the pivot post, then the Talaria Scientific manifesto.
- [Demo: Fixing agent issues in production with tab complete analytics](https://distributional.com/blog/demo-fixing-agent-issues-in-production-with-tab-complete-analytics.md) (March 30, 2026, by Nick Payton, archive): Google ADK calculator tutorial: multiple ingestion pathways, circumstance-and-pathway analysis, and insight-driven fixes. From the Distributional archive. The product described has been sunset: read the pivot post, then the Talaria Scientific manifesto.
- [Lesson notes from the "Hidden Signals in Production AI Logs" session](https://distributional.com/blog/lesson-notes-from-hidden-signals-in-production-ai-logs-session.md) (March 3, 2026, by Nick Payton, archive): Q&A notes on production AI log analytics: analytics versus monitoring versus logging, how behavioral analytics works, and deployment. From the Distributional archive. The product described has been sunset: read the pivot post, then the Talaria Scientific manifesto.
- [Analytics-driven agent A/B testing](https://distributional.com/blog/analytics-driven-agent-a-b-testing.md) (February 18, 2026, by Nick Payton, archive): End-to-end agent A/B testing: discover an issue from behavioral signals, root-cause it, fix it, and verify with distribution comparison. From the Distributional archive. The product described has been sunset: read the pivot post, then the Talaria Scientific manifesto.
- [DBNL fits your agent data](https://distributional.com/blog/dbnl-fits-your-agent-data.md) (February 11, 2026, by Renaud Bourassa, archive): Agent data ingestion: OTel traces versus SDK logs versus SQL pulls, the DBNL Semantic Convention, and concrete adapters. From the Distributional archive. The product described has been sunset: read the pivot post, then the Talaria Scientific manifesto.
- [Integrating DBNL with NVIDIA NeMo Agent Toolkit](https://distributional.com/blog/integrating-dbnl-with-the-nvidia-nemo-agent-toolkit.md) (January 21, 2026, by Nick Payton, archive): Seven-step tutorial for integrating DBNL with the NVIDIA NeMo Agent Toolkit, from install to analyzing traces. From the Distributional archive. The product described has been sunset: read the pivot post, then the Talaria Scientific manifesto.
- [Adapting product analytics to the AI era](https://distributional.com/blog/adapting-product-analytics-to-the-ai-era.md) (November 4, 2025, archive): Scott Clark's pivot-to-analytics thesis: production AI as a black box, behavior as the unit of analysis, and production-led development. From the Distributional archive. The product described has been sunset: read the pivot post, then the Talaria Scientific manifesto.
- [The Distributional Workflow: Define, Detect, Understand, and Improve](https://distributional.com/blog/the-distributional-workflow-define-detect-understand-and-improve.md) (August 26, 2025, by Nick Payton, archive): The Define, Detect, Understand, Improve workflow walked end to end on a customer-support agent. From the Distributional archive. The product described has been sunset: read the pivot post, then the Talaria Scientific manifesto.
- [Scale, Metrics, Tests, and Results in Distributional](https://distributional.com/blog/scale-metrics-tests-and-results-in-distributional.md) (August 18, 2025, by Nick Payton, archive): How scale, metrics, tests, and results compose Distributional's analysis model for translating unstructured LLM data into testing. From the Distributional archive. The product described has been sunset: read the pivot post, then the Talaria Scientific manifesto.
- [How Distributional tests for consistent app behavior in production](https://distributional.com/blog/how-distributional-tests-for-consistent-app-behavior-in-production.md) (August 15, 2025, by Harvey Cheng, archive): Prompts and responses as random variables: the hypothesis-testing framing behind testing AI apps for consistent production behavior. From the Distributional archive. The product described has been sunset: read the pivot post, then the Talaria Scientific manifesto.
- [How Distributional’s Similarity Index works](https://distributional.com/blog/how-distributionals-similarity-index-works.md) (August 15, 2025, by Harvey Cheng, archive): How the Similarity Index works: computing a 0-100 score per column and aggregating to app level to test behavioral consistency. From the Distributional archive. The product described has been sunset: read the pivot post, then the Talaria Scientific manifesto.
- [How Distributional balances security with insights](https://distributional.com/blog/how-distributional-balances-security-with-insights.md) (June 26, 2025, by Ian Dewancker, archive): Security architecture for AI testing: in-environment deployment, no call-home, namespace access control, and secure integration. From the Distributional archive. The product described has been sunset: read the pivot post, then the Talaria Scientific manifesto.
- [Model Evaluation: From ML to GenAI](https://distributional.com/blog/model-evaluation-from-ml-to-genai.md) (June 5, 2025, by Erin LeDell, archive): Why GenAI evaluation differs from classical ML: multi-component coupling, non-stationarity, non-determinism, and where distributional testing fits. From the Distributional archive. The product described has been sunset: read the pivot post, then the Talaria Scientific manifesto.
- [Designing for speed and scale: Integrating Distributional within an existing environment](https://distributional.com/blog/designing-for-speed-and-scale-integrating-distributional-within-an-existing-environment.md) (June 4, 2025, by Renaud Bourassa, archive): How the platform integrates within existing environments: cloud and VPC deployment, storage integration, and data-format agnosticism. From the Distributional archive. The product described has been sunset: read the pivot post, then the Talaria Scientific manifesto.
- [Understanding Distributional’s Platform Architecture](https://distributional.com/blog/understanding-distributionals-platform-architecture.md) (May 22, 2025, by Renaud Bourassa, archive): Distributional's platform architecture: integrate with existing storage, scheduled batch ingestion, and analytic-style processing of AI logs. From the Distributional archive. The product described has been sunset: read the pivot post, then the Talaria Scientific manifesto.
- [Adaptive testing for AI confidence](https://distributional.com/blog/adaptive-testing-for-ai-confidence.md) (May 21, 2025, by Alex Gutow, archive): A three-step methodology for adaptive AI testing: define desired behavior, understand change, continuously adapt as the state evolves. From the Distributional archive. The product described has been sunset: read the pivot post, then the Talaria Scientific manifesto.
- [Distributional Simplifies Adaptive Testing with Similarity Index and Key Insights](https://distributional.com/blog/distributional-simplifies-adaptive-testing-with-similarity-index-and-key-insights.md) (May 21, 2025, by Jaron Parnala, archive): Why static golden-dataset testing breaks, and how Similarity Index and Key Insights pinpoint and explain behavioral change. From the Distributional archive. The product described has been sunset: read the pivot post, then the Talaria Scientific manifesto.
- [Driving confidence in AI: Understanding deployment testing through model version updates](https://distributional.com/blog/driving-confidence-in-ai-understanding-deployment-testing-through-model-version-updates.md) (April 16, 2025, by Tobi Andreasen, archive): Deployment testing through LLM version updates: metric choices, vibe checks versus statistical perspective, and automating the comparison. From the Distributional archive. The product described has been sunset: read the pivot post, then the Talaria Scientific manifesto.
- [Bridging the AI Confidence Gap with adaptive behavioral testing](https://distributional.com/blog/bridging-the-ai-confidence-gap-with-adaptive-behavioral-testing.md) (March 5, 2025, by Alex Gutow, archive): Why AI confidence erodes between development benchmarks and production, and adaptive behavioral testing over distributions as the fix. From the Distributional archive. The product described has been sunset: read the pivot post, then the Talaria Scientific manifesto.
- [What is Distributional? Explaining AI behavior through the language of statistical distributions](https://distributional.com/blog/what-is-distributional-explaining-ai-behavior-through-the-language-of-statistical-distributions.md) (February 26, 2025, by Kembey Gbarayor, archive): Distributions as the native language of GenAI behavior: the AI Confidence Gap and statistical testing over behavior distributions. From the Distributional archive. The product described has been sunset: read the pivot post, then the Talaria Scientific manifesto.
- [The rise of AI platforms](https://distributional.com/blog/the-rise-of-ai-platforms.md) (February 19, 2025, by Nick Payton, archive): Why internal AI platforms echo the 2018 ML platform wave: the emerging component stack and how platform teams sequence adoption. From the Distributional archive. The product described has been sunset: read the pivot post, then the Talaria Scientific manifesto.
- [Driving confidence in AI: Foundational distributional testing explained through a RAG example](https://distributional.com/blog/driving-confidence-in-ai-foundational-distributional-testing-explained-through-a-rag-example.md) (January 29, 2025, by Tobi Andreasen, archive): Foundational distributional testing walked through a RAG example: baselines, distributional comparison, experiments, and production data. From the Distributional archive. The product described has been sunset: read the pivot post, then the Talaria Scientific manifesto.
- [The AI Software Development Lifecycle: A practical framework for modern AI systems](https://distributional.com/blog/the-ai-software-development-lifecycle-a-practical-framework-for-modern-ai-systems.md) (January 9, 2025, by Erin LeDell, archive): The AI SDLC framework: how the development lifecycle morphs across traditional software, ML, and GenAI, with risk rising as determinism falls. From the Distributional archive. The product described has been sunset: read the pivot post, then the Talaria Scientific manifesto.
- [Why testing is an enterprise problem that requires an enterprise solution](https://distributional.com/blog/why-testing-is-an-enterprise-problem-that-requires-an-enterprise-solution.md) (October 31, 2024, by Nick Payton, archive): Part 3 of the enterprise AI testing series: why production-grade AI testing needs an enterprise platform, not developer-tool latitude. From the Distributional archive. The product described has been sunset: read the pivot post, then the Talaria Scientific manifesto.
- [Why all ML and AI use cases need standardized testing](https://distributional.com/blog/why-all-ml-and-ai-use-cases-need-standardized-testing.md) (October 24, 2024, by Nick Payton, archive): Part 2 of the enterprise AI testing series: the testing gaps in classical ML workflows, from process visibility to false-alarm suppression. From the Distributional archive. The product described has been sunset: read the pivot post, then the Talaria Scientific manifesto.
- [Why generative AI creates unique testing challenges](https://distributional.com/blog/why-generative-ai-creates-unique-testing-challenges.md) (October 18, 2024, by Nick Payton, archive): Field notes from 1,000+ hours with Fortune 500 AI leaders on why GenAI breaks traditional testing: non-determinism, non-stationarity, and behavior definition. From the Distributional archive. The product described has been sunset: read the pivot post, then the Talaria Scientific manifesto.
- [We raised $11M for better AI testing](https://distributional.com/blog/we-raised-11m-for-better-ai-testing.md) (December 14, 2023, archive): Scott Clark's 2023 founding essay: a decade of AI testing problems through Yelp, SigOpt, and Intel, and the thesis behind Distributional. From the Distributional archive. The product described has been sunset: read the pivot post, then the Talaria Scientific manifesto.

---

Source: https://distributional.com/blog (Distributional)
