5 Things Product Managers Should Be Testing (But Probably Aren't)
A/B testing and user research are standard practices in product-led organizations. Yet most product teams are testing a small fraction of what they could — and would benefit from — testing. The tests that typically get run focus on conversion optimization, UI variations, and obvious product features. The tests that most teams don’t run include some of the highest-value opportunities available.
These five testing areas are systematically neglected not because product managers don’t believe they matter but because they fall into the gap between obviously important and obviously how to test them.
1. Onboarding Experience Variations
Most product teams have a default onboarding flow that was built during initial product development and has been incrementally modified since. Few systematically test meaningfully different onboarding approaches — fundamentally different sequences, different mental models introduced, different levels of hand-holding.
The activation rate — the percentage of new users who reach their first meaningful value moment — is one of the most consequential metrics for long-term retention and LTV. Improvements in onboarding are force multipliers that amplify the value of every other product investment. Yet most teams test cosmetic onboarding variations (button color, copy changes) rather than fundamental approach variations that could produce order-of-magnitude improvements.
2. Pricing and Packaging
Product teams rarely own pricing in isolation — finance and leadership are typically involved — but PMs have much more influence over pricing experiment design than they typically exercise. Whether a feature is included in the base tier or restricted to higher tiers changes the product’s commercial dynamics significantly; the product team’s understanding of how users value different features is directly relevant to pricing architecture.
Testing packaging variations — which features are bundled, what the upgrade triggers look like, how freemium limitations are designed — produces insights that simultaneously improve commercial performance and user experience.
3. Feature Removal or Simplification
Most product teams add features; few systematically test removing or simplifying them. Yet many products carry significant legacy functionality that adds complexity without creating proportionate value — features that were important two years ago but are now used by a small fraction of users.
Testing removal or significant simplification of low-adoption features typically reveals one of two things: either users don’t miss them (and the product becomes simpler and more focused), or unexpected users depend on them in ways that weren’t visible in adoption data (which is genuinely valuable information).
4. Different User Segments’ Responses
Most A/B tests are run on the full user population. Few segment results by user type, acquisition source, company size, or use pattern — even when the product serves meaningfully different user segments that might respond very differently to the same change.
A feature that performs neutrally in aggregate might create significant improvement for enterprise users while slightly harming SMB users. Testing results that don’t examine segment differences systematically miss insights that could substantially improve decision quality.
5. Assumptions Behind the Product Strategy
The most high-stakes testing opportunity that most product teams neglect is testing the strategic assumptions themselves: whether the target customer segment values what the strategy assumes they value, whether the proposed differentiation is actually meaningful to buyers, whether the assumed adoption patterns materialize.
Strategy testing doesn’t require large engineering investments — it requires user research and lightweight prototyping that can validate or invalidate strategic assumptions before significant development resources are committed.
Key Takeaways
The five neglected testing areas — onboarding variation, pricing and packaging, feature simplification, segment segmentation, and strategic assumption testing — represent some of the highest-value opportunities available to product teams. Adding them to the testing roadmap alongside conversion optimization and UI tests produces the more comprehensive learning that makes product decisions progressively better-calibrated.
The Systemic Benefits of Validation Discipline
Beyond the individual product decision improvements that validation produces, the discipline of not treating customer requests as specifications creates a healthier organizational dynamic: customers who understand that their requests will be investigated rather than implemented develop more informative relationships with the product team. Instead of escalating feature requests, they share problems. Instead of advocating for their specific solution, they describe their workflows. This shift produces better customer partnerships and better product intelligence simultaneously.
When to Accept Customer Requests Directly
The argument against treating customers as always right doesn’t mean product teams should disregard customer requests. Some requests are specific enough, well-validated enough, and common enough across the user base that validation adds less value than it costs time. The pragmatic PM distinguishes between requests that are self-evidently validated (clear problem, common occurrence, proposed solution is straightforward) and those that aren’t — investing validation effort proportional to the uncertainty in the request.
The long-term effect of consistent validation discipline is a product that serves the full user base effectively rather than the vocal minority specifically. Products built for the full population — grounded in patterns across many users rather than stories from a few — create the broad adoption that makes commercial outcomes sustainable.