> ## Documentation Index
> Fetch the complete documentation index at: https://docs.pushnotif.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# The holdout

> Why a slice of your users never gets a notification, and what that tells you.

A fixed share of every app's users, 10% by default, is in the **holdout**.
They receive no notifications from PushNotifAI, from flows or broadcasts.
That is permanent, and it's on purpose.

## Why

The usual push dashboard says "notifications drove 40,000 opens." Most of those
people would have opened the app anyway. The notification just got the
credit. Some programs even make things worse: notification opens go up while
people open the app on their own less, and the attribution numbers look great
the whole time.

Comparing the users who got notifications with the users who didn't is the only
way to see whether your notifications **cause** more use, or just claim it.

## How users are assigned

Each user is placed by a stable hash of the user id you pass to
[`identify`](https://docs.pushnotif.ai/sdk/identifying-users). The same user is always in the same
group: reinstalling, switching devices, or changing your flows never moves
anyone in or out.

## What it costs, and what to expect

The holdout gives up notifications to 10% of users. In return you get the
only trustworthy answer to "is this working?".

With modest traffic, small improvements take a long time to show up. Early
on, the holdout's main job is catching **harm**: a program that quietly
reduces how often people come back. The dashboard and the Composer both say
"not yet significant" rather than draw a line that isn't there.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.