How to choose a Monte Carlo sample count

This guide shows how to choose the iteration count of a montecarlo block or the samples argument of a simulation function. No fixed count guarantees accuracy. Choose a candidate count from the statistic being estimated, then accept it only after comparing the actual result across larger runs.

Step 1: Identify the estimate that matters

For a mean, every trajectory contributes whether the script uses an avg() accumulator or takes the mean of an empirical ranvar. The required count depends on the variance of the simulated quantity.

For a quantile or tail probability, identify the most extreme part of the distribution used downstream. An upper quantile q has an expected fraction 1 - q of trajectories beyond it. A lower quantile has an expected fraction q below it.

If the script turns a ranvar into an economic decision, use the most extreme part of the distribution that can still change that decision. A smooth-looking distribution is not, by itself, evidence that this part is estimated reliably.

Step 2: Size a candidate count for a mean

For independent trajectories with standard deviation $\sigma$, the standard error of a simulated mean based on n trajectories is:

$$ \operatorname{SE}(\bar X) = \frac{\sigma}{\sqrt n} $$

For an approximate confidence interval with target half-width $\epsilon$, use:

$$ n \ge \left(\frac{z\sigma}{\epsilon}\right)^2 $$

Run a pilot simulation to estimate $\sigma$. For an approximate 95% confidence interval, use $z = 1.96$. If the tolerance is relative, set $\epsilon$ to the accepted fraction of the pilot mean’s magnitude; use an absolute tolerance when the mean can be close to zero.

This calculation provides a candidate count, not a guarantee. A high-variance or heavy-tailed result can require far more trajectories, and its pilot variance can itself be unstable. Confirm the mean across larger repeated runs.

Step 3: Screen tail coverage

For an upper quantile q, compute the expected number of tail paths as:

$$ \text{tail paths} = n(1-q) $$

where n is the number of trajectories. As a screening rule, 25 expected tail paths provide an initial candidate and 100 provide a larger comparison count. These counts do not certify the accuracy of a tail estimate and do not apply to mean estimation.

This rule assumes independent trajectories and that the relevant uncertainty is redrawn inside every iteration.

Equivalently, for a target number m of upper-tail paths:

$$ n \ge \frac{m}{1-q} $$

For a fixed tail probability, 25 expected tail paths correspond to about 20% relative sampling standard deviation and 100 correspond to about 10%. This does not measure the numerical error of a quantile. Quantile error also depends on the distribution’s shape near the quantile and on any point masses.

The same reasoning applies to lower quantiles. P5 is statistically as demanding as P95, although a boundary such as zero or a large point mass can make P5 appear more stable.

Illustration: Compare three sample counts

The following script starts from a negative binomial ranvar whose quantiles can be calculated directly. It then approximates the same distribution through Monte Carlo blocks containing 100, 500, and 2,500 trajectories.

model = negativeBinomial(20, 4)

montecarlo 100 with
  sample empirical100 = ranvar(random.ranvar(model))

montecarlo 500 with
  sample empirical500 = ranvar(random.ranvar(model))

montecarlo 2500 with
  sample empirical2500 = ranvar(random.ranvar(model))

table Results = with
  [| as Samples |]
  [| 100 |]
  [| 500 |]
  [| 2500 |]

Results.ExpectedBeyondP95 = Results.Samples * (1 - 0.95)
Results.ExpectedBeyondP99 = Results.Samples * (1 - 0.99)
Results.EmpiricalP95 = if Results.Samples == 100 then quantile(empirical100, 0.95)
                       else if Results.Samples == 500 then quantile(empirical500, 0.95)
                       else quantile(empirical2500, 0.95)
Results.EmpiricalP99 = if Results.Samples == 100 then quantile(empirical100, 0.99)
                       else if Results.Samples == 500 then quantile(empirical500, 0.99)
                       else quantile(empirical2500, 0.99)

show summary "Model reference" with
  quantile(model, 0.95) as "P95"
  quantile(model, 0.99) as "P99"

show table "Empirical tail estimates" with
  Results.Samples
  Results.ExpectedBeyondP95 as "Expected paths beyond P95"
  Results.EmpiricalP95 as "Empirical P95"
  Results.ExpectedBeyondP99 as "Expected paths beyond P99"
  Results.EmpiricalP99 as "Empirical P99"

Run the script and compare each empirical quantile with the model reference. The tail-path columns report 5, 25, and 125 expected paths beyond P95, but only 1, 5, and 25 beyond P99. The empirical estimates do not necessarily approach the reference monotonically: a small sample can land close by chance. In particular, an apparently accurate P99 based on 100 trajectories is still informed by only about one expected tail path.

Step 4: Choose a tail candidate

Use the table only to select a candidate for the most extreme tail statistic that matters:

• Use 100 trajectories to exercise the script, not to claim tail accuracy.
• Try 500 as an initial P95 count, but not as a P99 count.
• Try 2,500 as an initial P99 count or a larger P95 comparison.
• Include 10,000 in the comparison when P99 behavior can change the decision.

Counts larger than these may still be necessary for irregular or heavy-tailed distributions. Conversely, a smaller count may be sufficient when a decision is insensitive to the remaining sampling variation. For a mean, derive the candidate from the variance instead of this table.

Step 5: Validate convergence on the result

Before applying an expensive count to a full dataset:

1. Select representative items, including high-volume, intermittent, and high-value cases.
2. Run the calculation at the candidate count and at one or more larger counts.
3. Compare the statistic actually consumed downstream, such as P95, an order quantity, an action ranking, or an economic return.
4. Repeat every count with several seeds or repeated executions; one comparison can agree by chance.
5. Define a business tolerance and keep the lowest count whose output remains within that tolerance across the repeated and larger runs.

Increasing the trajectory count fivefold generally reduces ordinary Monte Carlo sampling error by about the square root of five, not by a factor of five. Use the convergence test to decide whether the additional compute is valuable.

Step 6: Keep smoothing separate from sampling

Applying smooth() to an empirical ranvar regularizes its shape. It can make a chart easier to read, but it does not create observations in an unobserved tail and does not justify using a more extreme quantile. Select the trajectory count from the unsmoothed simulation and the downstream decision.

Apply the rule to action reward

The actionrwd functions use 2,500 trajectories by default and accept at most 10,000. The default represents about 125 expected paths beyond P95 and 25 beyond P99. These figures describe tail coverage, not the accuracy of the expected return, action ranking, or replenishment decision.

Keep the default unless repeated comparisons support another count. If P99 behavior can change a replenishment decision, include 10,000 trajectories in the comparison on a representative subset. At the 10,000 limit, only 10 paths are expected beyond P99.9, which is insufficient for a reliable distribution-free P99.9 claim.

See montecarlo for the block syntax and actionrwd.reward for the action-reward parameters.