P-Value Calculator

**Two-tailed p-value (from z-score):** p = 2 × P(Z > |z|) = 2 × (1 - Φ(|z|)) Where Φ is the cumulative standard normal distribution function. **One-tailed p-values:** - **Right-tailed (positive z)**: p = P(Z > z) = 1 - Φ(z) - **Left-tailed (negative z)**: p = P(Z < z) = Φ(z) **Worked example: z = 1.96, two-tailed** p = 2 × P(Z > 1.96) = 2 × (1 - 0.975) = 2 × 0.025 = **0.05** This is exactly the conventional significance threshold. **Common z and p relationships:** | Z-score | Two-tailed p | One-tailed p | |---|---|---| | 0.0 | 1.000 | 0.500 | | 1.0 | 0.317 | 0.159 | | 1.5 | 0.134 | 0.067 | | 1.645 | 0.100 | 0.050 (α = 0.05, one-tail) | | 1.96 | 0.050 | 0.025 (α = 0.05, two-tail) | | 2.0 | 0.046 | 0.023 | | 2.576 | 0.010 | 0.005 (α = 0.01, two-tail) | | 3.0 | 0.003 | 0.001 | | 3.291 | 0.001 | 0.0005 (α = 0.001) | **Hypothesis testing framework:** 1. **State null hypothesis (H₀)**: usually "no effect" or "no difference." 2. **State alternative hypothesis (H₁)**: what you're testing for. 3. **Choose significance level (α)**: commonly 0.05. 4. **Calculate test statistic** (z, t, F, χ², etc.). 5. **Compute p-value** from test statistic. 6. **Compare to α**: if p < α, reject H₀; if p ≥ α, fail to reject H₀. 7. **Interpret in context**: significance, effect size, practical meaning. **Critical z-values for two-tailed α:** | Confidence level | α | Critical z | |---|---|---| | 90% | 0.10 | ±1.645 | | 95% | 0.05 | ±1.960 | | 99% | 0.01 | ±2.576 | | 99.9% | 0.001 | ±3.291 | **One-tailed vs two-tailed:** - **Two-tailed**: testing if effect is different (in either direction). - **One-tailed**: testing if effect is in a specific direction. - Two-tailed is more conservative (requires more extreme data). - One-tailed has more power but tests only one direction. When in doubt, use two-tailed. **Common misconceptions about p-value:** ❌ "P = 0.04 means 4% chance the null is true." ✓ P = 0.04 means: assuming the null is true, there's a 4% chance of seeing data as extreme as observed. ❌ "P > 0.05 means no effect exists." ✓ P > 0.05 means data isn't strong enough to reject null hypothesis. Effect may exist but not detected. ❌ "P < 0.05 means a real, important effect." ✓ Could be a trivial effect with large sample, or a true effect, or chance occurrence. ❌ "Lower p means larger effect." ✓ Effect size and p-value are separate. p depends on both effect size and sample size. **Effect size (separate from p-value):** Always report effect size alongside p-value: - **Cohen's d** (continuous): small=0.2, medium=0.5, large=0.8. - **Pearson's r** (correlation): small=0.1, medium=0.3, large=0.5. - **Odds ratio**: 1=no effect, 2=double the odds. **Significance thresholds:** - **0.05**: traditional, very common, increasingly criticized. - **0.01**: more conservative; medical and high-stakes research. - **0.001**: very conservative; replication thresholds. - **0.000001**: physics standards (5-sigma confidence). - **No fixed threshold**: emerging best practice is to report p with context. **P-value criticism:** The American Statistical Association statement (2016) on p-values: 1. P-values don't measure the probability of the null hypothesis being true. 2. Significance doesn't imply causation or practical importance. 3. Don't draw conclusions from p < threshold alone. 4. Always report effect sizes and confidence intervals. **Pre-registration and HARKing:** - **Pre-registration**: declare hypotheses before collecting data; prevents post-hoc p-hacking. - **HARKing** (Hypothesizing After Results are Known): bad practice; inflates Type I errors. - **P-hacking**: trying many tests until one significant; deflates true alpha. **Family-wise error rate:** When testing multiple hypotheses, individual p-values aren't sufficient. Use: - **Bonferroni correction**: α / k for k tests. - **Benjamini-Hochberg**: control false discovery rate. - **Family-wise error rate**: probability of at least one Type I error.