Kolmogorov Probability Axioms

$P : \mathcal{F} \to \mathbb{R}$ is a probability measure if and only if it satisfies:

1. Non-negativity. $P(A) \geq 0$ for all $A \in \mathcal{F}$.
2. Normalization. $P(\Omega) = 1$.
3. Countable additivity. For any countable collection of pairwise disjoint events $A_1, A_2, \ldots \in \mathcal{F}$ (so $A_i \cap A_j = \emptyset$ for $i \neq j$), $P\!\left(\bigcup_{n=1}^\infty A_n\right) = \sum_{n=1}^\infty P(A_n).$

Axiom 1 rules out negative probability. Axiom 2 fixes the total mass at 1 (otherwise we'd be doing measure theory, not probability theory). Axiom 3 says probability behaves like a mass: putting probability on a countable disjoint union is the same as adding the masses on each piece. The choice of countable (not just finite) additivity is what gives probability its analytic strength: it forces continuity properties used in every limit theorem.

Every result in probability and statistics derives from these three axioms plus the structure of the chosen $(\Omega, \mathcal{F})$. The axioms are deliberately weak: they make no claim about how to assign probabilities to specific events, only about consistency requirements any such assignment must meet. This is what allows Bayesians, frequentists, and decision theorists to share a mathematical foundation while disagreeing on interpretation.

A function satisfying only finite additivity (sums for finite disjoint unions) is a finitely additive probability, not a (countably additive) probability measure. Finitely additive probabilities exist on any algebra, but they fail the continuity properties below and break the dominated convergence theorem. Real-valued probability theory uses countable additivity because the analytic payoff (limit theorems, Lebesgue integration of expectations) is enormous. The cost is that not every subset of an uncountable $\Omega$ is an event.

For any probability measure $P$ on $(\Omega, \mathcal{F})$:

1. $P(\emptyset) = 0$.
2. $P(\Omega) = 1$.
3. If $A \subseteq B$, then $P(A) \leq P(B)$.

The empty event carries no mass, the whole sample space carries all mass, and larger events cannot have smaller probability than their subsets. These are the first algebraic facts that let probability behave like a coherent size function on events.

This is the smallest reusable bridge from axioms to calculations. Union bounds, Borel-Cantelli, convergence theorems, and every probabilistic diagnostic later on assume these identities without restating them.

The statement is about events in the chosen sigma-algebra. If a subset of $\Omega$ is not measurable, $P$ is not required to assign it a probability.

If $A$ and $B$ are disjoint events in the same probability space, then

$P(A \cup B) = P(A) + P(B).$

Disjoint events occupy non-overlapping pieces of the sample space, so their probability masses add without correction terms.

Finite additivity is the step behind complement rules, inclusion-exclusion, and the two-event union bound. Most probability calculations start by splitting an event into disjoint pieces and adding their probabilities.

The disjointness assumption is material. If events overlap, adding $P(A)$ and $P(B)$ counts the intersection twice; inclusion-exclusion supplies the correction.

For pairwise disjoint events $A_0, A_1, \ldots$,

$P\!\left(\bigcup_{n=0}^{\infty} A_n\right) = \sum_{n=0}^{\infty} P(A_n).$

Countable additivity says probability mass has no hidden boundary charge at the limit of a countable disjoint decomposition. The mass of the countable union is exactly the infinite sum of the component masses.

This is the axiom that powers monotone convergence, Borel-Cantelli, laws of large numbers, and probability bounds over countable families of events.

Finite additivity alone does not imply this statement. Finitely additive probabilities can assign zero mass to each singleton in $\mathbb{N}$ while assigning mass one to the whole set, which violates the displayed countable sum identity.

For any event $A$ in a probability space,

$P(A^c) = 1 - P(A).$

An event and its complement split the sample space into two disjoint pieces. Since the whole sample space has mass one, whatever mass is not on $A$ is on $A^c$.

Complement conversion turns lower-tail bounds into upper-tail bounds and "at least one" events into "none" events. It is a small identity, but it sits inside Borel-Cantelli, hypothesis testing error control, and almost-sure arguments.

The statement is only about measurable events. For a non-measurable subset, neither $P(A)$ nor $P(A^c)$ is part of the probability space.

For any finite family of events $(A_i)_{i \in I}$,

$P\!\left(\bigcup_{i \in I} A_i\right) \leq \sum_{i \in I} P(A_i).$

Adding individual probabilities counts overlaps more than once, so the sum is an upper bound on the probability that at least one event occurs.

This is the probability move behind finite hypothesis-class generalization: bound the failure probability for each hypothesis, then union-bound over the finite class.

The bound can be loose when events overlap heavily. It is a worst-case one-sided bound, not an independence calculation.

For events $A_0, A_1, \ldots$,

$P\!\left(\bigcup_{n=0}^{\infty} A_n\right) \leq \sum_{n=0}^{\infty} P(A_n).$

Countable additivity gives equality for disjoint events. If the events overlap, assigning each point to its first covering event turns the union into a disjoint subfamily whose total mass is no larger than the original sum.

Countable union bounds power first Borel-Cantelli, tail-event estimates, and "bad event at some time" arguments. They are the bridge from probability axioms to concentration and learning-theory sample-complexity bounds.

If the sum of probabilities is larger than 1, the inequality is still true but often uninformative. Later concentration results matter because they make the right side small enough to use.

For events $A_1, \ldots, A_n \in \mathcal{F}$,

$P\!\left(\bigcup_{i=1}^n A_i\right) = \sum_{k=1}^n (-1)^{k+1} \!\!\!\!\sum_{1 \leq i_1 < \cdots < i_k \leq n} \!\!\!\! P\!\left(A_{i_1} \cap \cdots \cap A_{i_k}\right).$

For $n = 2$: $P(A_1 \cup A_2) = P(A_1) + P(A_2) - P(A_1 \cap A_2)$. For $n = 3$: $P(A_1 \cup A_2 \cup A_3) = P(A_1) + P(A_2) + P(A_3) - P(A_1 \cap A_2) - P(A_1 \cap A_3) - P(A_2 \cap A_3) + P(A_1 \cap A_2 \cap A_3)$.

Adding $P(A_1) + P(A_2)$ double-counts the overlap, so subtract $P(A_1 \cap A_2)$. With three sets, the three pairwise overlaps subtract too much from the triple overlap, so add it back. The alternating sign pattern generalizes this bookkeeping to any finite $n$.

Inclusion-exclusion is the workhorse for computing union probabilities when you can compute intersections. It appears in derangement counts, the union bound (a one-term truncation), and inclusion-exclusion bounds in combinatorial probability. The Bonferroni inequalities are obtained by truncating the alternating sum after an even or odd number of terms, yielding lower or upper bounds.

The number of terms grows as $2^n - 1$. For large $n$, computing every intersection probability is infeasible, and inclusion-exclusion becomes a theoretical tool rather than a computational one. The union bound $P(\bigcup_i A_i) \leq \sum_i P(A_i)$ is the cheap one-sided alternative used throughout learning theory.

$P$ is countably additive (and hence a probability measure) if and only if both of the following hold:

Continuity from below. For any increasing sequence $A_1 \subseteq A_2 \subseteq \cdots$ with $A_n \in \mathcal{F}$,

$P\!\left(\bigcup_{n=1}^\infty A_n\right) = \lim_{n \to \infty} P(A_n).$

Continuity from above. For any decreasing sequence $B_1 \supseteq B_2 \supseteq \cdots$ with $B_n \in \mathcal{F}$,

$P\!\left(\bigcap_{n=1}^\infty B_n\right) = \lim_{n \to \infty} P(B_n).$

"Increasing union" means the events grow to fill out their limit; the probabilities should grow to fill out the limit's probability. Without continuity, a sequence of events could grow to include "more and more" of $\Omega$ while their probabilities stayed pinned below the union's probability, which would break every limit argument in probability.

This is what countable additivity buys you. Every "$P(\text{eventually}) = \lim P(A_n)$" argument, every interchange of limit and probability, every dominated convergence application for indicator functions, sits on this continuity. The Borel-Cantelli lemmas and the modes of convergence of random variables both rely on it.

Continuity from above requires the events to be decreasing and at least one of them to have finite measure. For probability measures this is automatic (everything has measure at most 1), but for general measures (like Lebesgue measure on $\mathbb{R}$) the assumption is necessary. Example: the sets $B_n = [n, \infty)$ decrease to $\emptyset$ but each has infinite Lebesgue measure.

If $A_0 \subseteq A_1 \subseteq \cdots$ is an increasing sequence of events, then the sequence $P(A_n)$ tends to the probability of the union:

$P(A_n) \to P\!\left(\bigcup_{n=0}^\infty A_n\right).$

As the events grow, no probability mass disappears at the limit. Countable additivity lets the measure see the countably many incremental shells $A_{n+1} \setminus A_n$.

This is the exact form used in stopping-time, convergence, and Borel-Cantelli arguments: replace a growing event by its finite prefixes, then pass to the limit.

The monotonicity assumption is material. For arbitrary events, $P(A_n)$ can oscillate while the union stays fixed, so there is no reason for the displayed limit to hold.

If $B_0 \supseteq B_1 \supseteq \cdots$ is a decreasing sequence of events, then the sequence $P(B_n)$ tends to the probability of the intersection:

$P(B_n) \to P\!\left(\bigcap_{n=0}^\infty B_n\right).$

Decreasing events shave away mass. In a probability space the starting mass is finite, so the mass left after countably many removals is the limit of the remaining masses.

This is the form behind "eventually always" events, tail intersections, and limit arguments where bad sets shrink as a parameter grows.

For general infinite measures, one finite-measure assumption is necessary. The sets $[n,\infty)$ in Lebesgue measure decrease to the empty set but all have infinite measure, so the limit statement fails in the extended-real sense.