Self-study: Deep Learning by Goodfellow et al.

Linear Algebra

Probability and Information Theory

• “Researchers have made compelling arguments for quantifying uncertainty using probability since at least the 1980s.”
• Three sources of uncertainty:
  • Inherent stochasticity (e. g. quantum mechanics, card shuffling)
  • Incomplete observability (e. g. Monty hall problem)
  • Incomplete modeling
• Probability of used as degree of belief

Random variables (3.2)

• notation:
  • random variable: plain typeface
  • values it can take with lowerscript letters
  • vector-valued: bold

Probability distribution (3.3)

• discrete variables: probability mass function (notation: usually \(P\))
• notation: \(x \sim P(x)\)

General statistics

Law of total probability

\(B_n\): partition of entire sample space \(\rightarrow \sum p(B_n) = 1\) \[ P(A) = \sum_n P(A \cap B_n) \]

Statistical independence

\[ p(A|B) = p(A) \]

Conditional probability

\[ P(A | B) = \frac{A \cap B}{P(B)} \]

Bayes` theorem

\[ \begin{align} \frac{P(A|B)}{P(B|A)} &= \frac{P(A)}{P(B)} \\ P(A|B) \cdot P(B) &= P(B|A) \cdot P(A) \end{align} \]

Look at file “_notation.tex” in the corresponding folder!