Iliad

Prerequisites

Cluster 00 · Prerequisites

The prerequisites for the Iliad Intensive program.

We put a star (★) and boldface on content that we think is particularly important to understand.

We are aware that our participants have different backgrounds, that this is a lot of material, and that it may not be feasible to prepare all of it!

Background worldview and assumptions

The references on background worldview and assumptions are very informative to understand the motivation behind the course. They are less important for understanding its technical content, however.

Note that this section is on the speculative side: Working on AI alignment is important precisely because of assumptions and arguments about the future of AI. We can’t know the future of AI, and so all of this is inherently uncertain.

Why AI matters

Here, we simply argue that AI should concern us now at all, irrespective of any worldview on whether the outcomes are likely to be good or bad. Essentially, the claim is that the impact of AI might be enormous, potentially pretty soon.

AI misalignment

Having established that the impact of AI might soon be enormous, we now specifically turn to the risks. We start by discussing AI misalignment.

One operationalization of AI misalignment is the concern that AI systems may not do what their developers want them to do, with potentially catastrophic outcomes for very advanced AI systems.

Non-misalignment AI safety concerns

We now briefly discuss a spectrum of safety concerns that manifest even if we know how to steer AI systems effectively toward a given set of goals.

  • Individual people may misuse AI in catastrophic ways An Overview of Catastrophic AI Risks:
    • ★ Sections 2.1-2.3 argues for catastrophic misuse capabilities like bioterrorism, unleashing AI agents, and persuasive AIs. Misuse risk is particularly relevant to our course since it can also manifest as a misalignment concern: An AI that assists human users to carry out risks is often misaligned with the AI’s developer.
  • AI can give rise to global totalitarianism
    • Section 2.4 argues for the potential of a concentration of power, leading to global totalitarianism in the worst case.
  • We may get gradually disempowered even if there is alignment

→ You may also find it useful to read the risk decomposition from the international report on safe AI

Agent Foundations Background

In the Iliad Intensive, we will also have sections on agent foundations, where we discuss AI from a more "idealized" perspective, taking intelligence or rationality or optimization processes to a theoretical limit to analyze consequences. Additionally, this viewpoint also attempts to more formally talk about what agents or goals are, in a descriptive and mathematical way.

Useful readings:

Technical prerequisites

Many of the resources recommended in this section are taken verbatim from ARENA’s prerequisites, with thanks.

Engineering

  • ★ Bring your laptop: Some days involve coding.
  • ★ Get comfortable with the following. Most relevant:

    • Python. It's important to be strong in Python, because this is the language that the coding dayes of the program use. As a rough indication, we expect you to be comfortable with at least 80-90% of the material here, up to section "21. for/else". For a more thorough treatment of Python's core functionality, see here.

    • NumPy. Being familiar with NumPy is a staple for working with high-performance Python. Additionally, the syntax for working with NumPy arrays is very similar to how you work with PyTorch tensors (often there are only minor differences, e.g. Torch tends to use the keyword dim where NumPy uses axis). Working through these 100 basic NumPy exercises would be a good idea, or if you're comfortable with NumPy already then you could try doing them in PyTorch (see below).

    • PyTorch. PyTorch has several useful tutorials, and to get comfortable working with tensors you might want to implement the 100 basic NumPy exercises linked to above, using PyTorch instead. Another option would be this Learn the Basics tutorial. You can skip this if you can clearly explain:

      • At a high level, what is a torch.Tensor?
      • What is a nn.Parameter, and nn.Module?
      • When you call .backward(), where are your gradients stored?
      • What is a loss function? In general, what does it take for arguments, and what does it return?
      • What does an optimization algorithm do?
      • What is a hyperparameter, and how does it differ from a regular parameter?
      • What are some examples of hyperparameters?

    • Basic coding skills. If you've been accepted into this programme, then you probably already have this box ticked! However, polishing this area can't hurt. LeetCode is a good place to keep basic coding skills sharp, in particular practising the planning and implementation of functions in the medium-hard sections of LeetCode might be helpful. Practising problems on Project Euler is another idea.

    • Einops and Einsum for basic tensor operations. These are great libraries to get comfortable with, when manipulating tensors. If you're comfortable using them, then you can say goodbye to awkward NumPy/PyTorch methods like transpose, permute and squeeze!

      For einops, you can read through the examples up to "Fancy examples in random order". It's worth trying to play around with these in your own Jupyter notebook, to get more comfortable with them.

      For einsum, this page provides a basic intro to einstein summation convention, and shows some example tensor implementations. Note that we'll be using the einsum function from the einops library, which allows you to refer to dimensions by name rather than by a single letter.

  • Have access to an LLM that can help you, ideally on a paid plan. For coding specifically, Claude via Claude Code and GPT via Codex are popular choices.

Deep Learning

Linear Algebra

★ Make sure you understand all of the following:

  • Vectors, matrices, rank, null spaces, rank-nullity theorem, orthogonality, invertibility;
  • Positive definite, eigenvalues, spectral decomposition;
  • Singular values, Singular value decomposition (SVD).

→ Linear algebra lies at the core of a lot of machine learning. Resources for these topics:

This video series by 3B1B provides a good overview of these core topics (although you can probably skip it if you already have a reasonably strong mathematical background).

If you have a lot more time, Linear Algebra Done Right is the canonical textbook for covering this topic (although it will probably cover much more than you need to know). Alternatively, Neel Nanda has two YouTube videos covering linear algebra extensively.

Calculus

★ Get comfortable with all of the following:

  • Computing basic limits, derivatives, and integrals;
  • Partial and directional derivatives, gradients, Jacobians, and the chain rule in multiple dimensions;
  • Hessian, second-order Taylor expansion and remainder;
  • Integration: Multivariate integrals, volume in , change of variables;
  • Understand -notation and -notation.

→ It's essential to understand differentiation and partial differentiation, and helpful to understand the basics of vector calculus including the chain rule and Taylor series. Again, 3Blue1Brown has a good video series on this.

For one module, the implicit function theorem will be relevant.

Probability & Statistics

★ Understand all of the following:

  • Basic probability theory, notation for conditional probabilities and joint probabilities (or densities), Bayes rule, probability simplex;
  • expectation, variance, moments, Independence, the law of large numbers;
  • multivariate normal distributions.

→ ★ It's essential to understand the rules of probability, expected value and standard deviation, and helpful to understand independence and the normal distribution. This link covers some of the essentials.

The following concepts are also useful to know:

  • Bayesian statistics: The concept of the Likelihood, posterior distribution, partition function, and Bayesian free energy, see here, Chapter 1.
  • Bayesian networks
  • Causality – a Brief Introduction
  • Markov chains, row-stochastic matrices, hidden Markov models (HMMs)
  • Measure theory

Information theory

  • ★ Gain Intuitive understanding of entropy, mutual information, Kullback-Leibler (KL) divergence, and cross-entropy. It's helpful to understand information, entropy and KL divergence. These play key roles in interpreting loss functions.
    • For an overview of Kullback Leibler divergence (an important concept in information theory and machine learning), see Six (and a half) intuitions for KL divergence. Note that this probably won't make sense if you don't already have a solid grasp of what entropy is.

Furthermore, it may be useful to understand the following concepts from information theory:

  • Lossless compression:
    • Uniquely decodable codes
    • Shannon-Fano code
    • Shannon’s source coding theorem
  • Communication over noisy channels
    • Channel capacity
    • Channel coding theorem
  • Lossy compression: Rate-distortion theory

→ Elements of Information Theory by Cover and Thomas introduces all of these concepts.

Theoretical computer science

A classical source that covers most of the following topics is Sipser’s Introduction to the theory of computation. For some topics like Solomonoff induction we link separate texts.

  • Computability Theory
    • ★ Turing machines
    • ★ Church-Turing thesis: All algorithms can be represented with a Turing machine. That is, Turing machines are a universal model of computation.
      • This is used to avoid constructing Turing machines explicitly: Whenever we can describe an algorithm, we can simply claim the existence of a corresponding Turing machine.
    • ★ Kolmogorov complexity, also called descriptive complexity in Sipser’s book.
    • ★ An Intuitive Explanation of Solomonoff Induction
      • You should understand that Solomonoff induction is a universal learning algorithm that considers all computable hypotheses and weighs them by a simplicity prior. Also, it is optimal in some technical sense as long as the true universe is computable, too.
      • For a more technical and precise introduction, see here.
    • Non-deterministic Turing machines
  • Computational Complexity Theory
    • Basic complexity classes
      • P, NP, PSpace
    • Reduction. In Sipser’s book, this can be understood by reading:
      • Chapter 5.3: Mapping reducibility
      • Chapter 7.4 on NP-completeness discusses polynomial-time reductions

Formal logic is not covered sufficiently in Sipser’s book. Instead, look at:

Miscellaneous

Statistical mechanics: For some sections on physics-inspired deep learning theory and natural abstractions it can be helpful to have a basic understanding of statistical mechanics.