Forecasting Curriculum
Session 1 — Foundations of Forecasting (~2hrs)
Introduction
This session highlights the vital role of forecasting in decision-making. We explore key principles like uncertainty, collective intelligence, and calibration, aiming to provide you with a strong forecasting foundation.
Objectives
Highlight forecasting's impact on strategic and everyday decisions.
Apply the "10 commandments of superforecasting."
Grasp concepts like uncertainty, crowd wisdom, and calibration through exercises.
Content and Resources
Introduction to Forecasting:
› Video: "Superforecasting: The people that predict the future" by BBC Global (10 mins)
- Purpose: Introduces the Good Judgement Project, emphasizing forecasting as a learnable skill.
› Reading: "Superforecasting: The Art and Science of Prediction" by Philip E. Tetlock and Dan Gardner (Chapters 1-3, 45 mins)
- Purpose: Provides a comprehensive overview of forecasting's significance and the basics of subjective probability forecasting
Core Principles:
› Interactive Module: "Calibrate Your Judgement" by Open Philanthropy (20-30 mins)
- Purpose: Enhances probabilistic judgment skills through real-time feedback, aiming for better calibration in forecasts.
› Reading: "The 10 Commandments of Superforecasting" by Brandon Beckhardt (10 mins)
- Purpose: Summarizes essential guidelines for effective forecasting.
Practical Exercises:
› Forecasting Challenge: Metaculus Quarterly Cup (35 mins)
- Activity: Forecast three questions from the most recent Metaculus Quarterly Cup.
§ Solo Learners: Choose the first three questions that interest you and providing a brief rationale with each forecast.
§ Group Settings: The instructor selects three questions for the class and moderates an open discussion after 30 minutes of forecasting.
Session 2 — Advancing Your Forecasting Skills (~1.5hrs)
Introduction
Expanding on the first session, we delve into advanced forecasting methodologies and tools. This session explores prediction markets versus aggregators, incorporates case studies, and offers hands-on exercises.
Objectives
Differentiate between prediction markets and aggregators.
Implement advanced techniques and question decomposition for better forecasts.
Conduct pre- and post-mortem analysis to refine forecasting.
Content and Resources
Exploring Forecasting Platforms:
› Activity: Discover the unique features of prediction aggregators and markets by exploring at least two of each. (30 mins)
- Purpose: To understand the operational differences and offerings of prediction platforms.
§ Prediction Aggregators: Platforms like Metaculus, GJOpen, and INFER Public compile and aggregate probabilistic predictions from a community, creating a collective forecast.
§ Prediction Markets: Platforms such as Polymarket, Kalshi, and Manifold Markets operate on a buy-sell model, where the market price of contracts based on future events reflects the collective forecast.
§ Solo Learners: Visit at least one market and one aggregator, noting interesting findings for personal reflection. Set weekly reminders to follow interesting questions until closing and resolution.
§ Group Settings: Collectively explore selected platforms, discussing unique questions and features.
Forecasting Questions:
› Reading: "What Makes a Good Forecasting Question?" by Confido Institute (8 mins)
- Purpose: Identifies key elements that contribute to crafting effective forecasting questions.
› Reading: "Get better answers by asking better questions: Understanding strategic question decomposition" by Infer Pub (5 mins)
- Purpose: Improves forecasting accuracy by teaching how to break down complex forecasts into simpler questions.
Case Study and Group Exercise:
› Reading: "Modeling the End of Monkeypox" by Jared Lebowich (20 mins)
- Purpose: Offers insight into the methodology behind successful real-world forecasts.
› Forecasting Challenge: LLM Forecasting Assistant and Metaculus Conditional Cup (30 mins)
- Activity: Forecast on two new questions from the Metaculus Conditional Cup and update one prior question using the LLM Forecasting Assistant (from this paper).
§ Solo Learners: Independently select and work on forecasts, applying the LLM tool as needed.
§ Group Settings: Share and discuss forecasts, using group insights and the assistant to refine predictions.
Pre- and Post-Mortem Analysis:
› Activity: Apply the Pre-Mortem and Post-Mortem templates to forecast challenges, focusing on identifying potential biases and learning from outcomes. (20 mins)
- Purpose: To systematically evaluate forecasts, enhancing accuracy through reflective practices.
§ Solo Learners: Conduct analysis independently, using templates for structured reflection.
§ Group Settings: Collaboratively perform analysis, with volunteers sharing forecasts for group feedback.