BestWorld's Carolyn Meinel and Dawna Coutant participated in IARPA's FOCUS research.


.:What Does FOCUS Research Mean for You?


The objective of the FOCUS program was to enable "counterfactual reasoning," defined as "statements about what would have happened if different circumstances had occurred. For example, a postmortem review of an analysis failure may lead to a conclusion that analysts would have avoided the failure if they employed better analytic tradecraft; perhaps by having double checked assumptions, perhaps by having considered a broader range of hypotheses, etc. Counterfactual forecasts about what would have worked in past circumstances are very often the basis for lessons learned for what to do in the future. And such lessons often evolve, over time, into best practices and tradecraft."

But how can we test reasoning about intelligence failures? What about the U.S. evacuation from Afghanistan? Right now, there is a rush to assign blame on the Biden administration by Republican politicians, and to cast blame on past Republican Presidents by Democrat politicians. Who is right? The problem is that we can't rerun history to find out whether President Trump, if he had been reelected, would have done a better job.


However, in the FOCUS program we did rerun history -- that is, histories of bots competing against bots in computer games. Then we analyzed how well we counterfactual forecasters (including BestWorld's Dawna Coutant and Carolyn Meinel) were at predicting what would happen in these alternate histories by comparing our counterfactual reasoning with what actually happened when those evil geniuses who created these bots at the John Hopkins Advanced Physics Lab reran history.

Wow! Thanks to the training team at Good Judgment 2.0, led by the same people who inspired us to launch BestWorld, we did really well! Next, IARPA hired some Amazon Mechanical Turks -- people who make a living participating in psychology research projects -- and our training team taught them, too, how to be super good at counterfactual reasoning.



Top ten COvid-20 forecastersFOCUSGood Judgment 2.0 team logo

GJP2.0 COVID-19 Forecasting Tournament Top 10 Best Final Forecasts


About Us


© Carolyn Meinel 2022, all rights reserved. Webmaster meinel [at] bestworld.io


    August 2021: BestWorld's Dawna Coutant and Carolyn Meinel combated COVID-19 as participants in IARPA's FOCUS program. We did better than the big modeling teams for the first six months of the pandemic. Then our forecasts made on Aug. 8, 2020 continued for nearly a year without updating to be about even with the models.But then the Delta variant hit. Wow, what a surprise this was to most of us. See what we had forecasted for 2021 as of August 8, 2020 at Covid_forecasts_2021.html


    May 2020: ISIT's Carolyn Meinel explains how she used Natural Language Processing to forecast geopolitical questions in a webinar hosted by Basis Technology: Mirror, Mirror on the Wall -- which Country Will Rule them All?


    January 2020: The team of ISIT and KaDSci, LLC came in 4th place in IARPA's Geopolitical Forecasting Competition out of over 100 initial competitors by forecasting 305 questions within just 5 1/2 months. We also were the only team to use hybrid forecasting, and the only team to publish our results. More details here.