Boo! Shadow forecasts on SciCast

In time for Hallowe’en, we’ve added Shadow Forecasts and other features to help show the awesome power of combo.

  1. Shadowy: when linked forecasts affect a question, we show “Shadow Forecasts” in the history and the trend graph.legend
  2. Pointy: the trend graph now shows one point per forecast instead of just the nightly snapshot.
  3. Chatty: Comment while you forecast.  (But not while you drive.)

Read on for news about upcoming prizes.

Shadow Forecasts

The forecast history now has row entries for “nearby” trades that changed the chances of this question.  In the Testing screenshot below, the top two forecasts (most recent) are regular power-mode forecasts changing this question’s chances from 81% -> 1% and then from 1% to nearly 100%.  Nothing new there.

But the next two forecasts aren’t forecasts on this question at all.  They are mere shadows of forecasts on related questions Q.102 and Q.103.  But those questions are correlated with this one, and the trades were big enough to move our estimate a little bit: first from ~80% to ~79%, then from ~79% to ~81%.  The first (bottom) forecast is also a shadow forecast: @jkominek made some large forecast on Q.341 which moved our estimate from 18% to 13%.


There is a potential gotcha! here: the “Old Forecast” and “New Forecast” now always show the unconditional chances — the same chances tracked by the Trend graph. (Power Users: don’t worry, you will soon be able to toggle information overload.) Oops, not in this release. Old and new are still conditional when assumptions are made.

Trend Graph


The trend graph now shows a point for each forecast— including shadow forecasts.  For now it is still only updated nightly, but it has points for all the edits up until about midnight. (An upcoming change to the underlying widget will hopefully add today’s forecasts to the graph.)

Comment Your Forecast

You can now say a few words while you forecast.


Forecast comments show up in the forecast history:


They also show up in the Discussion tab, with a note showing it was a forecast comment:


Overly verbose comments may be trunc….


In early November, we will being a 4-month incentive study with prizes.  Look for the Au symbol and an associated leaderboard.  Stand by for details.

This entry was posted in Updates and tagged , , , , by ctwardy. Bookmark the permalink.

About ctwardy

2016 - : Senior Data Scientist: Jacobs (previously Sotera, then KeyW) 2015 - : Affiliate Professor, George Mason U. 2015-2016: Senior Data Scientist, Defense Suicide Prevention Office (via NTVI Federal) 2008-2015: Research Assistant Professor at George Mason University. Areas: Judgment & Decision-making; Machine learning; Statistical inference; Philosophy of Science; Collective Intellligence; Computational Philosophy.

10 thoughts on “Boo! Shadow forecasts on SciCast

      • I also prefer the unsmoothed plots. Even if they’re messy, they are better at showing the full range of trading prices.

        That said, I could see myself being convinced otherwise.

      • We’ll keep the unsmoothed line available. Might make it lighter and add a bolder smoothed trend. A bit more work would start that zoomed-out so each day is just a vertical bar showing the range. Zooming in would reveal the unsmoothed.

  1. Two comments: (1) I think it’s the case that when someone edits question A which links to multi-outcome question B, the new style “Trends and History” table for B shows the effect of the edit on *each* outcome of B. This is good. The same format for reporting the effects of @jkominek’s auto trades would be useful. (2) Since the update, the leaderboard has been frozen. Transient issue, or bug introduced by the patch?

  2. One more question. I noticed a HUGE jump in my available points recently. My only explanation for this is that the recent release fixed the issue of points locked up in outcomes that are already impossible (i.e. those which admin has zeroed and locked). Is that the case?

      • Huh. Surprising that (as we can see now that Brandon has restored the leaderboard-update cronjob) resolving these pending questions had such a big “PnL” effect, both on total score and available. It indicates to me that those forecasters who were ultimately correct on these questions were, by and large, very underconfident. Or perhaps most of these questions closed while the issues were actually still very uncertain? Looking forward to the next blog post on Brier scores and calibrations!

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s