Why are we running a prediction market? Prediction markets may be able to help universities decide which papers to submit for the REF, by giving crowd-sourced ratings of these papers.
Current methods for deciding which papers to submit are often time-intensive and subjective. Prediction markets may aid this process by gathering the wisdom of the crowd – and they are especially good for nuanced forecasting, as they allow participants to take into account the collective predictions of everyone else (reflected in the price of the shares for each paper).
NB: Our prediction market will not be used as a replacement for REF decisions – keep in mind this is simply a research project to determine whether markets can provide useful information to REF coordinators.
How do prediction markets work? A prediction market is a bit like the stock market, except instead of investing in companies, you invest in the outcomes of future events. You can either buy ‘YES’ shares that will only pay off if an event occurs, or conversely, you can buy ‘NO’ shares that will only pay off if the event does not occur. As more people bet that the event will occur (by buying YES shares), the price of YES shares rises, and the price of NO shares falls. The opposite is true when people buy NO shares.
In our prediction market, the future events you’re betting on are REF ratings: you’ll buy shares to forecast whether or not particular papers will receive high ratings (a strong 3* or above) in a mock REF assessment. A strong 3* rating refers to papers in the top half of the distribution of all 3* papers.
Your aim is to receive the highest payoff possible, by betting correctly on papers and buying shares for cheaper than you think they’re worth.
REF ratings. The REF is a process of expert review, which the UK government uses to allocate research funding to universities based on the quality of their research outputs. It operates within 34 Units of Assesment (UoAs), i.e., research fields. Chemistry is one of these UoAs.
Papers can be rated from 0-4 stars in the REF, with 4* being the highest rating. REF panels assess the quality of submitted research outputs in terms of their ‘originality, significance and rigour’, with reference to international research quality standards. See below for the criteria for awarding stars:
| Four star | Quality that is world-leading in terms of originality, significance and rigour. |
| Three star | Quality that is internationally excellent in terms of originality, significance and rigour but which falls short of the highest standards of excellence. |
| Two star | Quality that is recognised internationally in terms of originality, significance and rigour. |
| One star | Quality that is recognised nationally in terms of originality, significance and rigour. |
| Unclassified | Quality that is falls below the standard of nationally recognised work. Or work which does not meet the published definition of research for the purposes of this assessment. |