Prediction Markets for Paper Evaluations

using the online market interface


A short explanatory video available via the screencast tab page provides all relevant information on how to submit beliefs about REF ratings at the market interface. Detailed information are summarized below for your reference.

You have an initial allocation of 100 tokens – you may invest this however you wish across the various papers, in any decimal amounts (e.g., 5.74 tokens in one paper, 20 tokens in another, etc.) You can see relevant information about each paper at the 'papers' tab in the upper righthand corner of this website, refpredictionmarkets.com

The price of shares will always be between 0 and 1 (e.g., .600 or .374). The listed price indicates the cost of one positive/YES share (i.e., a share betting that the paper will be highly rated). The cost of a negative/NO share is the opposite: 1 minus the listed price (e.g., if the listed price is .600, a NO share costs .400 tokens.)

Remember the market price also represents the crowd’s collective forecast of how likely the event is to occur.

For example, for a paper with the price of .750, the collective wisdom believes the paper has a 75% chance of getting a 3* or above rating. The best way to play the market is to buy YES shares (increase your position) when you think the market price/forecast is too low, and buy NO shares (decrease your position) when you think the market price/forecast is too high.

The number of shares you hold in a particular paper when the market closes will determine the payoff you receive for that paper. This number can be positive or negative. Positive shares indicate a ‘YES’ bet, and negative shares indicate a ‘NO’ bet.

  • Payoff for positive holdings: you will receive a payoff of 1 Token per share if the paper gets at least 3* rating in the mock REF. There will be no payoff if it gets 2* or less.
  • Payoff for negative holdings: you will receive a payoff of 1 Token per share if the paper gets a 2* or lower rating in the mock REF. There will be no payoff if it gets 3* or more.

Once you have identified a paper that you would like to trade on, click anywhere on the paper name to access the trading pop-up box. Then you will see two possibilities to trade:

  • “Increase position by # Tokens”: if you believe that the paper is more likely to receive a high (3*+) rating than the market price suggests, you can invest here. Please enter how many tokens you would like to invest in this position, and press OK. If you use this option to reduce a negative position, tokens will be credited to your account. If you increase a positive position, tokens will be deducted.
  • “Decrease position by # Tokens”: if you believe that the paper is less likely to receive a high (3*+) rating than the market price suggests, you can invest here. Please enter how many tokens you would like to invest in this position, and press OK. If you use this option to reduce a positive position, tokens will be credited to your account. If you enter a negative position, tokens will be deducted.

Example:

You expect a paper to receive a 3* REF rating, so you increase your position by 10 Tokens at an initial price of 0.50, moving the price to 0.55 and buying 19.09 shares. Other traders later agree with you and invest another 20 Tokens moving the price further up to 0.63. The value of your initially invested 10 Tokens therefore increases to 11.59 Tokens.

You will only receive a payout for the shares you hold. Any Tokens not invested into shares when the market closes will be voided. Therefore it is advisable to invest all of your tokens before the market closes – in fact, it is best to invest all of your tokens as soon as possible. The sooner you buy shares, the more likely you are to get a good deal before others make their trades.

What is the best strategy for you to play the market?

In short, the best way to play the market is to focus on the papers for which you think the market price is off – either too high or too low. (Remember, the market price corresponds to the collective forecast of the probability that the paper will receive 3* or higher in the REF). When your own beliefs about a paper diverge from the market, that is your chance to make a profit. In those cases, you can buy shares cheaply, for less than you think they are actually worth. This works the same in either direction, whether you think the market has undervalued or overvalued the paper.

For instance, imagine a paper that you are 90% sure will get at least a 3* rating. If this paper’s market price was consistent with your beliefs, it would cost .90. But if the price is .60, you can buy YES shares at a discount of .30 token per share (.60 instead of .90). Similarly, imagine a paper you think is only 10% likely to get a 3* or above rating. If this paper’s market price was consistent with your beliefs, its price would be .10, and therefore buying NO shares would cost you .90 token per share. (Remember, NO shares always cost 1-p, where p is the price). But instead, if the market price were .40, you could buy NO shares much more cheaply, at .60 per share (1-.40), rather than .90 per share.

In short, if you think the market is too optimistic about a paper (i.e., price is too high), then buy NO shares (decrease your position). If you think the market is too pessimistic (i.e., price is too low), buy YES shares (increase your position). Paradoxically, this may mean you buy YES shares in a paper you think has a low chance of receiving a 3* rating, as long as the market thinks it has an even lower chance!

Every share you buy moves the price closer in the direction of your prediction – each NO share you buy moves the price closer to 0, and each YES share you buy moves the price closer to 1. But as a quick calculation, it’s still useful to think of your expected earnings as being the absolute difference between the market price and your judgement of the paper’s probability, times the number of shares you buy.

Remember, any tokens you leave uninvested at the end of the market will be lost – your winnings will only be determined by the shares you hold. Therefore, make sure to invest ALL of your tokens before the market closes!

For further advice on how to play prediction markets, see this excellent blog by Joe Hilgard: how to play prediction market?

When should I increase or decrease my position in a paper?

There are many papers in the market, so you will have to decide which trades to invest your resources in. You can trade on as many of the papers you want, as long as you have enough free tokens. If you need more tokens, you can–and should – sell some of your existing shares. Just as you would with normal trading, look for papers whose market price is off – specifically, papers in which you hold positive/YES shares and the market price seems too high, or papers in which you hold negative/NO shares and the market price seems too low.

Anonymity

Your trades in the market will be completely anonymous to the other traders, and your email address will only be used to allocate payoffs (i.e., prizes) by our collaborators who run the market interface. Once payments have been made, email addresses will be deleted from the raw dataset and only anonymized ID codes will be retained, meaning that your individual trades will not be identifiable, even to the researchers.