Cross-Category Calibration Analysis of Kalshi Mention Markets

Prediction markets promise a very clean deal: the price is supposed to be the probability. If a contract trades at 70 cents, it should resolve YES about 70% of the time (Wolfers & Zitzewitz, 2006). When that relationship breaks, the interesting question is not just whether the market is wrong. It is how it is wrong, where it is wrong, and whether the error is random noise or something more structural.

Recent work by Bürgi, Deng, and Whelan (2025) shows that Kalshi exhibits a favorite–longshot bias at the exchange level. But aggregate results only get you so far. Mention markets are a much better laboratory for looking at calibration because they are repetitive, short-lived, and unusually legible: the same types of events recur, the contract wording is clean, and outcomes can be checked quickly.

This article looks at 20,979 resolved mention contracts across 128 series in seven categories: Politicians, Sports, Earnings, Entertainment, Media/News, Finance, and a residual Other group. Using volume-weighted average prices (VWAPs) measured at several lookbacks from the final recorded trade, I examine how well prices track realized outcomes and how sensitive that answer is to the lookback. The main finding is that low-priced YES contracts resolve less often than their price implies in every category. The original claim that every category converges toward fair prices as the event approaches does not hold up once calibration is measured correctly.

The Dataset

The dataset is drawn from Kalshi, a CFTC-regulated prediction exchange. It combines trade-level records with contract metadata across 141 mention-series files.

Every count below belongs to one snapshot of that data, and each one has a different denominator:

• 38,450 raw markets, of which 36,214 are resolved.
• 21,975 resolved markets with at least one recorded trade.
• 20,979 contracts with a valid 2-hour VWAP. This is the analysis sample.
• 15.5 million trade positions (both sides of each trade).

The analysis sample covers contracts that closed between January 30, 2025 and July 5, 2026. The original version of this article reported 18,948 contracts and 7,320 political contracts from an older snapshot. Those numbers are now historical. Most of the difference comes from how contracts are assigned to categories: Kalshi metadata stores many Trump contracts under series values A and B, so mapping categories from the series field silently dropped them. Categories are now assigned by ticker prefix.

Each data source contributes different fields:

• Trade data: individual trade records with timestamps, prices in cents (1–99), trade counts, and taker-side identification.
• Market data: contract metadata including ticker, series, title, result, close time, volume, and open interest.

Table 1: Category structure of the corrected snapshot (contracts with a valid 2-hour VWAP). Series counts are distinct series values in the metadata.

Politicians account for 49% of the analysis sample, and Sports adds another 26%. Together, those two categories make up about three-quarters of it. The Other group is a gap in the category map rather than a meaningful category, so it is reported but not interpreted.

The Last-Trade Gap

Each contract carries two timestamps that are easy to confuse: close_time, the scheduled end of trading, and last_trade_time, the time of the final recorded trade. Payout is a third, separate event. The original version of this article called close_time the settlement time; that was wrong.

Trading often stops well before the scheduled close. Across the 20,749 mention contracts with a nonnegative gap:

• Median gap: 56 minutes
• 75th percentile: 173 minutes (2.9 hours)
• Maximum: 19,617 minutes (13.6 days)
• 48.4% of contracts have a gap greater than 1 hour
• 20.4% exceed 4 hours

This matters because a VWAP anchored to close_time can end up reflecting trades from long before the market stopped moving. Every VWAP in this article is anchored to the final recorded trade instead. That answers the question "what had the market been paying before trading stopped?" It is not the price before the event started, and it is not a settlement-time price.

For payout timing, a separate Trump-only extract from October 2, 2026 records a settlement timestamp. In that extract, settlement follows the scheduled close by a median of 30.2 minutes, with 17.4% of contracts over one hour.

Methodology

For each resolved contract, I compute VWAPs at eight lookbacks from the final recorded trade: 7 days, 3 days, 1 day, 6 hours, 2 hours, 1 hour, 30 minutes, and 10 minutes. The 2-hour VWAP is the main snapshot throughout the analysis.

Each VWAP is cumulative: it averages every trade at or before the cutoff, not just the trades inside the preceding window.

Prices are in cents, so a VWAP of 70 corresponds to a 70% market-implied YES probability.

Calibration is measured by sorting contracts into probability buckets based on VWAP and comparing each bucket's realized YES rate with the mean implied probability of the contracts in that bucket. The bucket midpoint is only a label. Prices are not spread evenly inside a bucket, so comparing against the midpoint, as the original version did, adds error that has nothing to do with the market. A perfectly calibrated market would sit on the 45-degree diagonal.

To summarize calibration quality in a single number, I use Mean Absolute Deviation (MAD):

where $\overline{p}_b$ is the mean implied probability in bucket $b$. MAD is expressed in percentage points and is computed across all buckets with at least five contracts. Lower values indicate better calibration.

Kalshi's taker fee schedule is built into the strategy benchmarks:

where p is the contract price in cents. Fees are highest near the middle of the probability range and decline toward the extremes.

To check whether the calibration errors are large relative to fees, I compute two simple threshold benchmarks on the 2-hour VWAP:

• Buy YES: purchase YES contracts when the 2-hour VWAP exceeds a given threshold.
• Buy NO: purchase NO contracts when the 2-hour VWAP falls below a given threshold.

Thresholds are tested at 10-point intervals from 10% through 90%. Net PnL per contract is total net PnL after Kalshi taker fees divided by total eligible contracts. These benchmarks assume a fill at the historical VWAP, so they are an accounting check, not a trading simulation.

Category-Level Calibration

The summary statistics show meaningful variation across categories:

Directional accuracy, which predicts YES when the 2-hour VWAP is above 50 and NO otherwise, exceeds 82% in every category. Entertainment leads at 92.6%, but that partly reflects its low base rate: only 32.3% of entertainment contracts resolve YES, so always predicting NO would already be right two-thirds of the time.

MAD tells a different story. Politicians have the lowest corrected MAD at 8.6 percentage points. Entertainment has the highest at 16.5 despite the best directional accuracy, which is what you would expect if its low-priced contracts are systematically overpriced rather than simply called wrong.

Table 2: Category-level summary statistics using the 2-hour VWAP in the corrected snapshot.

These are within-snapshot comparisons. They do not, on their own, show that one category will be better calibrated than another in a new period.

Calibration by Lookback

The original version of this article said that every category improves as the event approaches. The corrected charts show something different. The next set compares calibration curves at six lookbacks, from one day before the final recorded trade to ten minutes before it, for the four largest categories.

1 / 4

In every category, the curves get steeper at shorter lookbacks. Contracts priced below the middle resolve YES even less often than their price, and contracts priced above the middle resolve YES even more often. The 1-day and 6-hour curves sit closest to the diagonal.

Collapsing those curves into MAD makes the pattern explicit:

Politicians rise from about 6.5 points at 1 day to about 16.6 at 10 minutes, and Sports from about 5.8 to about 18.2. Finance dips at 6 hours before rising, and Entertainment peaks at 1 hour. There is no category where MAD declines monotonically from 1 day to 10 minutes.

That does not mean late prices are worse forecasts. Each lookback covers a different set of contracts: many contracts have no trades a day before their final trade, so the 1-day sample is smaller and different. The cumulative VWAP also includes every earlier trade. A fair test of whether prices improve near the event would compare the same contracts across lookbacks, with event-clustered uncertainty. Without that, the safe reading is that calibration error is sensitive to the lookback, and a single lookback should be fixed in advance. This article uses 2 hours.

Calibration Over Time

A second question is whether each category has become better calibrated over the months of its existence. To look at that, I compute cumulative MAD at weekly intervals for the four main categories.

1 / 4

Each category's cumulative MAD ends well below where it started. That is consistent with markets learning over time (Brown, Reade, & Vaughan Williams, 2019), but these charts cannot establish it. Each weekly point includes every earlier trade position, both sides of each trade are counted, and the event mix changes over time. Long flat stretches, as in Sports and Entertainment, mean little new data entered the sample, and Earnings drifts back up after its early decline. These are descriptive snapshots, not independent evidence.

Strategy Benchmarks

In-sample, the NO side looks stronger than the YES side across the corrected snapshot.

Buy YES turns positive above the 50% threshold and peaks at the 70% level, with net PnL of +5.1 cents per contract across 7,313 contracts and a 92.4% hit rate.

Buy NO is positive at every tested threshold and strongest at the <50% threshold: +12.4 cents per contract across 10,158 contracts, with a 90.2% hit rate. That is about 2.4 times the best YES result and is consistent with the low-side pattern in the calibration curves.

These numbers describe the past, not a strategy. The thresholds were evaluated on the same data that revealed the pattern, and the fill price is a retrospective VWAP. The original version averaged category results while showing summed trade counts; the figures above are pooled correctly as total PnL over total eligible contracts.

A stricter check exists only for Trump markets. Splitting Trump events chronologically at June 10, 2026, a fixed NO-below-40% rule earns +2.67 cents per contract before the split and +2.21 after it, and a YES-above-70% rule loses money in both periods. The NO rule survives the split, but at a fraction of its in-sample size.

VWAP Price Trajectories

The plots below show each contract's cumulative VWAP at each lookback, from seven days before the final recorded trade to ten minutes before it. Green traces resolve YES; red traces resolve NO.

1 / 4

These charts are a diagnostic, not a timeline. Contracts have different amounts of trading history, so far fewer lines reach back to 7 days, and lookbacks are measured from each contract's own final trade rather than a common event clock. They show YES and NO outcomes separating as prices move toward 0 or 100. They do not measure how fast information arrives; the calibration tables carry the result.

Category Differences

The in-sample NO-side benchmark is largest in Sports, Entertainment, and Earnings, each about 16 cents per contract at the <50% threshold, and smallest in Politicians at 9.8 cents. That ordering matches the MAD ranking: Politicians are the best calibrated category in this snapshot. Media/News is the only category where buying YES loses money at every threshold.

None of this ranks categories for deployment. A large in-sample gap is not attractive if liquidity is thin, fills are hard, or the result is concentrated in a few events. Those are exactly the things this analysis does not measure.

Limitations

The whole exercise is in-sample. The same data is used to describe the calibration errors and to evaluate the threshold rules built on them. The one chronological check, in Trump markets, shrinks the NO-side result substantially and turns the YES-side result negative.

Results are snapshot-specific. The counts, MAD values, and benchmarks in this version differ from the original because the data and the category mapping changed. In Trump markets, a refreshed audit also failed to replicate a favorite-side gap that an older snapshot showed. Each number should be read alongside the snapshot it came from.

The raw contract count overstates the amount of independent information. A single earnings call, sports broadcast, or political speech can generate a cluster of related mention contracts that move and resolve together. The category curves and benchmarks here do not account for that clustering, so their precision is overstated.

Finally, execution. The benchmarks assume fills at a cumulative historical VWAP, which is not a quote a trader could have acted on. They leave out spread, slippage, queue position, available size, and event-level drawdowns, and the sample excludes cancelled, delisted, and inactive contracts.

Conclusion

Mention markets are informative but not evenly calibrated. The 2-hour price picks the right side more than 82% of the time in every category, and Politicians, the largest category, have the lowest calibration error. The most consistent error is on the low side: low-priced YES contracts resolve less often than their price implies across every category.

Two parts of the original story did not survive the audit. Calibration does not simply improve as the event approaches; measured correctly, error rises at shorter lookbacks, and that comparison is confounded by changing samples. And the large in-sample NO-side benchmarks shrink sharply in the one chronological test available.

The defensible takeaway is descriptive. The same contract design produces different calibration profiles depending on the information environment around it, and in this snapshot the low-side mispricing is large enough to be visible after fees. Whether it is a durable, tradable edge is still an open question that needs a frozen holdout, event-level uncertainty, and a realistic fill model to answer.

Works Cited

1. Arrow, K.J., Forsythe, R., Gorham, M., Hahn, R., Hanson, R., et al. (2008). "The Promise of Prediction Markets." Science, 320(5878), 877–878. https://doi.org/10.1126/science.1157679
2. Barbosa, L.F. (2026). mention-analysis [Computer software]. GitHub. https://github.com/LuizFelipeBarbosa/mention-analysis
3. Brown, A., Reade, J.J., & Vaughan Williams, L. (2019). "When Are Prediction Market Prices Most Informative?" International Journal of Forecasting, 35(1), 420–428. https://doi.org/10.1016/j.ijforecast.2018.07.012
4. Bürgi, C., Deng, W., & Whelan, K. (2025). "Makers and Takers: The Economics of the Kalshi Prediction Market." CEPR Discussion Paper No. 20631; CESifo Working Paper No. 12122. https://ssrn.com/abstract=5502658
5. Kahneman, D. & Tversky, A. (1979). "Prospect Theory: An Analysis of Decision under Risk." Econometrica, 47(2), 263–291. https://doi.org/10.2307/1914185
6. Kalshi. (2026). "Fee Schedule." https://kalshi.com/docs/kalshi-fee-schedule.pdf
7. Kalshi Help Center. "How is Kalshi regulated?" https://help.kalshi.com/en/articles/13823765-how-is-kalshi-regulated
8. Kalshi Help Center. "Limit Orders." https://help.kalshi.com/en/articles/13823811-limit-orders
9. Page, L. & Clemen, R.T. (2013). "Do Prediction Markets Produce Well-Calibrated Probability Forecasts?" Economic Journal, 123(568), 491–513. https://doi.org/10.1111/j.1468-0297.2012.02561.x
10. Snowberg, E. & Wolfers, J. (2010). "Explaining the Favorite–Long Shot Bias: Is It Risk-Love or Misperceptions?" Journal of Political Economy, 118(4), 723–746. https://doi.org/10.1086/655844
11. Snowberg, E., Wolfers, J., & Zitzewitz, E. (2013). "Prediction Markets for Economic Forecasting." In Handbook of Economic Forecasting, Vol. 2, 657–687. Elsevier.
12. Whelan, K. (2024). "Risk Aversion and Favourite–Longshot Bias in a Competitive Fixed-Odds Betting Market." Economica, 91(361), 188–209.
13. Wolfers, J. & Zitzewitz, E. (2004). "Prediction Markets." Journal of Economic Perspectives, 18(2), 107–126. https://doi.org/10.1257/0895330041371321
14. Wolfers, J. & Zitzewitz, E. (2006). "Interpreting Prediction Market Prices as Probabilities." NBER Working Paper No. 12200.