Back to work in progress

Quantitative human-data collection

This is not really about AI. It is about doing a better job of respecting what people give us when we ask them what they think, in market research and well beyond it.

Choose a category

The same problem appears wherever we ask people what they think: market research, political research, public policy, and public health. We have leaned on surveys for decades, online surveys still rely on methods designed for paper, and confidence in them has rarely been lower. The aim is not to replace people with AI, but to use AI as a spine for generating and structuring hypotheses, then go to people for what only they can provide.

One honest response to the panel problem is not a perfect, representative panel, but a categorisable one. Recruit people from the communities you care about, group them with a small set of screening questions, report results by group rather than pretending they generalise to everyone, and pay participants directly for their time (Baker et al., 2013).

Then ask in a way people actually engage with. Instead of a long grid, present a quick choice between two options: a simple thumb flick, with the selected option taking over the frame. Neither side is inherently better, and this is not a pass or fail task. The strength of the choice is reflected in response speed; faster decisions indicate stronger preference, drawing on implicit association work (Greenwald, McGhee, & Schwartz, 1998). A down-swipe option allows “too close to call”, ensuring people are never forced into a false choice.

Underneath, interaction itself becomes a check on authenticity. Because a person must physically swipe, behaviour patterns are harder for bots to replicate, even as automated responses increasingly pass standard survey checks (Westwood, 2025). Data collection and fraud detection become part of the same mechanism. This remains experimental rather than finished: response sides and colours are randomised each round, and the full battery can be reversed to test for position bias before drawing conclusions.

Which do you prefer?1 of 10
swipe down for
too close to call

Swipe through the pairs and the picture builds below: who you preferred, how strongly, and who you ruled out.

Your results so far

Prototype only. The data is your own taps, held in your browser and sent nowhere. Works with a mouse or a touchscreen.

References

  1. Baker, R., Brick, J. M., Bates, N. A., Battaglia, M., Couper, M. P., Dever, J. A., Gile, K. J., & Tourangeau, R. (2013). Summary report of the AAPOR task force on non-probability sampling. Journal of Survey Statistics and Methodology, 1(2), 90–143. https://doi.org/10.1093/jssam/smt008
  2. Greenwald, A. G., McGhee, D. E., & Schwartz, J. L. K. (1998). Measuring individual differences in implicit cognition: The implicit association test. Journal of Personality and Social Psychology, 74(6), 1464–1480. https://doi.org/10.1037/0022-3514.74.6.1464
  3. Westwood, S. J. (2025). The potential existential threat of large language models to online survey research. Proceedings of the National Academy of Sciences, 122(47), e2518075122. https://doi.org/10.1073/pnas.2518075122
← Human data collection, done better Back to work in progress →