No single signal is the whole truth

Synthetic systems research is one of the most distinctive new signals in the category, but it is not a universal answer. Different questions require different methods.

Some are best addressed with traditional surveys, used carefully where sampling and instrument quality are strong enough. Others require qualitative engagement with real people to capture meaning that cannot be reduced to structured responses. Some depend on observing how customers discuss a category directly, while others rely on behavioural traces such as search share or panel data.

The clearest example is people. We have spent them on compressing experience into numbers, the very task machines now do well, when their real value lies in the questions only they can answer. Freed from that role, they can focus on depth, which is a shift worth its own attention.

The position is methodological agnosticism at the signal level. Each signal is held to its own standard of evidence, used where it is appropriate, and interpreted in relation to others. This aligns with Hunt (1991)’s concept of critical pluralism, and recent work by Allen and McDonald (2025) suggesting organisations that adopt it outperform those that rely on a single method. As Baker et al. (2013) note, a sample is never representative in the abstract; it is only adequate for a specific claim about a specific population.

The implication is straightforward: design instruments for the specific claim being made, validate before trusting outputs, and treat no single reading as complete. Intelligence comes from synthesis across signals, producing decisions that are both faster and more grounded than single-method approaches typically allow.

The discipline underneath all of it

It rests on a distinction the field too often lets blur: reliability and validity are not the same thing. Reliability is consistency, the same input giving the same answer. Validity is whether you are measuring the real thing at all, or just an artefact of the instrument. AI is becoming very reliable, ask it the same question and it answers consistently, but that says nothing about whether the answer is true, or whether you are measuring the market or only how the market is talked about online. Consistent is not correct, and holding the two apart is most of the job.

This is the same discipline I started with, years before these tools existed. Design the instrument carefully. Ask more than once. Change the context on purpose, to separate what genuinely moves the answer from what is just noise. Validate, and only then act. And make the output clear enough that the person using it can stand behind it in a room. Request, validate, deliver.