Are Personalized Ads a Waste of Money?个性化广告是浪费钱吗？

Several major tech companies have recently built platforms that claim to educate companies about how best to market themselves and their products online. Examples include Meta for Business （formerly Facebook for Business; “Get step-by-step guidance， industry insights and tools to track your progress， all in one place”）， Think with Google （“Take your marketing further with Google”）， and Twitter for Business （“Grow your business with Twitter ads”）.

几家大型科技公司最近都创建了自己的平台，声称将教企业如何最有效地在线推广自己和自己的产品，比如Meta for Business（前身是Facebook for Business；“提供一站式服务，让您获得循序渐进的指引、行业视野和跟踪进展的工具”），Think with Google（“让谷歌助您进一步开拓市场”），以及Twitter for Business（“推特广告助您业务攀升”）。

These sites are very appealing. They offer a variety of advertising tools and services designed to help those companies boost their performance.

这些网站极具吸引力，它们提供各种广告工具和服务以帮助企业迅速提升业绩。

All of these sites have the same basic goal. They want you to invest your marketing dollars in them.

所有这些网站都有共同的基本目标：赚取你的市场推广费。

Not as simple as it looks

并非看起来那么简单

In recent weeks， Facebook has been broadcasting ads that tell all sorts of inspiring stories about the small businesses that it has helped with its new services. My Jolie Candle， a French candlemaker， “find[s] up to 80% of their European customers through Facebook platforms.” Chicatella， a Slovenian cosmetics company， “attributes up to 80% of their sales to Facebook’s apps and services.” Mami Poppins， a German baby-gear supplier， “uses Facebook ads to drive up to half of their revenue.”

最近几周，脸书公司一直在播放广告，讲述各种励志故事，宣传它的新服务如何帮助小企业提升了业绩。广告中称，法国蜡烛生产商My Jolie Candle有“高达80%的欧洲客户来源于脸书公司的平台”；斯洛文尼亚化妆品公司Chicatella将“高达80%的销售额归功于脸书的应用程序及服务”；德国婴幼儿用品供应商Mami Poppins “一半的收入得益于脸书公司的广告”。

That sounds impressive， but should businesses really expect such large effects from advertising？ The fact is， when Big Tech companies “educate” small businesses about their services， they often are actually encouraging incorrect conclusions about the causal effects of advertising.

听起来振奋人心，但企业真能指望广告带来如此巨大的效益吗？事实上，当大型科技公司向小企业“宣传”他们的服务时，他们通常是在鼓励人们得出广告因果效应的错误结论。

Consider the case of a consulting client of ours， a European consumer goods company that for many years has positioned its brand around sustainability. The company wanted to explore if an online ad that makes a claim about convenience might actually be more effective than one that makes a claim about sustainability. With the help of Facebook for Business， it ran an A/B test1 of the two ads and then compared the return on advertising spend between the two conditions. The return， the test found， was much higher for the sustainability ad. Which means that’s what the company should invest in， right？

以我们的一个咨询客户为例，这是一家欧洲消费品公司，多年来其品牌定位在可持续性方面。该公司想知道一条在线广告如果主打产品的便利性是否比主打可持续性更有效。在 Facebook for Business的帮助下，该公司对两种广告做了“A/B测试”，然后比较两种情况下的广告投入所获收益。测试发现，主打可持续性广告的收益要高得多。这意味着公司应该在这方面投资，对吧？

Actually， we don’t know.

实际上，我们不知道。

There’s a fundamental problem with what Facebook is doing here： The tests it is offering under the title “A/B” tests are actually not A/B tests at all. This is poorly understood， even by experienced digital marketers.

这里脸书的操作存在一个根本问题：它提供的所谓“A/B测试”根本不是A/B测试。这不太好理解，哪怕对于经验丰富的数字营销师来说也是如此。

So what’s really going on in these tests？ Here’s one example：

这些测试到底是怎么回事？下面来看一个例子：

1） Facebook splits a large audience into two groups—but not everybody in the groups will receive a treatment. That is， many people actually won’t ever see an ad.

1）脸书将大量用户分成两组——但不是组中的每个人都能收到测试项目，就是说，很多人实际上根本看不到广告。

2） Facebook starts selecting people from each group， and it provides a different treatment depending on the group a person was sampled from. For example， a person selected from Group 1 will receive a blue ad， and a person selected from Group 2 will receive a red ad.

2）脸书开始在每组中选人，并根据组别给选中的人发送不同的测试项目。比如，来自1组的人将收到蓝色广告，而来自2组的人将收到红色广告。

3） Facebook then uses machine-learning algorithms to refine its selection strategy. The algorithm might learn， say， that younger people are more likely to click on the red ad， so it will then start serving that ad more to young people.

3）随后脸书用机器学习算法完善其选择策略。算法可能会发现，比如说，年轻人更喜欢点击红色广告，于是它开始将红色广告更多地推送给年轻人。

Do you see what’s happening here？ The machine-learning algorithm that Facebook uses to optimize ad delivery actually invalidates the design of the A/B test.

所以你发现了什么？脸书用来优化广告投放的机器学习算法实际上让A/B测试的设计失效了。

Here’s what we mean. A/B tests are built on the idea of random assignment. But are the assignments made in Step 3 above random？ No. And that has important implications. If you compare the treated people from Group 1 with the treated people from Group 2， you’ll no longer be able to draw conclusions about the causal effect of the treatment， because the treated people from Group 1 now differ from the treated people from Group 2 on more dimensions than just the treatment. The treated people from Group 2 who were served the red ad， for example， would end up being younger than the treated people from Group 1 who were served the blue ad. Whatever this test is， it’s not an A/B test.

这就是我们想表达的。A/B测试的设计基于随机分配理念。但上述第三步的分配是随机的吗？不是。而且它产生了重大影响。此时比较1组和2组中收到测试项目的人，你无法再得出关于测试项目因果效应的结论，因为1组的人和2 组的人现在不仅仅是在收到的测试项目上有所不同。比如，最终2组中收到红色广告的人要比1组中收到蓝色广告的人年纪小。不管这是什么测试，反正不是A/B测试。