Boston, May 1921. The lobby of the Copley Plaza Hotel. Albert Einstein, on his first visit to the United States, is surrounded by reporters. One of them hands him a sheet of paper that is circulating across the country: Thomas Edison’s hiring questionnaire. The question that catches his eye is translated into German for him: “What is the speed of sound?”
Einstein answers, in substance, that he does not keep that kind of information in his head, since it can be found in any textbook. On May 18, the New York Times reports that Einstein has failed the Edison test (Quote Investigator).
The most famous physicist of his time, flunked by a general knowledge quiz. Keep this scene in mind, we will come back to it. It contains almost everything that is wrong with the way great bosses have long hired people.
A bowl of soup, a salt shaker and zero evidence
Let’s start with a story you have probably come across on LinkedIn. Edison invites a candidate to lunch and orders soup. If the candidate salts it before tasting it, the interview is over: this man makes decisions without data.
The moral is appealing. It has one flaw: nothing proves the scene ever happened. Menlo Innovations looked into the anecdote and concluded that no facts support it and that it is probably an urban legend (Menlo Innovations). The Thomas Edison Center points out that Snopes also classifies it as a legend (Thomas Edison Center). The same story sometimes circulates with Henry Ford as the hero. The name changes, the missing source stays the same.
Enjoy the irony. A story meant to prove that you should taste before you salt has been shared thousands of times without anyone tasting it. The people who share it do exactly what the candidate in the legend does: they season before they check.
This is confirmation bias in its purest form, the tendency to retain information that supports what we already believe and to ignore what contradicts it. The anecdote flatters an idea we love, the genius who reads a personality at a glance, so we never question it (My book, chapter 6).
Edison’s real test: 146 questions and 32 survivors
The questionnaire, on the other hand, really existed. In 1921, Edison put candidates for management positions through a written exam of more than 140 questions, which varied by position. A few covered technical or business knowledge. Most tested memory for facts: the name of a famous violin maker, the state whose capital is Helena (Saturday Evening Post).
The outcome left no room for doubt. Of 718 applicants, 57 reached a score of 70%, and only 32 cleared the 90% bar Edison demanded (Mental Floss). Do the math: more than 95% failed at the target level. Theodore Edison, his own son, then a student at MIT, failed too (Saturday Evening Post).
Edison held one conviction: for him, intelligence boiled down to curiosity and memory (History.com). He distrusted college degrees, having himself received only a few months of formal schooling (Saturday Evening Post).
Then the questionnaire leaked. A rejected applicant, Charles Hansen, reconstructed the questions from memory and handed them to the New York Times. He called it a silly examination (Mental Floss). Once the questions were in print, Edison had to rewrite his test (History.com). The controversy filled the newspapers for weeks, and that is the context in which Boston reporters handed the sheet to Einstein.
Einstein was right, and AI proves him right a second time
Back to the Copley Plaza lobby. Einstein does not know the speed of sound by heart. He knows how to calculate it, he knows where to find it, and he understands wave propagation better than anyone in the room. Edison’s test still files him among the failures.
Edison measured what was easy to measure: the amount of facts stored in a head. Einstein defended what was hard to measure: the ability to reason with facts you go and look up when you need them.
In 1921, the debate was philosophical. In 2026, technology has settled it. An AI assistant answers the entire Edison questionnaire in a few seconds. A recruiter who selected on factual memory today would be hiring people in direct competition with a free tool.
So let me ask you an uncomfortable question. In your own hiring process, how many steps still assess something an AI does better than your candidates? Rate yourself from 0 to 5. I am not sure my own recruiting reflexes, shaped at Sagem, Apple and then Sony, would have earned a 5 a few years ago.
Xero’s coffee cup: a culture test in one gesture
Let’s change eras. Trent Innes led Xero for Australia and Asia for eight years (Startup Daily). His method, described in 2019 on The Venture podcast, made the rounds of the international press. At the start of the interview, he takes the candidate to the company kitchen and offers a drink. At the end, he watches for a single gesture: does the candidate offer to take the empty cup back?
For Innes, skills and experience can be gained with time and work, while attitude does not change. He wants people who take ownership of what needs doing on their own, in a culture where everyone washes their own cup (Directo al Paladar). According to him, only 5 to 10% of candidates fail, and what he assesses is the intention to offer, even when the candidate is not allowed to walk all the way to the sink (Click Petróleo e Gás).
I see the value of the test: it observes real behavior rather than a stated intention. But ask yourself two questions. Does a candidate whose career is on the line in the next hour really think about the cup? Does a candidate from a culture where, out of politeness, you never touch your host’s dishes come across as a poor team player?
The test may measure team spirit. It certainly measures cultural closeness to the recruiter. That is the territory of in-group bias, the preference for people who went through the same training and share the same codes (My book, chapter 7).
Musk’s riddle: the path matters more than the answer
Third boss. According to Ashlee Vance’s biography, Elon Musk personally interviewed nearly all of SpaceX’s first thousand employees, technicians and janitors included (CNBC). He liked to ask this riddle: you are standing on the surface of the Earth, you walk one mile south, one mile west and one mile north, and you end up exactly where you started. Where are you?
Most candidates eventually find the North Pole. Musk then follows up: where else? The second answer lies near the South Pole, one mile north of the circle of latitude that is exactly one mile around. Few engineers give it (CNBC). According to Vance, Musk cares most about how the engineer explains the reasoning (InsideHook).
Of the three bosses, Musk is the closest to Einstein. He watches someone reason out loud, a skill no database replaces.
The riddle still has a problem every recruiter knows: it gets around. Once printed in a bestseller and picked up by the press, it mostly measures who reads the press. Edison went through exactly the same misadventure in 1921.
What these tests really measure
Step back from these stories. They share one feature: a single signal, observed once, interpreted by the person who invented the test. Each leader projects his own values onto it. Edison valued memory, and his own was exceptional. Innes values the returned cup, because that is the culture he built.
Add the halo effect, which extends a good first impression to everything else about a person. A candidate who performs the expected gesture in the first five minutes will be seen as more competent for the rest of the interview, even on unrelated topics (My book).
These tests mostly flatter the person doing the hiring. They walk away feeling they have uncovered a secret the résumé was hiding. The feeling is pleasant, and it proves nothing about the candidate’s future performance.
What hiring science says
Work psychology has studied the validity of selection methods for decades. In 1998, Frank Schmidt and John Hunter published a meta-analysis summarizing 85 years of research on 19 selection methods (Plum). In their table, work sample tests had the highest validity, at 0.54 (Criteria).
In 2021, a re-analysis using more cautious statistical corrections revised these figures downward. Structured interviews moved to the top, with an estimated validity of 0.42. Their predictive power is more than double that of unstructured interviews, with nearly a third less bias (Criteria).
A structured interview asks every candidate the same questions, derived from a job analysis, and scores the answers on a grid set in advance. The exact opposite of the soup, the cup and the riddle asked on gut feeling.
Hire the way you innovate: experiment, measure
When I joined Sony Europe to launch the VAIO PCs, we were based in Londerzeel, in Flanders. HR had published the first job ads in Flemish. They only attracted Flemish candidates, while my team worked in French and English. I asked for the ads to run only in English, and beyond Flanders. In three months, we hired forty people from twenty-five nationalities. That team outperformed equivalent teams in Asia and North America by two to four times (My book, chapter 13).
The lever sat upstream of any interview: an assumption about the language of the ads that nobody had ever questioned. Assumptions are highly dangerous and should be killed with experimentation as early as possible.
So treat your hiring like an innovation project. Run it with indicators borrowed from growth hacking: the cost of acquiring a candidate, the cost of an interviewed candidate, the cost of a signed contract, employee retention and employee referrals (My book). If you are attached to your coffee cup test, measure it. Do the candidates who passed it perform better a year later? Do they stay longer? Without that measurement, you hold an anecdote. As far as I know, Edison never published how his 32 winners performed.
Finally, add the astonishment report. One week, one month and three months after they arrive, ask new hires what surprised them, for better or worse, including in the application and selection process. Your new hires are the only people who have lived your tests from the inside. Listen to them.
The essentials
Remember four things:
- The salt test attributed to Edison, or sometimes to Ford, rests on no source. The 1921 questionnaire, on the other hand, is documented: more than 140 questions, 32 passes out of 718 applicants.
- A test of factual memory selects profiles in direct competition with any AI. Einstein understood this as early as 1921.
- A gesture observed once and interpreted by the test’s inventor mostly measures resemblance to the recruiter.
- Recent re-analyses rank the structured interview first among predictive methods. Measure your own tests before you believe in them.
Hiring legends survive because they are pleasant to tell. Your next hire deserves better than a salt shaker. Taste before you salt: test your tests.
A corporate event, a seminar, an executive committee meeting, or a management committee meeting?
Philippe’s keynotes on innovation and AI are groundbreaking, you’ve been warned!
Complement a powerful keynote with innovative and impactful workshops.
A keynote inspires and raises awareness; workshops transform!
References
- (My book) https://philippeboulanger.com/book/
- (Quote Investigator) https://quoteinvestigator.com/2016/05/28/not-facts/
- (Menlo Innovations) https://menloinnovations.com/stories/menlo-bits/march-2022-menlo-bits
- (Thomas Edison Center) https://www.tomedison.org/blog/unveiling-edisons-favorite-fuels
- (Saturday Evening Post) https://saturdayeveningpost.com/2026/06/could-you-pass-thomas-edisons-job-application-test/
- (Mental Floss) https://www.mentalfloss.com/history/thomas-edisons-impossible-employment-test
- (History.com) https://www.history.com/articles/thomas-edison-test-job-interview
- (Startup Daily) https://www.startupdaily.net/topic/asx/xero-changes-the-guard-with-md-trent-innes-stepping-down-for-sales-director-joseph-lyons/
- (Directo al Paladar) https://www.directoalpaladar.com/actualidad-1/prueba-taza-cafe-que-puede-determinar-tu-futuro-entrevista-trabajo
- (Click Petróleo e Gás) https://en.clickpetroleoegas.com.br/chefe-revela-o-curioso-teste-da-xicara-de-cafe-usado-para-julgar-candidatos-durante-entrevistas-de-emprego-e-descartam-aqueles-que-nao-passam/
- (CNBC) https://www.cnbc.com/2018/10/09/why-spacex-ceo-elon-musk-asks-this-tricky-interview-question.html
- (InsideHook) https://www.insidehook.com/advice/elon-musk-job-interview-riddle
- (Plum) https://www.plum.io/blog/schmidt-hunter-meta-analysis
- (Criteria) https://www.criteriacorp.com/blog/updates-employee-selection-science







