Edison’s salt test is a legend. His 146 questions are not.

Boston, May 1921. The lobby of the Copley Plaza Hotel. Albert Einstein, on his first visit to the United States, is surrounded by reporters. One of them hands him a sheet of paper that is circulating across the country: Thomas Edison’s hiring questionnaire. The question that catches his eye is translated into German for him: “What is the speed of sound?”

Einstein answers, in substance, that he does not keep that kind of information in his head, since it can be found in any textbook. On May 18, the New York Times reports that Einstein has failed the Edison test (Quote Investigator).

The most famous physicist of his time, flunked by a general knowledge quiz. Keep this scene in mind, we will come back to it. It contains almost everything that is wrong with the way great bosses have long hired people.

A bowl of soup, a salt shaker and zero evidence

Let’s start with a story you have probably come across on LinkedIn. Edison invites a candidate to lunch and orders soup. If the candidate salts it before tasting it, the interview is over: this man makes decisions without data.

The moral is appealing. It has one flaw: nothing proves the scene ever happened. Menlo Innovations looked into the anecdote and concluded that no facts support it and that it is probably an urban legend (Menlo Innovations). The Thomas Edison Center points out that Snopes also classifies it as a legend (Thomas Edison Center). The same story sometimes circulates with Henry Ford as the hero. The name changes, the missing source stays the same.

Enjoy the irony. A story meant to prove that you should taste before you salt has been shared thousands of times without anyone tasting it. The people who share it do exactly what the candidate in the legend does: they season before they check.

This is confirmation bias in its purest form, the tendency to retain information that supports what we already believe and to ignore what contradicts it. The anecdote flatters an idea we love, the genius who reads a personality at a glance, so we never question it (My book, chapter 6).

This book is for CEOs, managers, and innovators ready to challenge everything and create exponential growth.

Be the first to get the english version!

Edison’s real test: 146 questions and 32 survivors

The questionnaire, on the other hand, really existed. In 1921, Edison put candidates for management positions through a written exam of more than 140 questions, which varied by position. A few covered technical or business knowledge. Most tested memory for facts: the name of a famous violin maker, the state whose capital is Helena (Saturday Evening Post).

The outcome left no room for doubt. Of 718 applicants, 57 reached a score of 70%, and only 32 cleared the 90% bar Edison demanded (Mental Floss). Do the math: more than 95% failed at the target level. Theodore Edison, his own son, then a student at MIT, failed too (Saturday Evening Post).

Edison held one conviction: for him, intelligence boiled down to curiosity and memory (History.com). He distrusted college degrees, having himself received only a few months of formal schooling (Saturday Evening Post).

Then the questionnaire leaked. A rejected applicant, Charles Hansen, reconstructed the questions from memory and handed them to the New York Times. He called it a silly examination (Mental Floss). Once the questions were in print, Edison had to rewrite his test (History.com). The controversy filled the newspapers for weeks, and that is the context in which Boston reporters handed the sheet to Einstein.

Einstein was right, and AI proves him right a second time

Back to the Copley Plaza lobby. Einstein does not know the speed of sound by heart. He knows how to calculate it, he knows where to find it, and he understands wave propagation better than anyone in the room. Edison’s test still files him among the failures.

Edison measured what was easy to measure: the amount of facts stored in a head. Einstein defended what was hard to measure: the ability to reason with facts you go and look up when you need them.

In 1921, the debate was philosophical. In 2026, technology has settled it. An AI assistant answers the entire Edison questionnaire in a few seconds. A recruiter who selected on factual memory today would be hiring people in direct competition with a free tool.

So let me ask you an uncomfortable question. In your own hiring process, how many steps still assess something an AI does better than your candidates? Rate yourself from 0 to 5. I am not sure my own recruiting reflexes, shaped at Sagem, Apple and then Sony, would have earned a 5 a few years ago.

Xero’s coffee cup: a culture test in one gesture

Let’s change eras. Trent Innes led Xero for Australia and Asia for eight years (Startup Daily). His method, described in 2019 on The Venture podcast, made the rounds of the international press. At the start of the interview, he takes the candidate to the company kitchen and offers a drink. At the end, he watches for a single gesture: does the candidate offer to take the empty cup back?

For Innes, skills and experience can be gained with time and work, while attitude does not change. He wants people who take ownership of what needs doing on their own, in a culture where everyone washes their own cup (Directo al Paladar). According to him, only 5 to 10% of candidates fail, and what he assesses is the intention to offer, even when the candidate is not allowed to walk all the way to the sink (Click Petróleo e Gás).

I see the value of the test: it observes real behavior rather than a stated intention. But ask yourself two questions. Does a candidate whose career is on the line in the next hour really think about the cup? Does a candidate from a culture where, out of politeness, you never touch your host’s dishes come across as a poor team player?

The test may measure team spirit. It certainly measures cultural closeness to the recruiter. That is the territory of in-group bias, the preference for people who went through the same training and share the same codes (My book, chapter 7).

Musk’s riddle: the path matters more than the answer

Third boss. According to Ashlee Vance’s biography, Elon Musk personally interviewed nearly all of SpaceX’s first thousand employees, technicians and janitors included (CNBC). He liked to ask this riddle: you are standing on the surface of the Earth, you walk one mile south, one mile west and one mile north, and you end up exactly where you started. Where are you?

Most candidates eventually find the North Pole. Musk then follows up: where else? The second answer lies near the South Pole, one mile north of the circle of latitude that is exactly one mile around. Few engineers give it (CNBC). According to Vance, Musk cares most about how the engineer explains the reasoning (InsideHook).

Of the three bosses, Musk is the closest to Einstein. He watches someone reason out loud, a skill no database replaces.

The riddle still has a problem every recruiter knows: it gets around. Once printed in a bestseller and picked up by the press, it mostly measures who reads the press. Edison went through exactly the same misadventure in 1921.

What these tests really measure

Step back from these stories. They share one feature: a single signal, observed once, interpreted by the person who invented the test. Each leader projects his own values onto it. Edison valued memory, and his own was exceptional. Innes values the returned cup, because that is the culture he built.

Add the halo effect, which extends a good first impression to everything else about a person. A candidate who performs the expected gesture in the first five minutes will be seen as more competent for the rest of the interview, even on unrelated topics (My book).

These tests mostly flatter the person doing the hiring. They walk away feeling they have uncovered a secret the résumé was hiding. The feeling is pleasant, and it proves nothing about the candidate’s future performance.

What hiring science says

Work psychology has studied the validity of selection methods for decades. In 1998, Frank Schmidt and John Hunter published a meta-analysis summarizing 85 years of research on 19 selection methods (Plum). In their table, work sample tests had the highest validity, at 0.54 (Criteria).

In 2021, a re-analysis using more cautious statistical corrections revised these figures downward. Structured interviews moved to the top, with an estimated validity of 0.42. Their predictive power is more than double that of unstructured interviews, with nearly a third less bias (Criteria).

A structured interview asks every candidate the same questions, derived from a job analysis, and scores the answers on a grid set in advance. The exact opposite of the soup, the cup and the riddle asked on gut feeling.

Hire the way you innovate: experiment, measure

When I joined Sony Europe to launch the VAIO PCs, we were based in Londerzeel, in Flanders. HR had published the first job ads in Flemish. They only attracted Flemish candidates, while my team worked in French and English. I asked for the ads to run only in English, and beyond Flanders. In three months, we hired forty people from twenty-five nationalities. That team outperformed equivalent teams in Asia and North America by two to four times (My book, chapter 13).

The lever sat upstream of any interview: an assumption about the language of the ads that nobody had ever questioned. Assumptions are highly dangerous and should be killed with experimentation as early as possible.

So treat your hiring like an innovation project. Run it with indicators borrowed from growth hacking: the cost of acquiring a candidate, the cost of an interviewed candidate, the cost of a signed contract, employee retention and employee referrals (My book). If you are attached to your coffee cup test, measure it. Do the candidates who passed it perform better a year later? Do they stay longer? Without that measurement, you hold an anecdote. As far as I know, Edison never published how his 32 winners performed.

Finally, add the astonishment report. One week, one month and three months after they arrive, ask new hires what surprised them, for better or worse, including in the application and selection process. Your new hires are the only people who have lived your tests from the inside. Listen to them.

The essentials

Remember four things:

  • The salt test attributed to Edison, or sometimes to Ford, rests on no source. The 1921 questionnaire, on the other hand, is documented: more than 140 questions, 32 passes out of 718 applicants.
  • A test of factual memory selects profiles in direct competition with any AI. Einstein understood this as early as 1921.
  • A gesture observed once and interpreted by the test’s inventor mostly measures resemblance to the recruiter.
  • Recent re-analyses rank the structured interview first among predictive methods. Measure your own tests before you believe in them.

Hiring legends survive because they are pleasant to tell. Your next hire deserves better than a salt shaker. Taste before you salt: test your tests.

References

Picture of Philippe Boulanger

Philippe Boulanger

Philippe Boulanger, international speaker on innovation and artificial intelligence, author, advisor, mentor and consultant.

Latest POSTS

41 Shades of Blue: Google’s $200 Million A/B Test

By testing 41 shades of blue, Google claims it earned an extra $200 million a year. Behind the cult A/B testing anecdote lies a debate every leader has to settle: when to trust the data, and when it stops you from innovating.

Read More »

Drone vs Camera Robot: Who Films Where the Sky Is Banned?

Banned from flying over crowds, drones run into regulation and noise. The Beni camera robot films from the ground. A textbook case of the framing bias.

Read More »

Friday the 13th: why superstition pays off for innovators

Friday the 13th destroys nothing. People who fear it change their behavior, and that change can be measured. Accidents, stock markets, the Jerusalem virus, a $900 million figure nobody verified: what superstition reveals about your innovation decisions, and how to use it.

Read More »

Naming an innovation: what Sapir showed in 1929

In 1929, Edward Sapir showed that most people link the invented word “mal” to a large table and “mil” to a small one. Almost a century later, sound symbolism still shapes how we perceive product names. What the numbers really say, what marketing took from it, and how to test a name before you pick it.

Read More »

AI-Generated Code: Who Still Understands Your Software?

A year after handing 100% of its code to AI, a company finds that nobody can explain it anymore. An anonymous testimony, the METR, GitClear and Veracode studies, and the method to stay out of the labyrinth.

Read More »

Why Coca-Cola’s Inventor Never Got Rich

John Pemberton invented Coca-Cola in 1886 and died poor two years later. Thirty-three years after the first glass, the company was worth $25 million. What this story reveals about the difference between invention and innovation, and about the Pemberton who may be sitting in your own company.

Read More »

Rare-Earth-Free Motors: France Had a 14-Year Head Start

Renault has built rare-earth-free motors since 2012. Then the stator of its next E7A motor went to China. How a technological lead gets lost in purchasing trade-offs.

Read More »

Leeuwenhoek: the cloth merchant who saw microbes first

In 1676, a Delft draper with no university education discovered microbes. The Royal Society didn’t take his word for it: it checked, then elected him. What the accurate story of Leeuwenhoek teaches leaders about ideas that come from outside.

Read More »

Are you a rule breaker?

You weren’t supposed to find this.

But here you are, because you did what most people don’t: you questioned, you explored, you clicked the thing you weren’t sure you should click.

That’s Innovational Intelligence™ in action.

Most people stay inside the lines. Follow the expected path. Click the obvious buttons. Accept things as they are.

Not you.

You’re one of those rare minds that refuses to accept “this is how it’s always been done.”

We need more people who think like you.

So here’s your reward for coloring outside the lines:

Get VIP pre-release access to the next assessment on Innovational Intelligence™:

You’ll be the first to know when it’s available.

Keep breaking rules. The world needs what you see.

Your daily dose (or almost) of Innovational Intelligence™!

Receive in your inbox my latest post before everyone else!