identity tests
Does who a candidate is change their rank?
We change one detail of an invented CV at a time — the name, the home city, a career break — and check whether the candidate moves. Every figure on this page comes from the recorded test runs, including the ones that show a problem.
1 · What the scorer does not read
Before a CV is scored, the candidate's name, contact details, home city and personal details (date of birth, nationality, marital status, religion, caste category, gender, parents' names, passport place of issue, blood group, and height and weight unless your job description sets them) are taken out of the text the model reads, along with photo lines, faith affiliations, gendered pronouns, career-break entries, the names of schools and universities (the degree and subject stay), and any language your job description does not ask for. This happens on every run, whatever you choose to see on screen, and the CV you read is unchanged. It is best-effort removal of what is written in recognisable places — see the known gaps below.
2 · Changing one detail at a time
We test this on invented CVs by changing one thing at a time. Before identity was taken out of the scored text, changing only the name moved a candidate down in 157 of 420 tests, by up to 25 places, and changing only the home city moved one down in 131 of 273, by up to 42. Now, changing the name, home city, a career break, nationality, age, marital status, religious affiliation, caste category, parents' names, passport place of issue, height, weight, blood group, a photo line, gendered wording, the university, a language your job description does not ask for, where a candidate worked, or the country of a past employer moves nobody, in plain-text and PDF-shaped CVs alike. Where a candidate worked was the strongest of these before it was removed: it moved the CV down in 267 of 360 tests, worst for Kochi, Kathmandu, Dhaka and Manila. That is a statement about those changes, on those CVs, in the layouts we used — not about every CV.
Before scores the CV with nothing taken out. Now scores the text the product scores today, once as plain text and once shaped like text pulled out of a PDF. Each cell counts the tests in which changing that one detail moved the candidate down the ranking.
| What changed | Before | Worst drop before, in places | Now, plain text | Now, PDF-shaped |
|---|---|---|---|---|
| Name | 157 of 420 | 25 | 0 of 420 | 0 of 420 |
| Home city | 131 of 273 | 42 | 0 of 271 | 0 of 277 |
| A career break | 15 of 42 | 7 | 0 of 42 | 0 of 42 |
| University | 19 of 280 | 2 | 0 of 273 | 0 of 280 |
| A language the job description does not ask for | 55 of 210 | 6 | 0 of 210 | 0 of 210 |
| Religious affiliation | 54 of 168 | 10 | 0 of 168 | 0 of 168 |
| Gendered wording | 1 of 42 | 2 | 0 of 42 | 0 of 42 |
| Nationality | 67 of 252 | 9 | 0 of 252 | 0 of 252 |
| Age | 10 of 84 | 3 | 0 of 84 | 0 of 84 |
| Marital status | 5 of 42 | 3 | 0 of 42 | 0 of 42 |
| Caste category | 34 of 126 | 4 | 0 of 126 | 0 of 126 |
| Parents' names | 33 of 126 | 6 | 0 of 126 | 0 of 126 |
| Passport place of issue | 11 of 126 | 4 | 0 of 126 | 0 of 126 |
| Height and weight | 26 of 126 | 5 | 0 of 126 | 0 of 126 |
| Blood group | 23 of 84 | 4 | 0 of 84 | 0 of 84 |
| A photo line | 20 of 84 | 6 | 0 of 84 | 0 of 84 |
| Where the candidate worked | 267 of 360 | 26 | 0 of 351 | 0 of 360 |
| The country of a past employer | 154 of 240 | 23 | 0 of 234 | 0 of 240 |
3 · Names and layouts the rules were not written around
We also test names and layouts the rules were not written around: initials, single names, Bahadur and Kumar middle names, bin and Al- names, names in capitals, names that are also ordinary words, and names that appear only in an email address. On the most recent of these sets — new names, new cities and new layouts, built after the rules were final, checked against the detector's own source, and measured once — the name and the home city were both removed in 100% of 4,788 cases, including where the city sits in a trailing personal-details block or in an opening sentence rather than the header. Changing who the candidate was still changed the score in 6 of 1764 tests, moving one candidate down by two places. Two other layout sets, measured in the same run, still leak a home city — see the known gaps.
| Cases | Tests | Name taken out | Home city taken out |
|---|---|---|---|
| All cases | 4788 | 100% | 100% |
| Kind of name | |||
| Initials | 684 | 100% | 100% |
| Single names | 684 | 100% | 100% |
| Bahadur and Kumar middle names | 684 | 100% | 100% |
| bin and Al- names | 684 | 100% | 100% |
| Names in capitals | 684 | 100% | 100% |
| Names that are also ordinary words | 684 | 100% | 100% |
| Name only in the email address | 684 | 100% | 100% |
| Layout | |||
| Two-column layout | 1596 | 100% | 100% |
| Personal details at the end | 1596 | 100% | 100% |
| Opening objective sentence | 1596 | 100% | 100% |
| File shape | |||
| Plain text | 2394 | 100% | 100% |
| Flattened PDF text | 2394 | 100% | 100% |
| Filename | |||
| Filename carries the candidate's name | 2394 | 100% | 100% |
| Filename carries no name | 2394 | 100% | 100% |
The two other layout sets
Measured in the same run, these two sets still leak a home city written outside the header. They are the first of the known gaps below.
| Layout set | Tests | Home city taken out | Flattened PDF tests | Home city taken out, PDF |
|---|---|---|---|---|
| Other layout set 1 | 4788 | 88.6% | 2394 | 77.2% |
| Other layout set 2 | 4788 | 88.6% | 2394 | 77.2% |
4 · Known gaps
- A home city written outside the header — a contact block at the foot of the CV, or an "about me" paragraph — can still reach the model. On two other held-out layout sets, measured at the same time, it was removed in 89% of cases, and in 77% when the CV arrives as one line of flattened PDF text. The set built around a trailing details block and an opening "residing in" sentence was clean; these two layouts are not.
- Employers are kept, so a well-known regional employer can still hint at where someone has worked, even though the location itself is removed. Taking employers out would remove real evidence of experience.
- A name with no email address or filename to confirm it can be missed when it is a single word or is also a job title, such as Mason.
- Everything here was tested on invented CVs in English. Real CVs were not used, and scanned CVs were not tested at all — a scan cannot be read today, so there is nothing to test.
- Signals that are not removed: how a CV is written, and which skills a person lists.
English CVs only, for now. The model was trained on English text, so a CV in Arabic, Hindi or another language is not read meaningfully and will rank low.
5 · What these tests cannot show
They show whether changing one written detail moves an invented candidate. They cannot show how a ranking treats groups of real applicants: real CVs were not used, and the signals listed above remain. That is why the adverse-impact monitor still measures outcomes after the fact. How the rest of the pipeline works is on the methodology page.