Skip to content
Meritline

identity tests

Does who a candidate is change their rank?

We change one detail of an invented CV at a time — the name, the home city, a career break — and check whether the candidate moves. Every figure on this page comes from the recorded test runs, including the ones that show a problem.

1 · What the scorer does not read

Before a CV is scored, the candidate's name, contact details, home city and personal details (date of birth, nationality, marital status, religion, caste category, gender, parents' names, passport place of issue, blood group, and height and weight unless your job description sets them) are taken out of the text the model reads, along with photo lines, faith affiliations, gendered pronouns, career-break entries, the names of schools and universities (the degree and subject stay), and any language your job description does not ask for. This happens on every run, whatever you choose to see on screen, and the CV you read is unchanged. It is best-effort removal of what is written in recognisable places — see the known gaps below.

2 · Changing one detail at a time

We test this on invented CVs by changing one thing at a time. Before identity was taken out of the scored text, changing only the name moved a candidate down in 157 of 420 tests, by up to 25 places, and changing only the home city moved one down in 131 of 273, by up to 42. Now, changing the name, home city, a career break, nationality, age, marital status, religious affiliation, caste category, parents' names, passport place of issue, height, weight, blood group, a photo line, gendered wording, the university, a language your job description does not ask for, where a candidate worked, or the country of a past employer moves nobody, in plain-text and PDF-shaped CVs alike. Where a candidate worked was the strongest of these before it was removed: it moved the CV down in 267 of 360 tests, worst for Kochi, Kathmandu, Dhaka and Manila. That is a statement about those changes, on those CVs, in the layouts we used — not about every CV.

Before scores the CV with nothing taken out. Now scores the text the product scores today, once as plain text and once shaped like text pulled out of a PDF. Each cell counts the tests in which changing that one detail moved the candidate down the ranking.

Swap tests: how often changing one detail moved a candidate down, before and after identity was taken out of the scored text
What changedBeforeWorst drop before, in placesNow, plain textNow, PDF-shaped
Name157 of 420250 of 4200 of 420
Home city131 of 273420 of 2710 of 277
A career break15 of 4270 of 420 of 42
University19 of 28020 of 2730 of 280
A language the job description does not ask for55 of 21060 of 2100 of 210
Religious affiliation54 of 168100 of 1680 of 168
Gendered wording1 of 4220 of 420 of 42
Nationality67 of 25290 of 2520 of 252
Age10 of 8430 of 840 of 84
Marital status5 of 4230 of 420 of 42
Caste category34 of 12640 of 1260 of 126
Parents' names33 of 12660 of 1260 of 126
Passport place of issue11 of 12640 of 1260 of 126
Height and weight26 of 12650 of 1260 of 126
Blood group23 of 8440 of 840 of 84
A photo line20 of 8460 of 840 of 84
Where the candidate worked267 of 360260 of 3510 of 360
The country of a past employer154 of 240230 of 2340 of 240

Test run of 12 Sept 2026.

3 · Names and layouts the rules were not written around

We also test names and layouts the rules were not written around: initials, single names, Bahadur and Kumar middle names, bin and Al- names, names in capitals, names that are also ordinary words, and names that appear only in an email address. On the most recent of these sets — new names, new cities and new layouts, built after the rules were final, checked against the detector's own source, and measured once — the name and the home city were both removed in 100% of 4,788 cases, including where the city sits in a trailing personal-details block or in an opening sentence rather than the header. Changing who the candidate was still changed the score in 6 of 1764 tests, moving one candidate down by two places. Two other layout sets, measured in the same run, still leak a home city — see the known gaps.

Held-out test, most recent set: how often the name and the home city were taken out, by kind of name, layout, file shape and filename
CasesTestsName taken outHome city taken out
All cases4788100%100%
Kind of name
Initials684100%100%
Single names684100%100%
Bahadur and Kumar middle names684100%100%
bin and Al- names684100%100%
Names in capitals684100%100%
Names that are also ordinary words684100%100%
Name only in the email address684100%100%
Layout
Two-column layout1596100%100%
Personal details at the end1596100%100%
Opening objective sentence1596100%100%
File shape
Plain text2394100%100%
Flattened PDF text2394100%100%
Filename
Filename carries the candidate's name2394100%100%
Filename carries no name2394100%100%

Test run of 12 Sept 2026. In the same set, changing who the candidate was changed the score in 6 of 1764 tests.

The two other layout sets

Measured in the same run, these two sets still leak a home city written outside the header. They are the first of the known gaps below.

Held-out test, two earlier layout sets: how often the home city was taken out, overall and in flattened PDF text
Layout setTestsHome city taken outFlattened PDF testsHome city taken out, PDF
Other layout set 1478888.6%239477.2%
Other layout set 2478888.6%239477.2%

4 · Known gaps

  • A home city written outside the header — a contact block at the foot of the CV, or an "about me" paragraph — can still reach the model. On two other held-out layout sets, measured at the same time, it was removed in 89% of cases, and in 77% when the CV arrives as one line of flattened PDF text. The set built around a trailing details block and an opening "residing in" sentence was clean; these two layouts are not.
  • Employers are kept, so a well-known regional employer can still hint at where someone has worked, even though the location itself is removed. Taking employers out would remove real evidence of experience.
  • A name with no email address or filename to confirm it can be missed when it is a single word or is also a job title, such as Mason.
  • Everything here was tested on invented CVs in English. Real CVs were not used, and scanned CVs were not tested at all — a scan cannot be read today, so there is nothing to test.
  • Signals that are not removed: how a CV is written, and which skills a person lists.

English CVs only, for now. The model was trained on English text, so a CV in Arabic, Hindi or another language is not read meaningfully and will rank low.

5 · What these tests cannot show

They show whether changing one written detail moves an invented candidate. They cannot show how a ranking treats groups of real applicants: real CVs were not used, and the signals listed above remain. That is why the adverse-impact monitor still measures outcomes after the fact. How the rest of the pipeline works is on the methodology page.