What a comparison can measure
NIST’s one-to-one verification track compares a probe with a claimed reference. Its one-to-many track searches across a predefined gallery. A public image-search product can also miss a relevant page because that page was never indexed, was blocked, or disappeared. This distinction matters when a product returns no result.
This is a proposed method. FaceIndex reports no benchmark scores on this page.
A reproducible test design
- Build a consented reference set
Use opt-in adult participants or images licensed for this exact purpose. Record the dataset version. Keep material off public search services if the consent or license does not cover their handling.
- Separate search conditions
Measure same-file reuse and known edited copies separately from one-to-many similarity. Include an authorized negative set and distinguish absent sources from retrieval misses.
- Compare the same inputs
Record the date, product/version where known, conditions, number of trials, and how no-result cases were counted. Vary one condition at a time when feasible.
- Review errors and disclose incentives
Count useful correct results and wrong candidates separately. Report denominators and limitations. Identify FaceIndex as a product operator in any comparison it publishes.
Read more
Build a scorecard you can reproduce
Set aside separate groups for known exact copies, authorized edits, and queries with no expected result. Record the expected destination for positive examples before searching. Search interruption, provider error, and a completed search with no result deserve different labels. Changing more than one photo condition at once makes it hard to interpret a miss.
Record the observations
- Known copy found
- Count a correct source only if the result opens to the expected public page or a documented equivalent. Report the number found over the number of eligible, accessible known copies.
- Incorrect candidate
- Record misleading candidates separately and identify which comparison produced them. Count negative queries that returned an incorrect lead as well as individual wrong candidates.
- Source unavailable
- Put a removed or blocked destination in its own category. Preserve the denominator and explain how unavailable sources affect it.
For repeat tests, preserve the approved reference set, the query date, and the decision threshold where the provider exposes one. Comparing two runs from different dates can mix changes to the index with changes to the algorithm.
Worked example: an edited copy falls out of the results
Suppose an exact duplicate appears, but a resized and cropped version does not. Before judging the tool, confirm the expected public destination remains accessible. Repeat the comparison with resize and crop as separate conditions. If only the crop drops out, report that observation for this set and this test version. Small examples help design a larger test. They do not establish a general accuracy claim.
Use a consenting or appropriately licensed test set. Confirm what each product may retain or disclose before including it in the study.
Industry benchmarks can explain the error terms, but their datasets and thresholds are different from a public search service’s coverage. NIST verification evaluation · NIST identification evaluation.

