0

I'm looking for a function (from a Python library or similar) that takes as input a name and produces as output an encoding that is shared amount variants/misspellings of that name, i.e.

nameEncoder('Jon Smith')->674453

nameEncoder('Johnathan Smith')->674453

nameEncoder('Elizabeth Doe')->435132

nameEncoder('Elisabeth Doe')->435132

Happy to have that encoding simply be the most common variant. I've looked at various fuzzy matching tools, but all seem to focus on comparing value pairs. I'd like to do this as a data-preprocessing step so subsequent joins/aggregation/deduping don't need to use fuzzy matching.

I realize that there will be ambiguity here, just as there is in fuzzy matching, so nothing will be perfect.

Various SAS tools have a feature called Match Code that do what I'm looking for, but only within their (expensive) ecosystem.

P.S. I'm also looking for similar functionality with addresses, although that's a topic for another post.

[Cross post from SO, under advice that this would be a more appropriate forum]

Improve this question
3
  • 1
    Cross post from SO, under advice that this would be a more appropriate forum Yes if you're looking for a library or some other pre-made program that does so, sure it is, just make sure to better define your requirements, as defined in the question guidelines. If you're asking for a piece of code, most likely this will be closed as off-topic.
    – Alejandro
    Oct 20, 2021 at 21:09
  • Thanks, yeah, certainly looking for a library - the code required for this would be too much to ask from a forum post
    – Robmattles
    Oct 20, 2021 at 23:37
  • The software itself would be easy to write. The real problem is finding a database of equivalent names. Mapping "Bob" to "Robert" or "Peg" to "Margaret" can't be done without it.
    – Ray Butterworth
    Nov 15, 2022 at 14:45

1 Answer 1

Reset to default
0

The system you are looking for is called Soundex.

It was developed nearly a century ago for exactly the purpose you laid out.

There are a number of implementations of the algorithm in Python. This will get you started.

Improve this answer
2
  • I founde Soundex - it doesn't get things like John vs. Johnathan. Soundex+a dataset of common nicknames gets close. But in soundex, Jane, Jan, and John are all the same name
    – Robmattles
    Oct 20, 2021 at 23:36
  • 1
    I don't see how it would be possible to both remove information from the source, and retain it and still allow matching. You are left with looking up spellings of names otherwise.
    – Bob Dalgleish
    Oct 21, 2021 at 17:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.