Commit Graph

9 Commits

Author SHA1 Message Date
Al
ee1aa564c4 [normalization] normalize tokens should not replace digits by default 2016-07-21 17:04:57 -04:00
Al
dd8f8b4d7b [fix] prefix/suffix regexes 2015-12-05 18:41:22 -05:00
Al
2a4210f93f [osm] Stripping standard city prefixes/suffies e.g. Township of 2015-12-05 15:42:22 -05:00
Al
f39090869e [fix] imports 2015-10-31 14:22:45 -04:00
Al
f473ff0dad [fix] encoding, different file 2015-10-31 14:18:47 -04:00
Al
3e43ac7255 [polygons/osm] Adding a unified neighborhood reverse geocoder incorporating Zetashapes, OSM and Quattroshapes. Uses the new Soft TFIDF implementation to approximately match OSM names to Quattroshapes/Zetashapes names and geohash indices for more coarse point-in-polygon tests (OSM neighborhoods are stored as points not polygons, so need to match with a geometry from the other sources) 2015-10-31 14:15:39 -04:00
Al
a38624ba59 [similarity] Adding NameDeduper base class for deduping geographic names using the new Soft TFIDF similarity 2015-10-31 00:57:02 -04:00
Al
a5c1296044 [similarity] Adding Jaccard similarity with word frequencies instead of simple sets, better for ideographic scripts (Han, Hangul, etc.) in the absence of word segmentation since there may be many high frequency characters 2015-10-30 14:35:38 -04:00
Al
cccc3e9cf5 [similarity] Using Soft-TFIDF for approximate name matching. Soft-TFIDF is a hybrid string distance metric which balances local token similarities (using Jaro-Winkler similarity by default) allowing for slight spelling errors with global TFIDF statistics so that very frequent words don't affect the score as much 2015-10-30 02:48:16 -04:00