Commit Graph

1029 Commits

Author SHA1 Message Date
Al
b3ef8ded12 [formatting] Adding OSM address components lookup by country 2015-11-17 11:39:39 -05:00
Al
0b74039a6a [formatting] Adding city_district as a separate format tag 2015-11-17 11:38:38 -05:00
Al
48a305c8c4 [fix] Reverting last two changes, have to fix on the OSM side 2015-11-01 16:23:41 -05:00
Al
90773294b9 [polygons] Only fixing polygons in cases with inner rings 2015-11-01 12:36:35 -05:00
Al
477300c061 [polygons] Eliminating fix_polygon 2015-11-01 03:12:58 -05:00
Al
aba2c51e65 [fix] name 2015-11-01 01:35:01 -04:00
Al
1dbfc6a87b [polygons/neighborhoods] Not counting local admin polys unless they match OSM, fix for Paris arrondissements 2015-11-01 01:24:43 -04:00
Al
4fdaef2638 [fix] Don't need Quattroshapes dir for OSM Rtree 2015-10-31 19:08:04 -04:00
Al
e9a6ea1d72 [fix] default index path 2015-10-31 18:47:24 -04:00
Al
4a35c50f92 [fix] index paths 2015-10-31 18:30:53 -04:00
Al
c54fb412a3 [fix] save_index 2015-10-31 17:59:44 -04:00
Al
0227d9335f [fix] Removing some debug code 2015-10-31 14:50:32 -04:00
Al
5882c2d64b [fix] command-line arg II 2015-10-31 14:25:17 -04:00
Al
66f8a2dc9e [fix] command-line arg 2015-10-31 14:24:23 -04:00
Al
e5d8812504 [fix] argument default 2015-10-31 14:23:37 -04:00
Al
f39090869e [fix] imports 2015-10-31 14:22:45 -04:00
Al
8166cd66c8 [fix] encoding yet again 2015-10-31 14:19:50 -04:00
Al
f473ff0dad [fix] encoding, different file 2015-10-31 14:18:47 -04:00
Al
a2eb40109c [fix] file encoding 2015-10-31 14:17:26 -04:00
Al
3e43ac7255 [polygons/osm] Adding a unified neighborhood reverse geocoder incorporating Zetashapes, OSM and Quattroshapes. Uses the new Soft TFIDF implementation to approximately match OSM names to Quattroshapes/Zetashapes names and geohash indices for more coarse point-in-polygon tests (OSM neighborhoods are stored as points not polygons, so need to match with a geometry from the other sources) 2015-10-31 14:15:39 -04:00
Al
a38624ba59 [similarity] Adding NameDeduper base class for deduping geographic names using the new Soft TFIDF similarity 2015-10-31 00:57:02 -04:00
Al
a5c1296044 [similarity] Adding Jaccard similarity with word frequencies instead of simple sets, better for ideographic scripts (Han, Hangul, etc.) in the absence of word segmentation since there may be many high frequency characters 2015-10-30 14:35:38 -04:00
Al
cbeb08f1d1 [python/normalize] importing options from the C module 2015-10-30 12:34:07 -04:00
Al
cccc3e9cf5 [similarity] Using Soft-TFIDF for approximate name matching. Soft-TFIDF is a hybrid string distance metric which balances local token similarities (using Jaro-Winkler similarity by default) allowing for slight spelling errors with global TFIDF statistics so that very frequent words don't affect the score as much 2015-10-30 02:48:16 -04:00
Al
e7f783477f [python/normalize] Adding remove parentheses options in Python normalize (would require compiling with the scanner to do it from C, but could switch) 2015-10-30 01:27:16 -04:00
Al
5076c0409b [similarity] Adding an in-memory IDF index for weighted similarities 2015-10-29 13:33:01 -04:00
Al
1c543a5271 [osm/formatting] Adding is_in tags to the address formatter as they're common in OSM, aliasing addr:district to state_district instead of suburb 2015-10-29 12:30:56 -04:00
Al
c7df3fcb3a [osm] Adding a list of various OSM name tags obtained from Nominatim 2015-10-29 11:44:56 -04:00
Al
cee9da05d6 [fix] using tokenize_raw API 2015-10-28 21:37:44 -04:00
Al
bbd10e97bd [fix] imports 2015-10-28 21:32:09 -04:00
Al
110451d6d6 [polygons] Polygon area calculations 2015-10-28 21:19:35 -04:00
Al
e946e63222 [polygons] Changing language polygon index to use new index_polygon method 2015-10-28 21:18:27 -04:00
Al
5fdbb7e832 [polygons] Adding a geohash polygon index which selects a prefix size based on the area of the polygon's bounding box 2015-10-28 21:17:33 -04:00
Al
094a5bf5f4 [dictionaries] adding Jnr and Snr forms for generational suffixes 2015-10-28 00:00:34 -04:00
Al
c2d112f4fc [fix] compile flags in Makefile.am 2015-10-27 19:01:37 -04:00
Al
9a92a1154d [python] Making normalized_tokens return token classes as well, mimicking the tokenize API 2015-10-27 17:07:50 -04:00
Al
7f5f056105 [python] don't need -O0 any more for normalization extension 2015-10-27 13:34:00 -04:00
Al
9f6e1387a0 [fix] Error condition in Python tokenize 2015-10-27 13:33:28 -04:00
Al
6aaa08c220 [fix] Usage on libpostal_data script 2015-10-27 13:33:03 -04:00
Al
40918812e2 [normalize] Adding hyphen elimination as a string option (changes tokenization) 2015-10-27 13:32:47 -04:00
Al
3fe2365234 [fix] signed size_t in trie_set_tail 2015-10-27 13:21:26 -04:00
Al
ad59ba7a7b [fix] Re-generating transliteration tables 2015-10-27 12:28:08 -04:00
Al
7f5cf89e84 [transliteration] Not escaping right side transliteration rules 2015-10-27 12:24:38 -04:00
Al
1a1d74785c [fix] Compiler warnings for casts/printf 2015-10-26 18:52:18 -04:00
Al
6b456025b4 [fix] warnings in klib/ksort.h 2015-10-26 18:50:22 -04:00
Al
3b3513ffe3 [fix] warnings in collections.h/vector_math.h 2015-10-26 18:49:58 -04:00
Al
83c6a87ab1 [build] substitution for use of LIBPOSTAL_DATA_DIR in Makefile.am 2015-10-26 18:47:07 -04:00
Al
f6b6a17335 [python/normalization] Adding Python bindings to the normalize module for use in OSM polygon matching 2015-10-26 18:07:53 -04:00
Al
a319c1f6a0 [build] defining LIBPOSTAL_DATA_DIR in Autoconf rather than Automake, becomes part of config.h 2015-10-26 18:06:05 -04:00
Al
8a188903b3 [python] Using tuples in pytokenize instead of list, pre-allocating 2015-10-26 18:04:13 -04:00