549 Commits

Author	SHA1	Message	Date
Al	1dbfc6a87b	[polygons/neighborhoods] Not counting local admin polys unless they match OSM, fix for Paris arrondissements	2015-11-01 01:24:43 -04:00
Al	4fdaef2638	[fix] Don't need Quattroshapes dir for OSM Rtree	2015-10-31 19:08:04 -04:00
Al	e9a6ea1d72	[fix] default index path	2015-10-31 18:47:24 -04:00
Al	4a35c50f92	[fix] index paths	2015-10-31 18:30:53 -04:00
Al	c54fb412a3	[fix] save_index	2015-10-31 17:59:44 -04:00
Al	0227d9335f	[fix] Removing some debug code	2015-10-31 14:50:32 -04:00
Al	5882c2d64b	[fix] command-line arg II	2015-10-31 14:25:17 -04:00
Al	66f8a2dc9e	[fix] command-line arg	2015-10-31 14:24:23 -04:00
Al	e5d8812504	[fix] argument default	2015-10-31 14:23:37 -04:00
Al	f39090869e	[fix] imports	2015-10-31 14:22:45 -04:00
Al	8166cd66c8	[fix] encoding yet again	2015-10-31 14:19:50 -04:00
Al	f473ff0dad	[fix] encoding, different file	2015-10-31 14:18:47 -04:00
Al	a2eb40109c	[fix] file encoding	2015-10-31 14:17:26 -04:00
Al	3e43ac7255	[polygons/osm] Adding a unified neighborhood reverse geocoder incorporating Zetashapes, OSM and Quattroshapes. Uses the new Soft TFIDF implementation to approximately match OSM names to Quattroshapes/Zetashapes names and geohash indices for more coarse point-in-polygon tests (OSM neighborhoods are stored as points not polygons, so need to match with a geometry from the other sources)	2015-10-31 14:15:39 -04:00
Al	a38624ba59	[similarity] Adding NameDeduper base class for deduping geographic names using the new Soft TFIDF similarity	2015-10-31 00:57:02 -04:00
Al	a5c1296044	[similarity] Adding Jaccard similarity with word frequencies instead of simple sets, better for ideographic scripts (Han, Hangul, etc.) in the absence of word segmentation since there may be many high frequency characters	2015-10-30 14:35:38 -04:00
Al	cccc3e9cf5	[similarity] Using Soft-TFIDF for approximate name matching. Soft-TFIDF is a hybrid string distance metric which balances local token similarities (using Jaro-Winkler similarity by default) allowing for slight spelling errors with global TFIDF statistics so that very frequent words don't affect the score as much	2015-10-30 02:48:16 -04:00
Al	5076c0409b	[similarity] Adding an in-memory IDF index for weighted similarities	2015-10-29 13:33:01 -04:00
Al	1c543a5271	[osm/formatting] Adding is_in tags to the address formatter as they're common in OSM, aliasing addr:district to state_district instead of suburb	2015-10-29 12:30:56 -04:00
Al	c7df3fcb3a	[osm] Adding a list of various OSM name tags obtained from Nominatim	2015-10-29 11:44:56 -04:00
Al	bbd10e97bd	[fix] imports	2015-10-28 21:32:09 -04:00
Al	110451d6d6	[polygons] Polygon area calculations	2015-10-28 21:19:35 -04:00
Al	e946e63222	[polygons] Changing language polygon index to use new index_polygon method	2015-10-28 21:18:27 -04:00
Al	5fdbb7e832	[polygons] Adding a geohash polygon index which selects a prefix size based on the area of the polygon's bounding box	2015-10-28 21:17:33 -04:00
Al	7f5cf89e84	[transliteration] Not escaping right side transliteration rules	2015-10-27 12:24:38 -04:00
Al	e3b80534ba	[polygons] Adding methods for fixing polygons in base RTreePolygonIndex, moving the current polygon index to an instance variable and adding ability to import from a general GeoJSON-like structure instead of just shapefiles	2015-10-25 18:36:12 -04:00
Al	2d4b3a6e2f	[parser/formatting] Appendinge suburb between the road line and any subsequent lines for all bottom-up address formats. Effectively inserts neighborhoods into our version without making the OpenCage formats overly verbose. Also fixing post-format replace with group capture	2015-10-24 01:31:35 -04:00
Al	da53d7ebac	[osm] Adding an OSM neighborhoods/suburbs data set for matching with Quattroshapes boundaries, updating definitions for admin boundaries	2015-10-22 11:37:11 -04:00
Al	6478e65a06	[osm] Moving Wikipedia title normalization to osm.extract	2015-10-22 11:35:38 -04:00
Al	ff3a3c2201	[fix] disambiguation tokenizer to pypostal	2015-10-21 16:35:55 -04:00
Al	6f6d04966b	[fix] role in OSM polygon extraction	2015-10-21 16:35:25 -04:00
Al	336bfe32ca	[osm/formatter] Switching back to OpenCageData repo	2015-10-21 16:34:24 -04:00
Al	d0aa3b9109	[polygons/osm] alt_name	2015-10-20 09:27:55 -04:00
Al	218eae548c	[fix] logger	2015-10-20 06:43:51 -04:00
Al	1e8e592e0b	[fix] import	2015-10-19 23:30:12 -04:00
Al	e5129957f8	[osm/polygons] Add relation id to OSM reverse geocoder	2015-10-19 18:00:45 -04:00
Al	5187e6073a	[fix] admin boundary imports	2015-10-19 17:14:48 -04:00
Al	0d213e426a	[fix] logger	2015-10-19 15:55:19 -04:00
Al	f5bd9b8371	[polygons/osm] logging during reverse geocoder construction	2015-10-19 15:54:07 -04:00
Al	8609ccbb1d	[polygons/osm] lon, lat	2015-10-19 15:40:43 -04:00
Al	ef94f1b712	[doc] Adding some comments to fetch_osm_address_data.sh	2015-10-19 15:39:31 -04:00
Al	83295b1b34	[polygons/osm] Adding in-memory OSM reverse geocoder for all admin boundaries	2015-10-19 15:38:23 -04:00
Al	4a3994c65e	[polygons/osm] Construct polygons from OSM relations using a number of space-saving optimizations in order to process planet in a reasonable amount of memory. Builds a graph of connected ways such that forming polygons is equivalent to finding strongly connected components.	2015-10-18 20:53:49 -04:00
Al	b44a72588f	[polygons/osm] Connecting OSM polygons from their constituent ways is an instance of finding strongly connected components in a graph, adding implementation	2015-10-18 18:23:27 -04:00
Al	ade0e2dc1f	[osm] Adding final .osm file variable for borders output	2015-10-16 00:46:40 -04:00
Al	b5f8b696bf	[osm] Moving parse_osm to a separate module, adding option to list dependencies	2015-10-15 20:22:12 -04:00
Al	ca629e295d	[osm] Adding admin boundaries filter in OSM data	2015-10-15 12:06:11 -04:00
Al	153c8c9cc4	[coordinates] Better handling for float DMS coordinates	2015-10-14 15:10:58 -04:00
Al	e584745061	[formatting] Adding STATE_DISTRICT to formatter for things like counties	2015-10-14 15:10:18 -04:00
Al	efba7987b5	[coordinates] sticking latlon_to_decimal in its own module, adding missing methods	2015-10-14 13:38:09 -04:00