363 Commits

Author	SHA1	Message	Date
Al	4f0d6fbf79	[fix] default arg again	2015-11-22 14:22:09 -05:00
Al	4cc275e313	[fix] doc and default arg	2015-11-22 14:21:20 -05:00
Al	c8f47b38a2	[osm/formatting] Adding OSM polygon lookups and neighborhood polygon lookups to the training data in order to provide more variations for the model to work with	2015-11-21 17:05:35 -05:00
Al	9fc60600dd	[fix] OSM reverse geocoder polygon ordering	2015-11-20 14:49:37 -05:00
Al	130518fe58	[polygons] OSM reverse geocoder sort levels	2015-11-20 13:52:30 -05:00
Al	b948a8ebd8	[osm] Adding global keys which map to OSM address components	2015-11-20 12:48:54 -05:00
Al	85667997cd	[formatting] Adding city_district and state_district tags to address formatting templates where it makes sense. These will not be in all addresses, tags can be added and removed from the training data with certain probabilities	2015-11-20 12:24:51 -05:00
Al	946bce1cb9	[osm] Adding a few more boundary types to planet admin borders	2015-11-17 11:40:42 -05:00
Al	b3ef8ded12	[formatting] Adding OSM address components lookup by country	2015-11-17 11:39:39 -05:00
Al	0b74039a6a	[formatting] Adding city_district as a separate format tag	2015-11-17 11:38:38 -05:00
Al	48a305c8c4	[fix] Reverting last two changes, have to fix on the OSM side	2015-11-01 16:23:41 -05:00
Al	90773294b9	[polygons] Only fixing polygons in cases with inner rings	2015-11-01 12:36:35 -05:00
Al	477300c061	[polygons] Eliminating fix_polygon	2015-11-01 03:12:58 -05:00
Al	aba2c51e65	[fix] name	2015-11-01 01:35:01 -04:00
Al	1dbfc6a87b	[polygons/neighborhoods] Not counting local admin polys unless they match OSM, fix for Paris arrondissements	2015-11-01 01:24:43 -04:00
Al	4fdaef2638	[fix] Don't need Quattroshapes dir for OSM Rtree	2015-10-31 19:08:04 -04:00
Al	e9a6ea1d72	[fix] default index path	2015-10-31 18:47:24 -04:00
Al	4a35c50f92	[fix] index paths	2015-10-31 18:30:53 -04:00
Al	c54fb412a3	[fix] save_index	2015-10-31 17:59:44 -04:00
Al	0227d9335f	[fix] Removing some debug code	2015-10-31 14:50:32 -04:00
Al	5882c2d64b	[fix] command-line arg II	2015-10-31 14:25:17 -04:00
Al	66f8a2dc9e	[fix] command-line arg	2015-10-31 14:24:23 -04:00
Al	e5d8812504	[fix] argument default	2015-10-31 14:23:37 -04:00
Al	f39090869e	[fix] imports	2015-10-31 14:22:45 -04:00
Al	8166cd66c8	[fix] encoding yet again	2015-10-31 14:19:50 -04:00
Al	f473ff0dad	[fix] encoding, different file	2015-10-31 14:18:47 -04:00
Al	a2eb40109c	[fix] file encoding	2015-10-31 14:17:26 -04:00
Al	3e43ac7255	[polygons/osm] Adding a unified neighborhood reverse geocoder incorporating Zetashapes, OSM and Quattroshapes. Uses the new Soft TFIDF implementation to approximately match OSM names to Quattroshapes/Zetashapes names and geohash indices for more coarse point-in-polygon tests (OSM neighborhoods are stored as points not polygons, so need to match with a geometry from the other sources)	2015-10-31 14:15:39 -04:00
Al	a38624ba59	[similarity] Adding NameDeduper base class for deduping geographic names using the new Soft TFIDF similarity	2015-10-31 00:57:02 -04:00
Al	a5c1296044	[similarity] Adding Jaccard similarity with word frequencies instead of simple sets, better for ideographic scripts (Han, Hangul, etc.) in the absence of word segmentation since there may be many high frequency characters	2015-10-30 14:35:38 -04:00
Al	cccc3e9cf5	[similarity] Using Soft-TFIDF for approximate name matching. Soft-TFIDF is a hybrid string distance metric which balances local token similarities (using Jaro-Winkler similarity by default) allowing for slight spelling errors with global TFIDF statistics so that very frequent words don't affect the score as much	2015-10-30 02:48:16 -04:00
Al	5076c0409b	[similarity] Adding an in-memory IDF index for weighted similarities	2015-10-29 13:33:01 -04:00
Al	1c543a5271	[osm/formatting] Adding is_in tags to the address formatter as they're common in OSM, aliasing addr:district to state_district instead of suburb	2015-10-29 12:30:56 -04:00
Al	c7df3fcb3a	[osm] Adding a list of various OSM name tags obtained from Nominatim	2015-10-29 11:44:56 -04:00
Al	bbd10e97bd	[fix] imports	2015-10-28 21:32:09 -04:00
Al	110451d6d6	[polygons] Polygon area calculations	2015-10-28 21:19:35 -04:00
Al	e946e63222	[polygons] Changing language polygon index to use new index_polygon method	2015-10-28 21:18:27 -04:00
Al	5fdbb7e832	[polygons] Adding a geohash polygon index which selects a prefix size based on the area of the polygon's bounding box	2015-10-28 21:17:33 -04:00
Al	7f5cf89e84	[transliteration] Not escaping right side transliteration rules	2015-10-27 12:24:38 -04:00
Al	e3b80534ba	[polygons] Adding methods for fixing polygons in base RTreePolygonIndex, moving the current polygon index to an instance variable and adding ability to import from a general GeoJSON-like structure instead of just shapefiles	2015-10-25 18:36:12 -04:00
Al	2d4b3a6e2f	[parser/formatting] Appendinge suburb between the road line and any subsequent lines for all bottom-up address formats. Effectively inserts neighborhoods into our version without making the OpenCage formats overly verbose. Also fixing post-format replace with group capture	2015-10-24 01:31:35 -04:00
Al	da53d7ebac	[osm] Adding an OSM neighborhoods/suburbs data set for matching with Quattroshapes boundaries, updating definitions for admin boundaries	2015-10-22 11:37:11 -04:00
Al	6478e65a06	[osm] Moving Wikipedia title normalization to osm.extract	2015-10-22 11:35:38 -04:00
Al	ff3a3c2201	[fix] disambiguation tokenizer to pypostal	2015-10-21 16:35:55 -04:00
Al	6f6d04966b	[fix] role in OSM polygon extraction	2015-10-21 16:35:25 -04:00
Al	336bfe32ca	[osm/formatter] Switching back to OpenCageData repo	2015-10-21 16:34:24 -04:00
Al	d0aa3b9109	[polygons/osm] alt_name	2015-10-20 09:27:55 -04:00
Al	218eae548c	[fix] logger	2015-10-20 06:43:51 -04:00
Al	1e8e592e0b	[fix] import	2015-10-19 23:30:12 -04:00
Al	e5129957f8	[osm/polygons] Add relation id to OSM reverse geocoder	2015-10-19 18:00:45 -04:00