Commit Graph

212 Commits

Author SHA1 Message Date
Al
293587bae9 [addresses] adding new config for postal codes around the world. Allows appending the ISO alpha-2 country code to the beginning of the postcode as in e.g. SI-1000 (only used if the postcode begins with a digit). This system was used for postal codes in continental Europe as a recommendation from the CEPT. Now 7 member states still use it, so in those countries add the country-code with higher probability. The config also contains the license plate codes for countries where e.g. L-1234 might be used instead of LU-1234. Allows configuring in which countries postcodes should be validated using Google's per-country validation regexes (and the ability to override with a custom regex), and in which countries other admin component names should be stripped. 2017-02-10 23:53:50 -05:00
Al
7a360f4211 [osm] addr:postcode can be all over the place in OSM. Start with postcodes containing commas or semicolons. If addr:postcode (on address of building) contains either, iterate over the values and pick the first one that matches a postcode validation regex for that country 2017-02-08 16:13:29 -05:00
Al
01d6d47b08 [osm] removing addr:place mapping to road as it's usually a village in post-Soviet states, etc. Can handle it down the road 2017-01-27 13:54:08 -05:00
Al
11345bf2bf [osm] using new constants in OSM formatting as well 2017-01-27 13:53:00 -05:00
Al
bc748b6d62 [addresses] supplying country arg when stripping name affixes both for OSM place-based data sets (ways, localities) and OpenAddresses (shouldn't affect any of the countries currently in OA though) 2017-01-23 23:30:33 -05:00
Al
a931c5ddc9 [osm] checking for valid street names in OSM street-only training data so e.g. the street name is not just a simple number like "831" 2017-01-19 02:34:29 -05:00
Al
05568194aa [fix] var initialization II 2017-01-18 01:54:18 -05:00
Al
b19ab0ae48 [fix] var initialization 2017-01-18 01:48:02 -05:00
Al
d498fa893c [fix] name 2017-01-16 22:15:25 -05:00
Al
8566cb4054 [addresses] refactoring place component cleanup into a method that can be reused with the place and ways training data 2017-01-16 20:43:55 -05:00
Al
35dbce59d2 [osm] base case for default_language, applying the ways/relations requirement again as the nodes are mostly motorway_junction and can often be just a city name, etc. 2017-01-16 19:10:27 -05:00
Al
96a98fc63c [fix] var name II 2017-01-16 18:57:29 -05:00
Al
582d042e95 [fix] var name 2017-01-16 18:56:20 -05:00
Al
b28728b017 [fix] tuple 2017-01-16 18:53:40 -05:00
Al
42b0a4cf68 [fix] var name 2017-01-16 18:46:08 -05:00
Al
4902e88b81 [fix] formatted OSM ways training data should use nodes as well as ways/relations 2017-01-16 18:39:53 -05:00
Al
449154d624 [fix] arg 2017-01-16 15:34:38 -05:00
Al
be763539d3 [fix] remove var 2017-01-16 15:31:26 -05:00
Al
8c92013c43 [fix] args to way_names 2017-01-16 15:29:16 -05:00
Al
a0150f37d0 [osm] better lat/lon conversion for admin_center point 2017-01-14 17:48:37 -05:00
Al
c7e644ca51 [fix] validating number ranges in extract_valid_postcodes as well 2017-01-12 14:09:33 -05:00
Al
59ed268558 [osm] require name tag for formatted places 2017-01-12 13:00:07 -05:00
Al
b90d88db3e [fix] import 2017-01-12 12:08:40 -05:00
Al
ba0f097d78 [boundaries] adding check for valid name key in formatted places, and removing short_name from the Sao Paulo relation as well 2017-01-12 12:05:42 -05:00
Al
122d7b2b79 [fix] only using the revised address components for CLDR country name 2017-01-12 02:33:16 -05:00
Al
88a80f4e30 [fix] using normalized tags throughout in OSM formatted place data 2017-01-12 02:25:17 -05:00
Al
bec569adaa [osm] adding new validity check to venue names so if the Jaccard(name tokens, street & house numer tokens) == 1 and the address does not have a known venue type e.g. a restaurant, the "venue name" is actually just the street address and can be discarded 2017-01-11 16:23:42 -05:00
Al
828b67d4f7 [osm] adding some new training data for simple road names and their surrounding admin boundaries 2017-01-07 15:34:43 -05:00
Al
6163dbae39 [osm/places] adding option to only format place tags for city and smaller admins, using for polygons as larger polys should be included elsewhere anyway 2016-12-27 03:37:15 -05:00
Al
8abbb273b2 [osm] adding the excellent ftfy (https://github.com/LuminosoInsight/python-ftfy) to fix Mojibake, etc. in address components 2016-12-26 21:18:14 -05:00
Al
e31ab33fe0 [fix] kwarg 2016-12-25 03:19:41 -05:00
Al
a185441ffa [osm] adding amenity=post_office to the generic place types (shouldn't be added as venue unless there's a known phrase in the name) 2016-12-25 03:14:07 -05:00
Al
e4e86261d1 [addresses/JP] just remove addr:neighborhood, addr:quarter, etc. in Japan as they're not applied consistently outside of cities 2016-12-24 20:13:48 -05:00
Al
9928d249a6 [addresses/JP] combining addr:quarter and addr:neighbourhood in Japan (based on info in https://wiki.openstreetmap.org/wiki/JA:%E4%BD%8F%E6%89%80) 2016-12-24 19:54:54 -05:00
Al
32e7637037 [fix] handle case where addr:conscriptionnumber exists but not addr:housenumber 2016-12-21 01:54:07 -05:00
Al
3b14613f1d [fix] restore original house number for subsequent formatting after addr:conscriptionnumber/addr:streetnumber 2016-12-21 00:51:44 -05:00
Al
484c7ef912 [osm] adding addresses with addr:conscriptionnumber and addr:streetnumber when available 2016-12-21 00:36:40 -05:00
Al
f2720db2f8 [osm] adding simple street name normalization for certain streets in OSM that also contain the house number (only when separated by commas and in a country/language where house number comes after street). There are other cases for normalization but need to better define them. 2016-12-19 02:13:44 -05:00
Al
bf3e9749ca [osm] during place formatting, add point-based cities for any places/polygons that are smaller than cities e.g. suburb or city_district, use admin_center as the point for reverse geocoding if available (instead of representative_point() which can be expensive or centroid which can be inaccurate) 2016-12-12 05:29:39 -05:00
Al
8f30987bdf [fix] checking if building is a rail station 2016-12-09 02:57:47 -05:00
Al
da36b71829 [addresses] adding new places index in OSM and OpenAddresses training data 2016-12-05 18:36:17 -05:00
Al
adab232674 [osm] don't include rail stations with no venue phrases (if there's a railway station at Foo, only include it if it's named "Foo Station", not just plain "Foo") 2016-12-01 02:03:38 -05:00
Al
cdbc102821 [boundaries] in addition to population, check if a city has an unambiguous Wikipedia 2016-11-25 13:36:49 -08:00
Al
5c3ccc3bc6 [places] better handling of population exceptions in places config 2016-11-25 00:38:49 -08:00
Al
a596d03309 [fix] return values 2016-11-19 12:45:39 -05:00
Al
e15036fcce [fix] if there are street types that are not venue words and not vice versa, then call the venue invalid as a standalone term 2016-11-19 04:11:33 -05:00
Al
8e905fd17d [fix] if no venue names are passed in to formatted_addresses_with_venue_names, remove any existing venue name from the components as well 2016-11-19 03:46:16 -05:00
Al
e6fe576ec7 [fix] var 2016-11-19 03:15:23 -05:00
Al
1f50481cad [fix] args 2016-11-19 03:14:06 -05:00
Al
4d14f80f0c [osm] using the new gazetteer methods to do more thorough checks on single house names (if there are no other components than the standalone venue name, make sure it contains venue words like {library, bar}, etc. and not street type words like {road, street}, etc. so we don't get training examples that are simply "Abbey/house Road/house" with no house number or street name). If the venue name equals the street name or house number, drop it. Same if the venue name equals one of the admin components and no house number or street is present. If the venue name is numeric, require both a house number and a street name. 2016-11-19 03:12:24 -05:00