Commit Graph

233 Commits

Author SHA1 Message Date
Al
efb0414cee [fix] renaming a few keys in fetch_osm_address_data.sh to use the convention expected by definitions module 2016-07-21 17:04:57 -04:00
Al
3d0d38f8da [osm] Adding OSM definitions to quickly determine if a set of properties meets one of the libpostal definitions (valid amenity, etc.) 2016-07-21 17:04:57 -04:00
Al
d595e48721 [polygons] Adding university and hospital polygons to the subdivisions index, (multiple sub-buildings but still want to know the parent entity) 2016-07-21 17:04:57 -04:00
Al
72fad2f5e3 [fix] six.iteritems 2016-07-21 17:04:57 -04:00
Al
b8513c0dc6 [osm] Adding osm_type_and_id function for handling all-to-nodes output from osmfilter. Using in neighborhoods as well as admin rtree. 2016-07-21 17:04:57 -04:00
Al
ad81095879 [osm] moving osm_address_components to its own module 2016-07-21 17:04:57 -04:00
Al
6cec1d99f7 [fix] import 2016-07-21 17:04:57 -04:00
Al
f00e425891 [osm] Adding parse_osm_number_range for addr:flats and addr:unit 2016-07-21 17:04:57 -04:00
Al
c8ea12e1eb [osm] Adding place=city/town/village/hamlet/municipality to admin borders data set 2016-07-21 17:04:57 -04:00
Al
99e634aaba [fix] some weirdness with the dateline and polygons that have a longitude of exactly 180.0 2016-07-21 17:04:57 -04:00
Al
a94debc4ed [osm] addr:place can be used for street name, expanded building polygon definitions, fixing boundary polygons 2016-07-21 17:04:57 -04:00
Al
03704fff6a [intersections] Lower memory version of intersection freader 2016-07-21 17:04:57 -04:00
Al
f3bbe2ee74 [fix] file rename 2016-07-21 17:04:57 -04:00
Al
9977a7a254 [mv] Moving osm_admin_boundaries to just admin_boundaries 2016-07-21 17:04:57 -04:00
Al
79368f3f02 [intersections] Intersections generator for OSM 2016-07-21 17:04:57 -04:00
Al
0f0af1f295 [osm/polygons] Adding properties in building polygons 2016-07-21 17:04:57 -04:00
Al
72ee2e00ae [osm] Moving OSM boundaries to YAML files instead of JSON for consistency 2016-07-21 17:04:57 -04:00
Al
1f52f8ddcc [osm/polygons] Same check for closed ways as for relations in OSM polygon readers 2016-07-21 17:04:57 -04:00
Al
2f862ca0ec [osm] Adding place=plot to subdivisions data set 2016-07-21 17:04:57 -04:00
Al
8db7f139ba [osm] Adding building polygon reader, including closed ways for admin polys 2016-07-21 17:04:57 -04:00
Al
12a688df36 [osm] Splitting out generic amenities like ATM, fuel, restrooms, etc. so they can be used in category queries. Adding subdivision polygons, postcode polygons, building polygons, adding a few types of place keys to venues data set 2016-07-21 17:04:57 -04:00
Al
fc689222da [osm] adding civil boundaries (e.g. postal areas in Dublin), fixing output files 2016-07-21 17:04:57 -04:00
Al
2b4a9f0962 [osm] Splitting category queries data into several files (amenities, buildings, natural features, waterways) 2016-07-21 17:04:57 -04:00
Al
b25682e761 [polygons/zones] Adding a polygon reader for OSM zones (named residential/commercial/industrial/military areas) which are closed ways and can be used in addresses e.g. in office parks, larger housing complexes, etc. 2016-07-21 17:04:57 -04:00
Al
ac18e383bd [osm] Building OSM file for deriving category queries, zone data for including the names of residential, commercial and industrial areas in the parser. Named landuse and historic features are considered valid places/venues. 2016-07-21 17:04:57 -04:00
Al
af73bb300d [fix] Adding islands to admin borders 2016-07-21 17:04:57 -04:00
Al
7696179843 [osm] Removing generic amenities like ATMs, parking, restrooms, etc. from addresses but keeping them in venues to support generic queries 2016-03-14 01:07:03 -04:00
Al
c5498c6c0c [osm] Incorporating airports, and only including certain values for tourism= and leisure= since not all are physical place types, adding building= to addresses 2016-03-12 15:02:31 -05:00
Al
a71fa7bd8d [osm] tourism= keys should only be included in some cases. Listing everything on taginfo with >= 100 uses 2016-03-10 14:17:38 -05:00
Al
d43fe201ff [osm] No longer requiring street name in OSM planet addresses. Adding leisure and tourism keys to capture things like parks, squares, etc. Adding place=locality for neighborhoods. 2016-03-09 18:19:33 -05:00
Al
00ce71223f [osm] Using the default probabilities for abbreviations in ways training data 2016-01-24 00:53:41 -05:00
Al
bab7a0f961 [osm] splitting streets (way names) on semicolons 2016-01-24 00:42:25 -05:00
Al
7646adfc0f [osm] Adding abbreviated street names in addition to the originals 2016-01-23 23:23:58 -05:00
Al
67130383ce [fix] converting semicolons to commas in OSM house numbers and picking one at random 2016-01-23 23:16:19 -05:00
Al
1bb797f783 [fix] spacing in phrases 2016-01-23 21:59:49 -05:00
Al
3a8c3dfcf6 [fix] spacing in phrases at end of string 2016-01-23 21:51:40 -05:00
Al
78450bfad9 [fix] Spaces in abbreviation 2016-01-23 21:36:20 -05:00
Al
308ceb5a5f [fix] convert UTF8 slices back to unicode before using with the Python trie 2016-01-23 20:20:23 -05:00
Al
5eb6bb309b [fix] Only adding whitespace back into tokenized strings during abbreviation if it existed in the original string 2016-01-23 20:09:45 -05:00
Al
d61207e95a [fix] var name 2016-01-23 18:01:02 -05:00
Al
e44cba1d06 [fix] geonames db not required in OSM training data 2016-01-23 17:59:55 -05:00
Al
4f03711e60 [osm] Adding abbreviated training examples to ways language training data 2016-01-23 14:10:47 -05:00
Al
c9fb4ee69d [osm/formatting] Dropping state more often than not, except in the US and Canada where those fields are more commonly used 2016-01-22 17:58:24 -05:00
Al
ea9bb3f2d5 [fix] Abbreviation probabilities should only apply once, not once per dictionary. Also fixing issues where some of the abbreviations were doubled 2016-01-22 15:48:21 -05:00
Al
f9f6558e06 [fix] simple whitespace field splits for the limited format training data (used for language classification) 2016-01-22 04:34:42 -05:00
Al
cd1db7b288 [fix] Making sure rare components are dropped first, adding state and country back in 2016-01-22 04:17:19 -05:00
Al
adc3a00264 [fix] var name 2016-01-22 04:10:16 -05:00
Al
261beffa36 [fix] Actually better to remove country and state from rare components and let them use the standard dropout probabilities 2016-01-22 04:00:45 -05:00
Al
a6cc3d0114 [fix] Adding state to the more frequently dropped components 2016-01-22 03:56:38 -05:00
Al
bca3dae004 [fix] state full name probabilities for limited vs. full formatted OSM training sets 2016-01-22 03:54:20 -05:00