5299 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
Patrick Nordahl	aba8131a6d	mention rust-postal as an available bindings library	2019-07-12 15:48:30 -05:00
Al	21601eb4b8	[fix/expand] adding test case for new per-language expansions. Also fixes #354	2019-02-16 22:31:59 -05:00
Al	42207769ef	[expand] adding a --root flag to the libpostal script for root-only expansion	2019-02-16 22:27:39 -05:00
Al	b7052caf6b	[dedupe] allow near-dupe hashes if only a small containing boundary is present (e.g. county/state district).	2019-02-16 22:26:13 -05:00
Al	3610ffaa05	[expand/dedupe] expansion with multiple languages (or multiple predicted languages) can sometimes produce weird string trees and thus either too many results or incorrect results, particularly for root expansions which we depend on for matching/deduping. Making one call per language identified. This may slightly affect performance on languages that are highly ambiguous (even that's doubtful, as libpostal usually identifies one or two languages with high accurracy and/or people are using a known geography) but should improve the results and was simpler implementation-wise than trying to use the single single string tree for multiple languages where, say, a two word phrase in one language might simply be token-space-token in another.	2019-02-16 22:20:36 -05:00
Al Barrentine	ea0878120c	Merge pull request #407 from openvenues/fix_transliteration ... fix for transliteration bug causing infinite loop or incorrect results in some expansions	2019-02-13 05:19:51 -05:00
Al	8a022698ee	[fix] transliteration bug causing potential infinite loop, closes #351, closes #378	2019-02-13 04:01:29 -05:00
Al Barrentine	af8b343109	Merge pull request #406 from openvenues/new_build_process ... New build process	2019-02-12 15:16:01 -05:00
Al	7a23d8922a	[build] data dir version handling	2019-02-12 14:10:02 -05:00
Al	3c0e97ae59	[fix] patch version also has to be an integer	2019-02-12 14:09:03 -05:00
Al	6a50eb7b22	[fix/build] base_data should currently be at v1.0.0 in terms of where the files are uploaded	2019-02-12 13:34:54 -05:00
Al	fdb3b7e32e	[build] fixing multi-part downloads, which no longer know the Content-Length so no longer know the byte ange for the last chunk. No longer needs to use any file dates/timestamps (compatibility 🎉), only the version numbers that are checked in to the repo	2019-02-12 13:33:08 -05:00
Al	acebc951b6	[fix] != instead of -ne	2019-02-12 12:32:34 -05:00
Al	a60aa9bfb1	[fix] seem to not be able to use a non-numeric version number in the Windows configure script, hopefully that fixes the Appveyor build	2018-12-29 20:16:14 -05:00
Al	9ae41dfaa4	[fix] base_data should be v1.1-alpha	2018-12-29 17:45:57 -05:00
Al	78fc457e75	[build/windows] attempting same changes to the Windows configure script	2018-12-13 14:39:24 -05:00
Al	9542c8c435	[build] adding libposatl_data.in which allows us to fetch data files from Github without using the API	2018-12-13 13:35:01 -05:00
Al	641395e811	[build] configure now uses the in-repo version strings and builds the libpostal_data script from a template	2018-11-18 11:53:46 -05:00
Al	b94519122e	[build] adding new in-repo version files for each of the data files so we don't have to use the Github API on Travis builds and a template version of the download script so that the version numbers can be configured based on those files.	2018-11-18 11:46:37 -05:00
Al Barrentine	eae01d3e60	Updating issue template to not use checkboxes	2018-05-24 16:55:03 -04:00
Al Barrentine	43795a3d90	Merge pull request #328 from openvenues/github_model_releases ... GitHub model releases	2018-03-11 03:55:53 -04:00
Al	027fbc5afc	[fix] filename	2018-03-11 03:26:37 -04:00
Al	b4a0c79d64	[fix] adding the tempfiles in the datadir, where the user must have permissions for this to work anyway	2018-03-10 22:24:03 -05:00
Al	15cb5f68ad	[fix] making this work with sh...wondering if supporting the few shells that don't have bash is worth losing array functionality.	2018-03-10 22:12:46 -05:00
Al	270bae7a32	[docs] adding the links to the Internet Archive versions of the training data as well as a file that contains the current parser training set date	2018-03-10 19:33:24 -05:00
Al	0c91379424	[build/fix] using Github for the model releases rather than the Mapzen S3 buckets which are no longer working after the shutdown. It requires a little more effort to get the metadata, but downloads should still be just as fast since Github releases are on S3 as well. Note: still need to implement the upload piece, but this at least provides a model endpoint for users.	2018-03-10 19:03:30 -05:00
Travis	fabd040860	[auto][ci skip] Adding data files from Travis build #381	2018-03-08 01:50:54 +00:00
Al Barrentine	6d41863068	Merge pull request #324 from veremenko-y/feature-update-street-types ... Updated street types for EN	2018-03-07 20:37:30 -05:00
Yaroslav Veremenko	c0b770101a	Removed final period	2018-03-07 17:51:34 -07:00
Yaroslav Veremenko	fd2b864b7d	Updated street types for EN	2018-03-07 12:48:30 -07:00
Al	6575bdc339	[dedupe/test] adding remaining options to near_dupe_test	2018-02-24 00:34:58 -05:00
Al	835de327c3	[dedupe] for near-dupe hashing, remove whitespace from root expansions so something like "Ocean Walk Dr" and "Oceanwalk Dr" will have a chance of matching downstream	2018-02-24 00:34:09 -05:00
Al	283be99b44	[numex] helper function to retrieve ordinal suffix lengths from a tokenized string for use in deduping	2018-02-24 00:31:26 -05:00
Al	b2dcb18d7e	[dedupe] account for missing ordinal suffixes in Soft-TFIDF deduping i.e. to count 1st Place and 1 Plce as the same where there might be a misspelling and the phrase wouldn't match under exact expansions	2018-02-23 23:44:05 -05:00
Al	b03fbdd681	[dedupe] adding multi-word phrase alignments to deduping	2018-02-23 01:28:02 -05:00
Al	591891951d	[utils] adding utf8 case insensitive comparison	2018-02-23 01:22:58 -05:00
Al	2b4e7073c2	[similarity] adding a multi-word alignmnet algorithm for streets and names like "de la cruz" vs. "dela cruz" or "Oceanwalk Ter" vs. "Ocean Walk Ter"	2018-02-23 01:22:12 -05:00
Al	c5bb9d8daa	[normalize/api] exposing normalize_string_languages and normalized_tokens_languages to the API for pre-normalizing numeric expressions at tokenization time	2018-02-22 18:47:36 -05:00
Al	0edb897143	[docs][ci skip] adding definition for the street_names.txt dictionary	2018-02-22 02:25:27 -05:00
Travis	cf2156c0e2	[auto][ci skip] Adding data files from Travis build #375	2018-02-22 03:32:16 +00:00
Al	afd225048a	[fix] DICTIONARY_STREET_NAME applies only to the street component, unlike synonyms, which would apply to any compnoent. This makes street names a good place to add synonyms found in streets that are not exactly thoroughfare types i.e. could not be removed from the string and have it retain more or less the same meaning.	2018-02-21 22:17:04 -05:00
Al	0f20613c13	[expand] using street name dictionaries as a possible root component instead of having to pollute the synonyms dictionary, which also affects the parser and might be a better place for general purpose synonyms affecting all components.	2018-02-21 22:16:07 -05:00
Al	ab67e0864a	[dictionaries] adding new street_names.txt dictionary and movign all the synonyms to there, generating the new dictionary type in address_dictionaries.py	2018-02-21 22:13:51 -05:00
Al	cd7bf81522	[dictionaries] reverting synonyms.txt back to previous commit	2018-02-21 22:13:12 -05:00
Travis	8075ee6f49	[auto][ci skip] Adding data files from Travis build #374	2018-02-22 01:21:23 +00:00
Al	a7b7568159	[dictionaries] removing "Corner" from the synonyms dictionaries for #320	2018-02-21 19:55:57 -05:00
Al	09408b1075	[fix] for regular expansion, use gazetteer components or overrides	2018-02-15 18:55:37 -05:00
Al	c5429de4d7	[fix] no gazetteer changes yet, breaks the parser	2018-02-15 18:52:16 -05:00
Al	ee2bac66d3	[build] rebuild dictionaries when gazteeer_data.c changes as well	2018-02-15 18:18:01 -05:00
Al	78d621ac85	[fix] adding street type gazetteer to name component as well for things like "24th St Cheese Co"	2018-02-15 18:13:27 -05:00