tommy/libpostal
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
1,131 Commits 2 Branches 0 Tags
596c5ffdd323e8ad1c286c1279663cb3840329e3
Commit Graph

501 Commits

Author SHA1 Message Date
Al
596c5ffdd3 [fix] Tokenized trie search 2015-12-05 15:21:52 -05:00
Al
24208c209f [parsing] Adding a training data derived index of complete phrases from suburb up to country. Only adding bias and word features for non phrases, using UNKNOWN_WORD and UNKNOWN_NUMERIC for infrequent tokens (not meeting minimum vocab count threshold). 2015-12-05 14:34:19 -05:00
Al
f41158b8b3 [osm] Avoid using the alternate name (e.g. Brooklyn instead of Kings County) when it is the same as city 2015-12-05 14:21:07 -05:00
Al
25e89bcc41 [fix] tokenized trie search edge case where tail is stored on the space node 2015-12-03 12:25:21 -05:00
Al
43287db90a [normalization/phrases] Fixing a bug which occurs with an already-separated elision 2015-12-02 16:04:39 -05:00
Al
746b5d0f34 [fix] transliterate using string_equals 2015-12-02 13:09:43 -05:00
Al
d0aaff1482 [utils] string_equals with NULL check 2015-12-01 13:12:08 -05:00
Al
f322ae0a1c [build] adding shuffle.c to Makefile rule 2015-12-01 11:28:33 -05:00
Al
b94264b745 [parser] Forgot to add shuffle.h/.c 2015-12-01 11:25:28 -05:00
Al
116fe857db [parser] gshuf (Mac equivalent of shuf) is quite a bit slower than shuf, so removing it. Need to train on Linux unless a better alternative is found for shuffling large files on Mac 2015-12-01 11:24:44 -05:00
Al
5f13041140 [parsing/build] Makefile changes for address parser 2015-11-30 14:51:43 -05:00
Al
4ca911baf8 [parsing] Adding a command-line client (with history) to test address parsing 2015-11-30 14:51:01 -05:00
Al
89677d94a3 [parsing] Initial commit of the address parser, training/testing, feature function, I/O 2015-11-30 14:48:13 -05:00
Al
e62eb1e697 [math] Matrix file I/O 2015-11-30 12:53:18 -05:00
Al
5682c347ac [fix] close file handle 2015-11-30 12:51:13 -05:00
Al
feab77970b [cli] Adding antirez's linenoise for command-line interfaces 2015-11-29 11:28:31 -05:00
Al
d3040036ec [fix] moving separator definitions 2015-11-28 13:53:13 -05:00
Al
094a5bf5f4 [dictionaries] adding Jnr and Snr forms for generational suffixes 2015-10-28 00:00:34 -04:00
Al
c2d112f4fc [fix] compile flags in Makefile.am 2015-10-27 19:01:37 -04:00
Al
6aaa08c220 [fix] Usage on libpostal_data script 2015-10-27 13:33:03 -04:00
Al
40918812e2 [normalize] Adding hyphen elimination as a string option (changes tokenization) 2015-10-27 13:32:47 -04:00
Al
3fe2365234 [fix] signed size_t in trie_set_tail 2015-10-27 13:21:26 -04:00
Al
ad59ba7a7b [fix] Re-generating transliteration tables 2015-10-27 12:28:08 -04:00
Al
1a1d74785c [fix] Compiler warnings for casts/printf 2015-10-26 18:52:18 -04:00
Al
6b456025b4 [fix] warnings in klib/ksort.h 2015-10-26 18:50:22 -04:00
Al
3b3513ffe3 [fix] warnings in collections.h/vector_math.h 2015-10-26 18:49:58 -04:00
Al
83c6a87ab1 [build] substitution for use of LIBPOSTAL_DATA_DIR in Makefile.am 2015-10-26 18:47:07 -04:00
Al
a319c1f6a0 [build] defining LIBPOSTAL_DATA_DIR in Autoconf rather than Automake, becomes part of config.h 2015-10-26 18:06:05 -04:00
Al
309d41a652 [math] adding matrix_zero method 2015-10-25 21:38:59 -04:00
Al
a2ad829d52 [math] matrix scalar arithmetic 2015-10-16 16:26:27 -04:00
Al
080ccf0ddd [fix] logging warnings in transliterate 2015-10-12 13:50:42 -05:00
Al
baef090793 [logging] Wrapping logging statements in a do while (0) so the compiler always at least sees debug code 2015-10-12 13:42:10 -05:00
Al
b88f237d82 [build] Adding separate Makefile target for downloading geodb 2015-10-11 22:27:25 -05:00
Al
588cf1df86 [build] Changing options to libpostal_data script to allow downloading geodb, uploaded first version to S3 2015-10-11 22:25:37 -05:00
Al
372e952cd3 [geodb] Adding some logging to geodb 2015-10-11 01:00:08 -05:00
Al
cb334b9fb1 [geodisambig] Shaving a few hundred more megabytes off of the geodb by only adding a single geohash prefix and not indexing the neighbors (query can use its neighbors) 2015-10-11 00:45:26 -05:00
Al
2394f817e4 [phrases] Fixing fallback at the end of a string in trie search 2015-10-11 00:13:21 -05:00
Al
29bc0fd11e [build] Makefile changes for the new geodb 2015-10-09 15:54:44 -04:00
Al
a6fbd48bec [geodb] geodb builder changes to support the new, more compact geodb 2015-10-09 15:53:56 -04:00
Al
bf596b9184 [utils] integer string sizes 2015-10-09 15:40:47 -04:00
Al
4dad121334 [fix] Initializing booleans in postal code constructor 2015-10-09 15:40:28 -04:00
Al
44da2e446b [geodb] Additional filenames and struct members in geodb.h 2015-10-09 15:37:10 -04:00
Al
67d128c386 [graph] graph_load and graph_save 2015-10-09 15:36:14 -04:00
Al
9fe2250521 [geodb] Using a trie for geo disambiguation features rather than the sparkey hashtable, sparkey simply contains the ids or code/country pairs in the case of postal codes 2015-10-09 15:35:50 -04:00
Al
cd6a0ab90b [geodb] Prefixing features with name for geo disambiguation (better trie compression) and removing the longer geohash prefix features 2015-10-09 15:16:08 -04:00
Al
77c4bb10c6 [utils] Adding kh_foreach_key 2015-10-09 11:51:32 -04:00
Al
1e98932b82 [fix] setting array->n after reading in both graph and sparse_matrix implementations 2015-10-06 19:28:28 -04:00
Al
5a231fb709 [graph] Builder for graphs not constructed in vertex-sorted order 2015-10-06 19:03:10 -04:00
Al
4984352eda [graph] Simple sparse graph implementation, essentially a sparse matrix with no values array 2015-10-06 18:58:18 -04:00
Al
3084fc929b [geodb] Was missing country boundary type in GeoDB causing some misses in parsing 2015-10-06 16:01:22 -04:00