libpostal

cmp

[geodisambig] C msgpack serialization dependency

2015-04-12 15:14:01 -04:00

geohash

[fix] Removing C++ checks from all but the main API functions

2015-08-07 17:15:39 -04:00

klib

[fix] warnings in klib/ksort.h

2015-10-26 18:50:22 -04:00

linenoise

[cli] Adding antirez's linenoise for command-line interfaces

2015-11-29 11:28:31 -05:00

log

[logging] Wrapping logging statements in a do while (0) so the compiler always at least sees debug code

2015-10-12 13:42:10 -05:00

murmur

[geodisambig] adding MurmurHash3, used by the Bloom filter

2015-03-11 17:47:57 -04:00

sparkey

[build] pulling in change from parser-data that allows user to pass CFLAGS

2016-12-21 14:39:27 -05:00

utf8proc

[strings] reverting to utf8proc v1.3.1, as 2.0 and above can chop off certain sequences

2017-01-01 20:03:23 -05:00

address_dictionary_builder.c

[expansion] using easier-to-access data structure for address dictionaries

2016-11-27 00:56:48 -08:00

address_dictionary.c

[merge] merging master

2017-01-13 19:58:49 -05:00

address_dictionary.h

[merge] merging master

2017-01-13 19:58:49 -05:00

address_expansion_data.c

[dictionaries] regenerating address_expansion_data.c from upstream changes

2017-01-01 14:26:54 -05:00

address_expansion_rule.h

[fix] Adding stdint.h include to most of the header files for portability

2015-08-07 02:43:44 -04:00

address_parser_cli.c

[fix] free json-encoded string in parser client output

2017-02-09 14:34:15 -05:00

address_parser_io.c

[fix] memory cleanup in address_parser_data_set, logging any bad input lines

2016-12-26 16:18:15 -05:00

address_parser_io.h

[parser] header changes for the data set struct

2016-12-09 13:37:45 -05:00

address_parser_test.c

[parser] address_parser_test memory cleanup, add print-errors option to print individual parser errors on held-out data

2017-02-12 16:05:11 -05:00

address_parser_train.c

[fix] var name in address_parser_train

2017-02-15 22:22:33 -05:00

address_parser.c

[parser] adding phrase features for category, unit, level, entrance, staircase, and po_box phrases from the libpostal dictionaries, excluding phrases which match the toponyms dictionary (e.g. US states that can also be found in street/venue names, useful for expansion but not here), if the current token is part of both an address dictionary phrase and a component phrase derived from the training data, use the longer of the two, or both if they are the same length

2017-02-17 03:00:48 -05:00

address_parser.h

[parser] adding long-context features which help classify the first token in the string by finding the relative positions of a) the first numeric token and b) the first street-level phrase like "Ave" or "Calle"

2017-02-14 18:42:51 -05:00

averaged_perceptron_tagger.c

[fix] Labels in averaged perceptron tagger

2015-09-29 13:06:34 -04:00

averaged_perceptron_tagger.h

[doc] Averaged perceptron tagger

2015-09-23 19:37:55 -04:00

averaged_perceptron_trainer.c

[merge] merging master

2017-01-13 19:58:49 -05:00

averaged_perceptron_trainer.h

[mv] Moving trie_new_from_hash to a module

2016-01-05 16:43:17 -05:00

averaged_perceptron.c

[parser] using new zeros API in averaged_perceptron.c

2017-02-19 14:02:54 -08:00

averaged_perceptron.h

[parser] Averaged perceptron training

2015-09-10 10:26:24 -07:00

bench.c

[api] Default options accessible through getters, not static structs

2016-02-15 17:34:00 -05:00

bloom.c

[fix] calloc instead of malloc when performing initialization on structs that may fail halfway and need to clean up while partially initialized (calloc will set all the bytes to zero so the member pointers are NULL instead of garbage memory)

2017-01-13 18:30:04 -05:00

bloom.h

[fix] Adding stdint.h include to most of the header files for portability

2015-08-07 02:43:44 -04:00

chunked_shuffle

[utils] adding a chunked shuffle as the concatenated file sizes may get larger than memory

2016-11-21 14:04:34 -05:00

collections.h

[utils] fixing key_type in hash_get, adding int64_double map

2017-02-15 22:20:36 -05:00

constants.h

[constants] max country code length

2015-09-19 01:39:58 -04:00

features.c

[utils] char_array_add_vjoined to stay consistent (add_* methods NUL termiante)

2017-01-09 16:10:07 -05:00

features.h

[features] No copy versions of feature counts functions

2016-01-05 16:39:50 -05:00

file_utils.c

[merge] merging master

2017-01-13 19:58:49 -05:00

file_utils.h

[merge] merging master

2017-01-13 19:58:49 -05:00

float_utils.c

[math] Floating point equality with relative epsilon comparisons

2016-01-02 15:39:49 -05:00

float_utils.h

[math] Floating point equality with relative epsilon comparisons

2016-01-02 15:39:49 -05:00

gazetteer_data.c

[gazetteers] adding ADDRESS_PO_BOX component

2017-02-15 22:23:28 -05:00

gazetteers.h

[dictionaries] Making new component for near/nearby prepositions

2016-07-21 17:04:57 -04:00

geo_disambiguation.c

[geodisambig] Shaving a few hundred more megabytes off of the geodb by only adding a single geohash prefix and not indexing the neighbors (query can use its neighbors)

2015-10-11 00:45:26 -05:00

geo_disambiguation.h

[geodisambig] Shaving a few hundred more megabytes off of the geodb by only adding a single geohash prefix and not indexing the neighbors (query can use its neighbors)

2015-10-11 00:45:26 -05:00

geodb_builder.c

[merge] merging master

2017-01-13 19:58:49 -05:00

geodb.c

[fix] calloc instead of malloc when performing initialization on structs that may fail halfway and need to clean up while partially initialized (calloc will set all the bytes to zero so the member pointers are NULL instead of garbage memory)

2017-01-13 18:30:04 -05:00

geodb.h

[geodb] Additional filenames and struct members in geodb.h

2015-10-09 15:37:10 -04:00

geonames_fields.h

[geonames] Re-generating postal/geonames fields headers

2015-07-08 17:02:59 -04:00

geonames.c

[fix] Check the return of malloc() in geonames.c

2016-02-25 14:53:31 -06:00

geonames.h

[geodb] Adding separate bitset for geonames place types and using NFC normalization instead of NFD (requires retraining)

2016-07-21 17:04:57 -04:00

graph_builder.c

[geodb] Additional filenames and struct members in geodb.h

2015-10-09 15:37:10 -04:00

graph_builder.h

[geodb] Additional filenames and struct members in geodb.h

2015-10-09 15:37:10 -04:00

graph.c

[fix] calloc instead of malloc when performing initialization on structs that may fail halfway and need to clean up while partially initialized (calloc will set all the bytes to zero so the member pointers are NULL instead of garbage memory)

2017-01-13 18:30:04 -05:00

graph.h

[graph] graph_load and graph_save

2015-10-09 15:36:14 -04:00

json_encode.c

[api] Simple JSON encoding for strings, UTF-8 rather than Unicode

2015-12-17 12:25:05 -05:00

json_encode.h

[api] Simple JSON encoding for strings, UTF-8 rather than Unicode

2015-12-17 12:25:05 -05:00

language_classifier_cli.c

[language_classification] Making directory optional on language_classifier client/test program

2016-01-27 03:18:53 -05:00

language_classifier_io.c

[language_classification] Automatic hyperparameter optimization using either the cross-validation set or two distinct subsets of the training set

2016-01-17 21:11:37 -05:00

language_classifier_io.h

[language_classification] Automatic hyperparameter optimization using either the cross-validation set or two distinct subsets of the training set

2016-01-17 21:11:37 -05:00

language_classifier_test.c

[language_classification] Making directory optional on language_classifier client/test program

2016-01-27 03:18:53 -05:00

language_classifier_train.c

[math] Generic dense matrix implementation using BLAS calls for matrix-matrix multiplication if available

2016-08-06 00:40:01 -04:00

language_classifier.c

[merge] merging master

2017-01-13 19:58:49 -05:00

language_classifier.h

[math] Generic dense matrix implementation using BLAS calls for matrix-matrix multiplication if available

2016-08-06 00:40:01 -04:00

language_features.c

[language_classification] Adding script feature for any non-Latin script. Even if the script doesn't directly identify the language, it can act as a modified intercept (all Han script addresses will share the Han feature, even if we haven't seen one of the > 80k Han characters)

2016-01-17 21:37:45 -05:00

language_features.h

[language_classifier] Features for address languages classification, quadgrams for most languages, unigrams for ideographic characters, script for single-script languages like Thai, Hebrew, etc.

2016-01-09 03:42:57 -05:00

libpostal_config.h

[build] adding libpostal_setup_datadir, libpostal_setup_parser_datadir, libpostal_setup_language_classifier_datadir functions for configuring the datadir at runtime

2017-01-09 16:11:26 -05:00

libpostal_data

[fix] Use UTC date reference to avoid repeating S3 downloads.

2016-12-26 12:04:02 +11:00

libpostal.c

[api] include some of the new components in default address_components for the libpostal expansion API

2017-02-15 22:29:22 -05:00

libpostal.h

[gazetteers] adding ADDRESS_PO_BOX component

2017-02-15 22:23:28 -05:00

logistic_regression_trainer.c

[math] Generic dense matrix implementation using BLAS calls for matrix-matrix multiplication if available

2016-08-06 00:40:01 -04:00

logistic_regression_trainer.h

[math] Generic dense matrix implementation using BLAS calls for matrix-matrix multiplication if available

2016-08-06 00:40:01 -04:00

logistic_regression.c

[math] Generic dense matrix implementation using BLAS calls for matrix-matrix multiplication if available

2016-08-06 00:40:01 -04:00

logistic_regression.h

[math] Generic dense matrix implementation using BLAS calls for matrix-matrix multiplication if available

2016-08-06 00:40:01 -04:00

logistic.c

[math] Generic dense matrix implementation using BLAS calls for matrix-matrix multiplication if available

2016-08-06 00:40:01 -04:00

logistic.h

[math] Generic dense matrix implementation using BLAS calls for matrix-matrix multiplication if available

2016-08-06 00:40:01 -04:00

main.c

[api] Default options accessible through getters, not static structs

2016-02-15 17:34:00 -05:00

Makefile.am

[merge] merging master

2017-01-13 19:58:49 -05:00

matrix.h

[math] Generic dense matrix implementation using BLAS calls for matrix-matrix multiplication if available

2016-08-06 00:40:01 -04:00

minibatch.c

[fix] NULL check

2016-01-09 01:43:25 -05:00

minibatch.h

[features] Functions for dealing with minibatches

2016-01-08 00:48:11 -05:00

msgpack_utils.c

[utils] cstring_array_terminate, moving msgpack_utils to separate file

2015-07-25 18:41:02 -04:00

msgpack_utils.h

[fix] Adding stdint.h include to most of the header files for portability

2015-08-07 02:43:44 -04:00

ngrams.c

[ngrams] changing args to ngrams

2016-12-21 18:09:45 -05:00

ngrams.h

[ngrams] changing args to ngrams

2016-12-21 18:09:45 -05:00

normalize.c

[utils] string_replace_char does single byte/character replacement, new string_replace to do full string replacement, again using char_array for safety, string_replace_with_array function for memory reuse

2017-02-17 13:58:51 -05:00

normalize.h

[normalization] Adding NORMALIZE_STRING_SIMPLE_LATIN_ASCII option so parser can normalize punctuation and HTML entities, etc. without touching the alphanumeric parts of the original input

2016-08-21 19:45:32 -04:00

numex_data.c

[numex] regenerated numex data file

2016-11-30 15:58:55 -08:00

numex_rule.h

[numex] Separating rules from keys for Linux gcc compilation

2015-08-09 01:00:57 -04:00

numex_table_builder.c

[fix] numex_table builder program using new API (heap-allocated strings)

2016-03-02 16:28:28 -05:00

numex.c

[fix] calloc instead of malloc when performing initialization on structs that may fail halfway and need to clean up while partially initialized (calloc will set all the bytes to zero so the member pointers are NULL instead of garbage memory)

2017-01-13 18:30:04 -05:00

numex.h

[build] adding libpostal_setup_datadir, libpostal_setup_parser_datadir, libpostal_setup_language_classifier_datadir functions for configuring the datadir at runtime

2017-01-09 16:11:26 -05:00

postal_fields.h

[geonames] Re-generating postal/geonames fields headers

2015-07-08 17:02:59 -04:00

scanner.c

[tokenization] Re-generating scanner

2016-07-21 17:04:57 -04:00

scanner.h

[fix] stdint include

2015-09-20 20:10:47 -04:00

scanner.re

[tokenization] Hyphens, etc. between non-ASCII digits (e.g. Unicode full-width numbers) should be single tokens

2016-07-21 17:04:57 -04:00

shuffle.c

[parser] Forgot to add shuffle.h/.c

2015-12-01 11:25:28 -05:00

shuffle.h

[parser] Forgot to add shuffle.h/.c

2015-12-01 11:25:28 -05:00

sparse_matrix_utils.c

[fix] sparse_matrix_new_from_matrix uses new matrix types

2016-11-30 10:15:12 -08:00

sparse_matrix_utils.h

[math] Generic dense matrix implementation using BLAS calls for matrix-matrix multiplication if available

2016-08-06 00:40:01 -04:00

sparse_matrix.c

[merge] merging master

2017-01-13 19:58:49 -05:00

sparse_matrix.h

[math] Generic dense matrix implementation using BLAS calls for matrix-matrix multiplication if available

2016-08-06 00:40:01 -04:00

stochastic_gradient_descent.c

[math] Generic dense matrix implementation using BLAS calls for matrix-matrix multiplication if available

2016-08-06 00:40:01 -04:00

stochastic_gradient_descent.h

[math] Generic dense matrix implementation using BLAS calls for matrix-matrix multiplication if available

2016-08-06 00:40:01 -04:00

string_utils.c

[utils] string_replace_char does single byte/character replacement, new string_replace to do full string replacement, again using char_array for safety, string_replace_with_array function for memory reuse

2017-02-17 13:58:51 -05:00

string_utils.h

[utils] string_replace_char does single byte/character replacement, new string_replace to do full string replacement, again using char_array for safety, string_replace_with_array function for memory reuse

2017-02-17 13:58:51 -05:00

token_types.h

[fix] is_numeric_token includes IDEOGRAPHIC_NUMBER

2017-02-12 15:11:56 -05:00

tokens.c

[fix] calls and NULL checks

2016-12-12 11:37:27 -05:00

tokens.h

[tokenization] tokenized string with source

2015-10-04 18:27:04 -04:00

transliterate.c

[merge] merging master

2017-01-13 19:58:49 -05:00

transliterate.h

[merge] merging master

2017-01-13 19:58:49 -05:00

transliteration_data.c

[fix] also fix german-ascii transliteration on uppercase U with umlaut

2017-01-05 04:07:29 -05:00

transliteration_rule.h

[transliteration] Using revisit strings instead of keeping a backtrack count so we don't have to later map logical characters to the actual string, removing any duplicate keys in the table builder so that if any rules happen to overlap within a step, the first will take precedence

2015-05-29 16:54:05 -04:00

transliteration_scripts_data.c

[transliteration] regenerated transliteration_scripts_data.c

2017-01-02 13:52:48 -05:00

transliteration_table_builder.c

[logging] converting most of the steps in building the transliteration table to use debug logging

2017-01-02 00:41:11 -05:00

trie_search.c

[phrases] fixing trie suffix search

2017-02-14 03:36:29 -05:00

trie_search.h

[phrases] adding token_phrase_memberships to trie_search for reuse

2017-02-08 01:59:39 -05:00

trie_utils.c

[mv] Moving trie_new_from_hash to a module

2016-01-05 16:43:17 -05:00

trie_utils.h

[mv] Moving trie_new_from_hash to a module

2016-01-05 16:43:17 -05:00

trie.c

[fix] calloc instead of malloc when performing initialization on structs that may fail halfway and need to clean up while partially initialized (calloc will set all the bytes to zero so the member pointers are NULL instead of garbage memory)

2017-01-13 18:30:04 -05:00

trie.h

[fix] All file_read_uint64 calls that use stack variables read into a uint64_t not a size_t so as not to smash the stack under a 32-bit arch (issue #18 )

2016-02-29 22:36:00 -05:00

unicode_script_types.h

[unicode] Regenerating unicode_script_data.c

2016-01-17 20:53:44 -05:00

unicode_scripts_data.c

[unicode] Regenerating unicode_script_data.c

2016-01-17 20:53:44 -05:00

unicode_scripts.c

[unicode] Adding SCRIPT_INHERITED as a common script so diacritics like COMBING CEDILLA don't break the current script and produce false word breaks

2016-01-11 16:39:21 -05:00

unicode_scripts.h

[transliteration] Regenerating transliteration data files

2015-09-23 04:04:38 -04:00

vector_math.h

[math] Adding fast SIMD exponent using the Remez algorithm for vectorized exp

2016-08-06 00:31:16 -04:00

vector.h

[merge] merging master

2017-01-13 19:58:49 -05:00