libpostal

cmp

[geodisambig] C msgpack serialization dependency

2015-04-12 15:14:01 -04:00

geohash

[utils] geohash

2015-03-03 18:51:49 -05:00

klib

[fix] fixing some formatting

2015-03-03 12:54:27 -05:00

log

[fix] fixing some formatting

2015-03-03 12:54:27 -05:00

murmur

[geodisambig] adding MurmurHash3, used by the Bloom filter

2015-03-11 17:47:57 -04:00

utf8proc

[unicode] Upgrading to JuliaLang's utf8proc (Unicode 7, maintained)

2015-03-17 12:20:08 -04:00

bloom.c

[geodisambig] Bloom filter implementation for quick probabilistic set membership tests before hitting disk. 100% recall and bounded precision, saves disk seeks for keys that definitely do not exist (useful for Geonames disambiguation-related lookups and in-process deduping).

2015-03-11 17:47:15 -04:00

bloom.h

[geodisambig] Bloom filter implementation for quick probabilistic set membership tests before hitting disk. 100% recall and bounded precision, saves disk seeks for keys that definitely do not exist (useful for Geonames disambiguation-related lookups and in-process deduping).

2015-03-11 17:47:15 -04:00

collections.h

[utils] two set types in collections.h

2015-04-19 09:32:53 -04:00

config.h

[config] Adding config header

2015-06-01 15:40:59 -04:00

constants.h

[utils] constants.h

2015-06-02 12:26:19 -04:00

features.c

[utils] More consistent naming in string_utils

2015-03-27 21:12:08 -04:00

features.h

[utils] More consistent naming in string_utils

2015-03-27 21:12:08 -04:00

file_utils.c

[utils] using unsigned ints in file_utils, adding doubles

2015-05-27 16:03:36 -04:00

file_utils.h

[utils] using unsigned ints in file_utils, adding doubles

2015-05-27 16:03:36 -04:00

gazetteers.h

[fix] gazetteers typo

2015-06-12 17:26:14 -04:00

geonames.c

[geodisambig] Models for geonames with msgpack serialization/deserialization

2015-04-12 16:47:01 -04:00

geonames.h

[geodisambig] Models for geonames with msgpack serialization/deserialization

2015-04-12 16:47:01 -04:00

numex_data.c

[numex] Re-generated numex data file

2015-06-12 16:15:53 -04:00

numex_rule.h

[numx] header and table builder changes to support whole words languages

2015-06-12 16:10:57 -04:00

numex_table_builder.c

[numx] header and table builder changes to support whole words languages

2015-06-12 16:10:57 -04:00

numex.c

[numex] Working numex implemenation. Tested on most languages, Germanic, Latin/whole_tokens_only, English concatenated or with separators, French numerals like quatre-vignt-douze, Spanish multiple-token ordinals, Japanese numerals, etc. All looking good

2015-06-12 16:21:36 -04:00

numex.h

[numx] header and table builder changes to support whole words languages

2015-06-12 16:10:57 -04:00

scanner.c

[tokenization] Re-generating scanner

2015-06-16 12:52:37 -04:00

scanner.h

[tokenization] Adding url, email, US/international phone numbers, a separate type for ideographic numbers, more general quotes, paren types

2015-03-24 16:43:53 -04:00

scanner.re

[tokenization] Adding a Hangul syllable class in tokenization for syllables written out as Jamo

2015-06-16 12:52:04 -04:00

string_utils.c

[utils] utf8 comparison can handle a non-valid UTF-8 sequence e.g. for trie suffix comparison where we may be in the middle of a multi-byte character. Adding a standard utf8_common_prefix method

2015-06-12 16:11:40 -04:00

string_utils.h

[utils] utf8 comparison can handle a non-valid UTF-8 sequence e.g. for trie suffix comparison where we may be in the middle of a multi-byte character. Adding a standard utf8_common_prefix method

2015-06-12 16:11:40 -04:00

token_types.h

[tokenization] Adding a Hangul syllable class in tokenization for syllables written out as Jamo

2015-06-16 12:52:04 -04:00

tokens.c

[fix] better allocation sizes for tokenized strings

2015-04-05 22:02:31 -04:00

tokens.h

[fix] type name/import

2015-04-01 02:54:14 -04:00

transliterate.c

[fix] transliteration states

2015-06-05 00:09:29 -04:00

transliterate.h

[numex] Using the new representation for generated data

2015-06-02 12:28:07 -04:00

transliteration_data.c

[transliteration] New ggenerated data file with the German/Scandinavian additions

2015-06-03 22:56:50 -04:00

transliteration_rule.h

[transliteration] Using revisit strings instead of keeping a backtrack count so we don't have to later map logical characters to the actual string, removing any duplicate keys in the table builder so that if any rules happen to overlap within a step, the first will take precedence

2015-05-29 16:54:05 -04:00

transliteration_scripts.h

[transliteration] New transliterator_scripts file

2015-05-31 02:09:28 -04:00

transliteration_table_builder.c

[fix] only count non-set chars in parse_groups

2015-05-29 19:42:05 -04:00

trie_search.c

[fix] trie_search logging

2015-06-12 16:17:41 -04:00

trie_search.h

[phrases] Adding *_from_index methods to trie_search

2015-06-09 11:14:42 -04:00

trie.c

[phrases] trie_add_len

2015-06-04 02:41:48 -04:00

trie.h

[phrases] trie_add_len

2015-06-04 02:41:48 -04:00

unicode_script_types.h

[fix] NUM_CHARS => NUM_CODEPOINTS

2015-05-17 13:53:19 -04:00

unicode_scripts_data.c

[unicode] Adding generated unicode script data

2015-03-18 17:01:03 -04:00

unicode_scripts.c

[unicode] Adding generated unicode script data

2015-03-18 17:01:03 -04:00

unicode_scripts.h

[unicode] header

2015-03-18 17:25:53 -04:00

vector_math.h

[utils] Adding min, max, argmin, argmax and log_sum_exp to generic vector math header

2015-04-12 12:11:04 -04:00

vector.h

[fix] vector_*_pop returns the element

2015-05-16 23:20:28 -04:00