Al a08d59c277 [fix] NFD normalization should be the default in normalize.c, not NFKD, as NFKD does some unwanted things like converting superscripts and the Latin-ASCII transliterator does a better, more thorough job while staying faithful to the original string

2015-07-05 15:28:07 -04:00

..

cmp

[geodisambig] C msgpack serialization dependency

2015-04-12 15:14:01 -04:00

geohash

[geohash] Modified geohash implementation (based on python-geohash) with no mallocs

2015-07-04 01:30:30 -04:00

klib

[fix] fixing some formatting

2015-03-03 12:54:27 -05:00

log

[fix] fixing some formatting

2015-03-03 12:54:27 -05:00

murmur

[geodisambig] adding MurmurHash3, used by the Bloom filter

2015-03-11 17:47:57 -04:00

utf8proc

[unicode] Upgrading to JuliaLang's utf8proc (Unicode 7, maintained)

2015-03-17 12:20:08 -04:00

bloom.c

[bloom] bloom filter I/O

2015-06-20 12:29:11 -05:00

bloom.h

[bloom] bloom filter I/O

2015-06-20 12:29:11 -05:00

collections.h

[utils] two set types in collections.h

2015-04-19 09:32:53 -04:00

config.h

[config] constants

2015-07-01 13:01:22 -04:00

constants.h

[config] constants

2015-07-01 13:01:22 -04:00

features.c

[utils] More consistent naming in string_utils

2015-03-27 21:12:08 -04:00

features.h

[utils] More consistent naming in string_utils

2015-03-27 21:12:08 -04:00

file_utils.c

[utils] using unsigned ints in file_utils, adding doubles

2015-05-27 16:03:36 -04:00

file_utils.h

[fix] typo in constant

2015-07-02 16:12:24 -04:00

gazetteers.h

[fix] gazetteers typo

2015-06-12 17:26:14 -04:00

geonames_disambiguation.c

[geodisambig] feature functions for GeoNames disambiguation

2015-07-04 10:35:56 -04:00

geonames_disambiguation.h

[geodisambig] feature functions for GeoNames disambiguation

2015-07-04 10:35:56 -04:00

geonames_fields.h

[geonames] generated geonames TSV fields

2015-07-02 15:59:54 -04:00

geonames.c

[fix] Removing feature length variables from geonames.c

2015-07-04 10:33:08 -04:00

geonames.h

[geodisambig] Adding new fields to geonames struct, plus I/O

2015-07-01 13:02:00 -04:00

normalize.c

[fix] NFD normalization should be the default in normalize.c, not NFKD, as NFKD does some unwanted things like converting superscripts and the Latin-ASCII transliterator does a better, more thorough job while staying faithful to the original string

2015-07-05 15:28:07 -04:00

normalize.h

[normalization] string normalization module for tokens and full strings

2015-07-01 14:52:28 -04:00

numex_data.c

[numex] Regenerating numex data file

2015-06-26 12:36:40 -04:00

numex_rule.h

[numx] header and table builder changes to support whole words languages

2015-06-12 16:10:57 -04:00

numex_table_builder.c

[numex] Adding method to get ordinal suffixes, using single representation

2015-06-25 17:28:06 -04:00

numex.c

[fix] moving int string size constants to string_utils.h

2015-07-02 17:50:09 -04:00

numex.h

[numex] Adding method to get ordinal suffixes, using single representation

2015-06-25 17:28:06 -04:00

postal_fields.h

[geonames] generated postal code TSV fields

2015-07-02 16:00:06 -04:00

scanner.c

[tokenization] Adding an acronym token type for things like U.N. so we can delete internal periods on those tokens

2015-06-29 03:00:46 -04:00

scanner.h

[tokenization] Adding a tokenizer method for appending to an existing tokens array (e.g. can stop/start tokenizing on a script change)

2015-06-25 10:03:34 -04:00

scanner.re

[tokenization] Adding an acronym token type for things like U.N. so we can delete internal periods on those tokens

2015-06-29 03:00:46 -04:00

string_utils.c

[utils] string_is_lower/string_is_upper method

2015-07-01 14:49:22 -04:00

string_utils.h

[fix] moving int string size constants to string_utils.h

2015-07-02 17:50:09 -04:00

token_types.h

[tokenization] Adding an acronym token type for things like U.N. so we can delete internal periods on those tokens

2015-06-29 03:00:46 -04:00

tokens.c

[utils] cstring_array_get_token=>cstring_array_get_string

2015-06-25 10:05:35 -04:00

tokens.h

[fix] type name/import

2015-04-01 02:54:14 -04:00

transliterate.c

[transliterate] no need to strdup transliterator names if they are lowercased, breaking on NUL byte

2015-07-01 14:51:22 -04:00

transliterate.h

[utf8] Moving language length constant

2015-06-30 19:17:20 -04:00

transliteration_data.c

[transliteration] Re-generating transliteration data file

2015-06-29 15:03:59 -04:00

transliteration_rule.h

[transliteration] Using revisit strings instead of keeping a backtrack count so we don't have to later map logical characters to the actual string, removing any duplicate keys in the table builder so that if any rules happen to overlap within a step, the first will take precedence

2015-05-29 16:54:05 -04:00

transliteration_scripts_data.c

[fix] using empty string instead of NULL for script languages so we can use fixed length arrays

2015-06-23 15:20:09 -05:00

transliteration_table_builder.c

[utils] cstring_array_get_token=>cstring_array_get_string

2015-06-25 10:05:35 -04:00

trie_search.c

[phrases] trie_search_suffixes uses the NUL-byte prefix by default but the _from_index version can start from another node. fixing single character suffixes

2015-06-25 17:24:19 -04:00

trie_search.h

[phrases] Adding *_from_index methods to trie_search

2015-06-09 11:14:42 -04:00

trie.c

[phrases] trie_add_len

2015-06-04 02:41:48 -04:00

trie.h

[phrases] trie_add_len

2015-06-04 02:41:48 -04:00

unicode_script_types.h

[fix] NUM_CHARS => NUM_CODEPOINTS

2015-05-17 13:53:19 -04:00

unicode_scripts_data.c

[fix] renaming structure

2015-06-23 02:12:24 -05:00

unicode_scripts.c

[utf8] Adding a boolean struct member for string_script_t return values, set to true if the string is ASCII (no transliteration needed, should be frequent for English addresses)

2015-06-28 19:37:58 -04:00

unicode_scripts.h

[utf8] Moving language length constant

2015-06-30 19:17:20 -04:00

vector_math.h

[utils] Adding min, max, argmin, argmax and log_sum_exp to generic vector math header

2015-04-12 12:11:04 -04:00

vector.h

[fix] vector_*_pop returns the element

2015-05-16 23:20:28 -04:00