[dedupe] adding a function to acronyms module to detect existing/known acronyms like MS for middle school, HS for high school, etc. Forms like MS have to be deined in the dictionaries specifically but any acronym written like M.S. will be detected as such by the tokenizer

This commit is contained in:
Al
2018-01-15 23:46:49 -05:00
parent 133381f439
commit 0ee18b4f6c
4 changed files with 64 additions and 0 deletions

View File

@@ -107,6 +107,7 @@ bool string_is_digit(char *str, size_t len);
bool string_is_ignorable(char *str, size_t len);
ssize_t string_next_hyphen_index(char *str, size_t len);
bool string_contains(char *str, char *sub);
bool string_contains_hyphen(char *str);
bool string_contains_hyphen_len(char *str, size_t len);