tommy/libpostal
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

[dedupe] adding a function to acronyms module to detect existing/known acronyms like MS for middle school, HS for high school, etc. Forms like MS have to be deined in the dictionaries specifically but any acronym written like M.S. will be detected as such by the tokenizer

Browse Source
This commit is contained in:
Al
2018-01-15 23:46:49 -05:00
parent 133381f439
commit 0ee18b4f6c
4 changed files with 64 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
src/acronyms.h
View File

@@ -10,6 +10,7 @@
#include "token_types.h"
bool stopword_positions(uint32_array *stopwords_array, const char *str, token_array *tokens, size_t num_languages, char **languages);
bool existing_acronym_phrase_positions(uint32_array *existing_acronyms_array, const char *str, token_array *token_array, size_t num_languages, char **languages);
phrase_array *acronym_token_alignments(const char *s1, token_array *tokens1, const char *s2, token_array *tokens2, size_t num_languages, char **languages);