2 Commits

Author	SHA1	Message	Date
Al	0689f936c9	[tokenization] scanner/tokenizer (generated with re2c)	2015-03-03 12:35:22 -05:00
Al	5216aba1b6	[utils] string utils, file utils, contiguous arrays of strings used for storing tokenized strings, klib for generic hashtables and vectors, antirez's sds for certain types of string building, utf8proc for iterating over utf-8 strings and unicode normalization	2015-03-03 12:33:13 -05:00