Al
|
9a92a1154d
|
[python] Making normalized_tokens return token classes as well, mimicking the tokenize API
|
2015-10-27 17:07:50 -04:00 |
|
Al
|
9f6e1387a0
|
[fix] Error condition in Python tokenize
|
2015-10-27 13:33:28 -04:00 |
|
Al
|
40918812e2
|
[normalize] Adding hyphen elimination as a string option (changes tokenization)
|
2015-10-27 13:32:47 -04:00 |
|
Al
|
f6b6a17335
|
[python/normalization] Adding Python bindings to the normalize module for use in OSM polygon matching
|
2015-10-26 18:07:53 -04:00 |
|
Al
|
8a188903b3
|
[python] Using tuples in pytokenize instead of list, pre-allocating
|
2015-10-26 18:04:13 -04:00 |
|
Al
|
4f784060a3
|
[python] Adding word_token_types
|
2015-10-25 18:33:09 -04:00 |
|
Al
|
236737eab3
|
[tokenization/osm] Using utf8 encoded version of string for tokens in python tokenizer
|
2015-09-21 17:27:43 -04:00 |
|
Al
|
5b2fd0be50
|
[fix] pytokenize compilation on Ubuntu/gcc
|
2015-09-21 03:24:14 -04:00 |
|
Al
|
5485ea2197
|
[python] Adding initial pypostal bindings for tokenize so we can remove address_normalizer dependency. Not tested on Python 3.
|
2015-09-20 14:59:39 -04:00 |
|