Al
|
b85ed70674
|
[utils] adding a function for checking if files exists (yay C), or at least the closest agreed-upon method for it (may return false if the user doesn't have permissions, but that's ok for our purposes here)
|
2017-03-10 13:39:52 -05:00 |
|
Al
|
b97de96ab4
|
[parser] fixing chunked shuffle, making awk splitting work on Mac
|
2017-03-05 15:06:02 -05:00 |
|
Al
|
b76b7b8527
|
[parser] adding chunked shuffle as a C function (writes each line to one of n random files, runs shuf on each file and concatenates the result). Adding a version which allows specifying a specific chunk size, and using a 2GB limit for address parser training. Allowing gshuf again for Mac as it seems the only problem there was not having enough memory when testing on a Mac laptop. The new limited-memory version should be fast enough.
|
2017-03-05 02:15:11 -05:00 |
|
Al
|
b94264b745
|
[parser] Forgot to add shuffle.h/.c
|
2015-12-01 11:25:28 -05:00 |
|