3 Commits

Author	SHA1	Message	Date
Al	b97de96ab4	[parser] fixing chunked shuffle, making awk splitting work on Mac	2017-03-05 15:06:02 -05:00
Al	b76b7b8527	[parser] adding chunked shuffle as a C function (writes each line to one of n random files, runs shuf on each file and concatenates the result). Adding a version which allows specifying a specific chunk size, and using a 2GB limit for address parser training. Allowing gshuf again for Mac as it seems the only problem there was not having enough memory when testing on a Mac laptop. The new limited-memory version should be fast enough.	2017-03-05 02:15:11 -05:00
Al	b94264b745	[parser] Forgot to add shuffle.h/.c	2015-12-01 11:25:28 -05:00