DataMystic

Hi,

I have a huge file containing history of all trades for a given stock for a year. Say, the stock is MSFT (microsoft). Each line/record begins with a datestamp of the format: MM/DD/YYYY. Each day has variable number of trades and the number of trades per day is in thousands.

Assuming there are 200 days, I would like to split the huge file into 200 (or so) files, each file containing the history of all trades for one day.

So far, I have been able to specify a date, say 29 Dec. 2000 using a "Retain lines matching perl pattern" filter and have been able to extract a day's data and save it. Then I go and rename the file to MSFT29Dec.TXT.

Is it possible to do the following two things in TextPipe:
1. Specify a generic date filter and have the large input file split into multiple files based on each trading day of the year?
2. Specify date as part of the output file's name so as to obtain 200 or so files after the operation is complete?

Would appreciate any pointers.
Yogi

Hi,

Use an EasyPattern match to identify date changes, and insert a special character which a subsequent split filter will split the file on,

Next you use a Restrict to Filename filter with a VBScript filter inside it to set the filename.