Split a file into several files based on datestamp matching

Get help with installation and running here.

Moderators: DataMystic Support, Moderators, DataMystic Support, Moderators, DataMystic Support, Moderators

Post Reply
Posts: 1
Joined: Thu Sep 30, 2004 2:45 am

Split a file into several files based on datestamp matching

Post by yogi »


I have a huge file containing history of all trades for a given stock for a year. Say, the stock is MSFT (microsoft). Each line/record begins with a datestamp of the format: MM/DD/YYYY. Each day has variable number of trades and the number of trades per day is in thousands.

Assuming there are 200 days, I would like to split the huge file into 200 (or so) files, each file containing the history of all trades for one day.

So far, I have been able to specify a date, say 29 Dec. 2000 using a "Retain lines matching perl pattern" filter and have been able to extract a day's data and save it. Then I go and rename the file to MSFT29Dec.TXT.

Is it possible to do the following two things in TextPipe:
1. Specify a generic date filter and have the large input file split into multiple files based on each trading day of the year?
2. Specify date as part of the output file's name so as to obtain 200 or so files after the operation is complete?

Would appreciate any pointers.
User avatar
DataMystic Support
Site Admin
Posts: 2227
Joined: Mon Jun 30, 2003 12:32 pm
Location: Melbourne, Australia

Post by DataMystic Support »


Use an EasyPattern match to identify date changes, and insert a special character which a subsequent split filter will split the file on,

Next you use a Restrict to Filename filter with a VBScript filter inside it to set the filename.

Simon Carter, https://www.DataMystic.com
https://www.JadeDiabetes.com - Insulin dose calculator for Type 1 diabetes
https://www.DownloadPipe.com - 250,000 free software downloads
Post Reply