Extracting multiple files from a single file with one filter

Get help with installation and running here.

Moderators: DataMystic Support, Moderators, DataMystic Support, Moderators, DataMystic Support, Moderators

Post Reply
Posts: 2
Joined: Tue Jan 04, 2005 10:28 am
Location: Baltimore, Maryland USA

Extracting multiple files from a single file with one filter

Post by forward1 »

I have 4 years of log files that need to be processed. Each file contains one day’s traffic from over 100 domains on my server. I need to extract individual files from each log – one for each domain.

I can do it one domain at a time, but I’d like to have one filter, not 100. I’m using the Pro version of TextPipe

Thanks in advance,
User avatar
DataMystic Support
Site Admin
Posts: 2229
Joined: Mon Jun 30, 2003 12:32 pm
Location: Melbourne, Australia

Post by DataMystic Support »


The best approach here is to script this from a language such as VBScript. First design the filter to extract for one domain, then use File\Export to VBScript to generate the script.

Take this script and code it so it will run for each domain. Another approach is to use 100 consecutive Restrict to Matching Line filters to direct each domain to a new file. This filter can also be created using VBScript to make it easier.

Simon Carter, https://www.DataMystic.com
https://www.JadeDiabetes.com - Insulin dose calculator for Type 1 diabetes
https://www.DownloadPipe.com - 250,000 free software downloads
Post Reply