New filter proposal: Repeat Filter(s)

Get help with installation and running here.

Moderators: DataMystic Support, Moderators, DataMystic Support, Moderators, DataMystic Support, Moderators

Post Reply
alnico
Posts: 74
Joined: Fri Oct 12, 2007 11:57 pm

New filter proposal: Repeat Filter(s)

Post by alnico »

I am not sure if this is possible with the way TextPipe processes...but here goes...

Since it is possible to create nice structured sub filter processes (trees) in TextPiple...
What if it were possible to insert a parent filter called "Repeat Filter(s)" that simply relies on Vbscript "Do Until Loop" (in essence it would dynamically wrap the tree with a Do Until loop).
It's parameters would be: Repeat until: [macro] [operator] [macro/number].

I know there is many ways to loop...but having one simple way up front would be a great enhancement.

Possible? Problems with my logic ;-)
User avatar
DataMystic Support
Site Admin
Posts: 2227
Joined: Mon Jun 30, 2003 12:32 pm
Location: Melbourne, Australia
Contact:

Re: New filter proposal: Repeat Filter(s)

Post by DataMystic Support »

Hi - this is actually possible already by using subfilters of a VBScript filter. The VBscript filters controls when and how the subfilters get applied. See the help file for more details.
alnico
Posts: 74
Joined: Fri Oct 12, 2007 11:57 pm

Re: New filter proposal: Repeat Filter(s)

Post by alnico »

Ya, I figured modularizing and bringing this out of the programming environment (like all other filters) would benefit non-programmers and be quick and easy to set up.
User avatar
DataMystic Support
Site Admin
Posts: 2227
Joined: Mon Jun 30, 2003 12:32 pm
Location: Melbourne, Australia
Contact:

Re: New filter proposal: Repeat Filter(s)

Post by DataMystic Support »

I understand your logic, but then making the filter able to cope with all the possible variations of loop start and end conditions etc - better off just delegating this to a custom script.

How did you plan to use this filter?
alnico
Posts: 74
Joined: Fri Oct 12, 2007 11:57 pm

Re: New filter proposal: Repeat Filter(s)

Post by alnico »

Basically this is what I need:

|--Repeat Filter(s) (Do Until Loop)
| |--Find (Replace first only in turn)
| | |--Transform
| |--Sort
|--Loop

Even using the VBScript filter…I am not convinced this could be done. As I can switch sub-filters off/on, but I cannot repeat a filter when “Replace first only” is active.
TextPipe as a line processor combined with VBScript gets me a little confused…as I am always in a restriction ;-)

Thinking about this more…

Maybe the Repeat Filter(s) suggestion is not needed if it were possible to solve the same problem by adding checkbox option:
“Replace first in turn” to the “Perl pattern” filter.

Thus its implementation could be like this:

|--Find (Replace first in turn)
| |--Transform
|--Sort

This would repeat all filters below the 'Find' on the same tree level in turn until no more matches are found (does have the potential for an endless loop however, maybe there's a way to control it?).
To control the loop...can add it to a restriction.
User avatar
DataMystic Support
Site Admin
Posts: 2227
Joined: Mon Jun 30, 2003 12:32 pm
Location: Melbourne, Australia
Contact:

Re: New filter proposal: Repeat Filter(s)

Post by DataMystic Support »

Thanks, but can I please see a practical example of where this would be applied?
alnico
Posts: 74
Joined: Fri Oct 12, 2007 11:57 pm

Re: New filter proposal: Repeat Filter(s)

Post by alnico »

I am normalizing html tables that contain the attribute: rowspan (duplicating and positioning x times).

Each table row, cell is given a multipart line ID...such as 001.001.001
Each rowspan line is duplicated x times and line ID incremented using VBScript
Then the entire <table> must be sorted
Then line IDs must be reset using VBScript
Then repeat the process (loop) for the next rowspan found.

The problem is that TextPipe can only do work within restrictions all at once...even using variables and 'add section header', etc wont work...because always limited by the restriction.

Here is an example table where the first rowspan is duplicated 3 times and line ID updated.
If we now sort this it would be correct...but with subsequent rowspans they sometimes will be one position off (that is because we are duplicating all the rowspan's at once and then one sort (it needs to loop so that line IDs may be reset).

I have this all working, but of course the looping part is missing.
This is a very complicated process...the model I have designed works...but just getting TextPipe to do it is challenging ;-)

Code: Select all

<table width="500" border="1">
001.001.001	<tr>
001.002.001	<td rowspan="3">a</td>
002.002.000	<td rowspan="3">a</td>
003.002.000	<td rowspan="3">a</td>
001.003.001	<td rowspan="4">c</td>
001.004.001	<td>d</td>
001.005.001	</tr>
002.001.001	<tr>
002.002.001	<td>f</td>
002.003.001	</tr>
003.001.001	<tr>
003.002.001	<td>h</td>
003.003.001	</tr>
004.001.001	<tr>
004.002.001	<td>i</td>
004.003.001	<td>j</td>
004.004.001	</tr>
005.001.001	<tr>
005.002.001	<td>k</td>
005.003.001	<td>l</td>
005.004.001	<td rowspan="3">m</td>
005.005.001	</tr>
006.001.001	<tr>
006.002.001	<td>n</td>
006.003.001	<td>o</td>
006.004.001	</tr>
007.001.001	<tr>
007.002.001	<td>p</td>
007.003.001	<td>q</td>
007.004.001	</tr>
</table>
User avatar
DataMystic Support
Site Admin
Posts: 2227
Joined: Mon Jun 30, 2003 12:32 pm
Location: Melbourne, Australia
Contact:

Re: New filter proposal: Repeat Filter(s)

Post by DataMystic Support »

Ok, I am confused - why are you doing this? Are you trying to sort the table by ColA, ColB, ColC?
alnico
Posts: 74
Joined: Fri Oct 12, 2007 11:57 pm

Re: New filter proposal: Repeat Filter(s)

Post by alnico »

Sorry,

Here is what exactly I am trying to accomplish. http://www.andrewjwelch.com/code/xslt/t ... ation.html

I currently have to do part of my processing in TextPipe then with 3rd party app (without command line) then bring the data back into TextPipe to continue.
It is a messy process that I am trying to simplify.
User avatar
DataMystic Support
Site Admin
Posts: 2227
Joined: Mon Jun 30, 2003 12:32 pm
Location: Melbourne, Australia
Contact:

Re: New filter proposal: Repeat Filter(s)

Post by DataMystic Support »

Ok, that's messy. I still don't know why you would want to do this - surely there are very few situations where you want to repeat table cells, unless they are &nbsp;'s or blank (or a fixed background color).
I still maintain the looping is highly unnatural for text stream processing.

Anyway:
but I cannot repeat a filter when “Replace first only” is active.
When you call subFilterEntireText the entire text you pass to the subfilters is treated as one file - so Replace First Only works as it would for a file. If you don't want it to work, then you could use

Code: Select all

initSubFilters()
subFilterChunk( inputString : string )
subFilterChunk( inputString : string )
subFilterChunk( inputString : string )
...
flushSubfilters()
repeatedly.
alnico
Posts: 74
Joined: Fri Oct 12, 2007 11:57 pm

Re: New filter proposal: Repeat Filter(s)

Post by alnico »

Thanks Simon,

I just got this 'normalization' process figured out by running an external process and bringing it back into TextPipe...no small feat ;-)

You ask "why I would want to do this"...imagine exporting html tables from print catalog layout software (with merged cells) and preparing it for database import. Tables/data MUST be normalized.

Thanks,
Brent
User avatar
DataMystic Support
Site Admin
Posts: 2227
Joined: Mon Jun 30, 2003 12:32 pm
Location: Melbourne, Australia
Contact:

Re: New filter proposal: Repeat Filter(s)

Post by DataMystic Support »

Glad to hear it!

Thanks - now I understand the why!
Post Reply