Page 1 of 1

New filter proposal: Repeat Filter(s)

Posted: Mon Feb 14, 2011 12:41 am
by alnico
I am not sure if this is possible with the way TextPipe processes...but here goes...

Since it is possible to create nice structured sub filter processes (trees) in TextPiple...
What if it were possible to insert a parent filter called "Repeat Filter(s)" that simply relies on Vbscript "Do Until Loop" (in essence it would dynamically wrap the tree with a Do Until loop).
It's parameters would be: Repeat until: [macro] [operator] [macro/number].

I know there is many ways to loop...but having one simple way up front would be a great enhancement.

Possible? Problems with my logic ;-)

Re: New filter proposal: Repeat Filter(s)

Posted: Mon Feb 14, 2011 8:05 am
by DataMystic Support
Hi - this is actually possible already by using subfilters of a VBScript filter. The VBscript filters controls when and how the subfilters get applied. See the help file for more details.

Re: New filter proposal: Repeat Filter(s)

Posted: Tue Feb 15, 2011 12:44 am
by alnico
Ya, I figured modularizing and bringing this out of the programming environment (like all other filters) would benefit non-programmers and be quick and easy to set up.

Re: New filter proposal: Repeat Filter(s)

Posted: Tue Feb 15, 2011 8:21 am
by DataMystic Support
I understand your logic, but then making the filter able to cope with all the possible variations of loop start and end conditions etc - better off just delegating this to a custom script.

How did you plan to use this filter?

Re: New filter proposal: Repeat Filter(s)

Posted: Wed Feb 16, 2011 2:58 am
by alnico
Basically this is what I need:

|--Repeat Filter(s) (Do Until Loop)
| |--Find (Replace first only in turn)
| | |--Transform
| |--Sort
|--Loop

Even using the VBScript filter…I am not convinced this could be done. As I can switch sub-filters off/on, but I cannot repeat a filter when “Replace first only” is active.
TextPipe as a line processor combined with VBScript gets me a little confused…as I am always in a restriction ;-)

Thinking about this more…

Maybe the Repeat Filter(s) suggestion is not needed if it were possible to solve the same problem by adding checkbox option:
“Replace first in turn” to the “Perl pattern” filter.

Thus its implementation could be like this:

|--Find (Replace first in turn)
| |--Transform
|--Sort

This would repeat all filters below the 'Find' on the same tree level in turn until no more matches are found (does have the potential for an endless loop however, maybe there's a way to control it?).
To control the loop...can add it to a restriction.

Re: New filter proposal: Repeat Filter(s)

Posted: Wed Feb 16, 2011 7:31 am
by DataMystic Support
Thanks, but can I please see a practical example of where this would be applied?

Re: New filter proposal: Repeat Filter(s)

Posted: Thu Feb 17, 2011 1:17 am
by alnico
I am normalizing html tables that contain the attribute: rowspan (duplicating and positioning x times).

Each table row, cell is given a multipart line ID...such as 001.001.001
Each rowspan line is duplicated x times and line ID incremented using VBScript
Then the entire <table> must be sorted
Then line IDs must be reset using VBScript
Then repeat the process (loop) for the next rowspan found.

The problem is that TextPipe can only do work within restrictions all at once...even using variables and 'add section header', etc wont work...because always limited by the restriction.

Here is an example table where the first rowspan is duplicated 3 times and line ID updated.
If we now sort this it would be correct...but with subsequent rowspans they sometimes will be one position off (that is because we are duplicating all the rowspan's at once and then one sort (it needs to loop so that line IDs may be reset).

I have this all working, but of course the looping part is missing.
This is a very complicated process...the model I have designed works...but just getting TextPipe to do it is challenging ;-)

Code: Select all

<table width="500" border="1">
001.001.001	<tr>
001.002.001	<td rowspan="3">a</td>
002.002.000	<td rowspan="3">a</td>
003.002.000	<td rowspan="3">a</td>
001.003.001	<td rowspan="4">c</td>
001.004.001	<td>d</td>
001.005.001	</tr>
002.001.001	<tr>
002.002.001	<td>f</td>
002.003.001	</tr>
003.001.001	<tr>
003.002.001	<td>h</td>
003.003.001	</tr>
004.001.001	<tr>
004.002.001	<td>i</td>
004.003.001	<td>j</td>
004.004.001	</tr>
005.001.001	<tr>
005.002.001	<td>k</td>
005.003.001	<td>l</td>
005.004.001	<td rowspan="3">m</td>
005.005.001	</tr>
006.001.001	<tr>
006.002.001	<td>n</td>
006.003.001	<td>o</td>
006.004.001	</tr>
007.001.001	<tr>
007.002.001	<td>p</td>
007.003.001	<td>q</td>
007.004.001	</tr>
</table>

Re: New filter proposal: Repeat Filter(s)

Posted: Fri Feb 18, 2011 8:11 am
by DataMystic Support
Ok, I am confused - why are you doing this? Are you trying to sort the table by ColA, ColB, ColC?

Re: New filter proposal: Repeat Filter(s)

Posted: Sat Feb 19, 2011 1:21 am
by alnico
Sorry,

Here is what exactly I am trying to accomplish. http://www.andrewjwelch.com/code/xslt/t ... ation.html

I currently have to do part of my processing in TextPipe then with 3rd party app (without command line) then bring the data back into TextPipe to continue.
It is a messy process that I am trying to simplify.

Re: New filter proposal: Repeat Filter(s)

Posted: Mon Feb 21, 2011 11:11 am
by DataMystic Support
Ok, that's messy. I still don't know why you would want to do this - surely there are very few situations where you want to repeat table cells, unless they are &nbsp;'s or blank (or a fixed background color).
I still maintain the looping is highly unnatural for text stream processing.

Anyway:
but I cannot repeat a filter when “Replace first only” is active.
When you call subFilterEntireText the entire text you pass to the subfilters is treated as one file - so Replace First Only works as it would for a file. If you don't want it to work, then you could use

Code: Select all

initSubFilters()
subFilterChunk( inputString : string )
subFilterChunk( inputString : string )
subFilterChunk( inputString : string )
...
flushSubfilters()
repeatedly.

Re: New filter proposal: Repeat Filter(s)

Posted: Fri Feb 25, 2011 1:48 am
by alnico
Thanks Simon,

I just got this 'normalization' process figured out by running an external process and bringing it back into TextPipe...no small feat ;-)

You ask "why I would want to do this"...imagine exporting html tables from print catalog layout software (with merged cells) and preparing it for database import. Tables/data MUST be normalized.

Thanks,
Brent

Re: New filter proposal: Repeat Filter(s)

Posted: Fri Feb 25, 2011 7:36 am
by DataMystic Support
Glad to hear it!

Thanks - now I understand the why!