Alternative to processing multiple files in succession

Get help with installation and running here.

Moderators: DataMystic Support, Moderators, DataMystic Support, Moderators, DataMystic Support, Moderators

alnico
Posts: 74
Joined: Fri Oct 12, 2007 11:57 pm

Re: Alternative to processing multiple files in succession

Post by alnico »

[Macro]

Simon, maybe we can solve this problem more easily...

In the Search/Replace list filter, there are these input options plus my proposal:
  • 1. From file
    2. Using these values
    3. From Macro (proposal)
    • a. Choose deliminator: Tab, comma, etc.
How about allowing input from Macros?
If this were possible I could CREATE AND USE my Search/Replace patterns all in one single TextPipe engine. This would also serve as pseudo variables for Search/Find; as they could be used singularly.

Example: in one part of the TextPipe tree using the T-Filter, I can create my LIST of search/replace stings and then Capture to @SearchReplace macro, i.e. "The (<[^>]+>)red(<[^>]+>) fox","El zorro $1rojo$2"
Then in another part of the tree I use the Search/Replace filter and use @SearchReplace as the input.

What do you think?

Thanks,
Brent
User avatar
DataMystic Support
Site Admin
Posts: 2227
Joined: Mon Jun 30, 2003 12:32 pm
Location: Melbourne, Australia
Contact:

Re: Alternative to processing multiple files in succession

Post by DataMystic Support »

Hi Brent, thanks for your suggestion.

But again - it suffers from the problem of knowing at what time a filter is run. TextPipe tries to decouple the filters so that they all run at the same time, so assuming any sort of sequential execution leads to confusion (and makes TP slower).

I am going back to look again at your original suggestion again!
alnico
Posts: 74
Joined: Fri Oct 12, 2007 11:57 pm

Re: Alternative to processing multiple files in succession

Post by alnico »

Hi Simon,

I slept on my idea and realized the issue with simultaneous processing, argh.

Well if we want to continue to test this idea...

Would a new 'Special' filter for flow control be helpful: "Pause until complete" where all parent nodes of "Pause until complete" filter would have to be completed before continuing down the tree?
Just ideas.

However, the original idea a few posts back may be the best and most powerful solution.

Thanks,
Brent
User avatar
DataMystic Support
Site Admin
Posts: 2227
Joined: Mon Jun 30, 2003 12:32 pm
Location: Melbourne, Australia
Contact:

Re: Alternative to processing multiple files in succession

Post by DataMystic Support »

Yes - I think the flow control idea is the best - it lends itself to the largest number of processing tasks, without being complex to understand.
alnico
Posts: 74
Joined: Fri Oct 12, 2007 11:57 pm

Re: Alternative to processing multiple files in succession

Post by alnico »

Ya I agree the original idea is most powerful, however...

Doesn't Capture to variable then Add Header or any of the other filters that can Add Macro/variable info, suffer from the same issues you stated above?
How are these two scenarios different?

Thanks,
Brent
User avatar
DataMystic Support
Site Admin
Posts: 2227
Joined: Mon Jun 30, 2003 12:32 pm
Location: Melbourne, Australia
Contact:

Re: Alternative to processing multiple files in succession

Post by DataMystic Support »

You're right - they are no different.

We normally advise (and I think this is in the help file), that the Capture filters need to be inside a 'Restrict to each line in turn' filter so that TP is forced to process that section as one block.
alnico
Posts: 74
Joined: Fri Oct 12, 2007 11:57 pm

Re: Alternative to processing multiple files in succession

Post by alnico »

Hi Simon,

Both solutions discussed here may have worth.

If allowing macros in the Search/Replace List filter is not much different than already existing flow...then why not add it?
This I am guessing would be an easy addition...do we really have anything to loose?

Then later, add the master of all solutions ;-)

Thanks,
Brent
User avatar
DataMystic Support
Site Admin
Posts: 2227
Joined: Mon Jun 30, 2003 12:32 pm
Location: Melbourne, Australia
Contact:

Re: Alternative to processing multiple files in succession

Post by DataMystic Support »

The search/replace list filter still has the same speed issues with recompiling - since it can use regex for every item in the list. In addition, if you use simultaneous search, then it combines ALL the search terms into one (or more) humungous pattern. A straight search/replace would be easier to tackle!
alnico
Posts: 74
Joined: Fri Oct 12, 2007 11:57 pm

Re: Alternative to processing multiple files in succession

Post by alnico »

Okay Simon,

I will await the ultimate solution then...I think there will be many unforeseen advantages with this addition ;-)

Any ideas on timeline?

Thanks,
Brent
User avatar
DataMystic Support
Site Admin
Posts: 2227
Joined: Mon Jun 30, 2003 12:32 pm
Location: Melbourne, Australia
Contact:

Re: Alternative to processing multiple files in succession

Post by DataMystic Support »

We are on annual leave at the moment - I would expect some change by end of Feb. Sorry!
Post Reply