DataMystic

Hello,

I thought I had this working correctly a few months ago, but I am not sure now

I am using a T-Filter, some matching filters, and then write results to file via Secondary Output Filter [test.tab].
Immediately upon leaving the T-Filter a Replace List: From File filter points to the file just written to in the T-Filter [test.tab]...it is not picking up the new [test.tab] content.

It appears that the T-Filter "Secondary Output Filter" does not close the file [test.tab]...thus making it unavailable for use by the Replace List until the entire TextPipe script is done.

If I run the script twice it then works of course...1st round it creates the test.tab file but is not written to until script is completely done. 2nd round test.tab has now been written to and now is available for Replace List.

Is this the way it works? If so, shouldn't the [test.tab] close...as if the T-Filter did not exist per your help for this filter? It would sure be powerful if did close so that a person could use that newly created file/content in the same TextPipe script further down the stream.

Thanks,

Brent


|--T-Filter
| +--Restrict to filenames matching perl pattern [input\.txt]
| |--Perl pattern [pattern]
| +--Merge output to file C:\test.tab
|
|--Restrict to filenames matching perl pattern [PartNumbers\.txt]
| |--Perl pattern [pattern]
| +--Replace list: C:\test.tab
| +--Merge output to file C:\results.txt
|
+--Output to clipboard

Hi Brent,

It can't work like this, because of the way that TextPipe works. For efficiency TP loads the entire test.tab into memory before the filter starts (for use in the replace list). In theory it could load this on first use only, but there is no way of knowing whether the input will be all type A filenames followed by all Type B (which would work) - or interspersed - which wouldn't.

The best option is to use File\Export to script the filter to javascript then run one filter after the other.

Hi Simon,

Your explanation makes my idea fail...thanks for clarifying.

If it could be done within TextPipe, it would be easier to manage/maintain than exporting to javascript as you suggested. So before I put this to rest...I will make one more suggestion...not sure if it is feasible though...

What if a checkbox called "Load on Completion" were added to the Secondary Output Filter?

The logic my brain sees is this:

Textpipe starts...looks for all files to load into memory...but also checks the Secondary Output Filters for checked option "Load on Completion". If "Load on Completion" is checked then that file does NOT get loaded initially; but is marked for dynamic loading.

As Textpipe continues to run and completes a Secondary Output Filter with "Load on Completion" checked...it would then load that file into memory (checks the 'marked for dynamic loading list' for load confirmation). It would then be available for ANY load file filter, (not just the Replace list filter...which would be very powerful).

Problems:

What if I wanted to load a 'dynamic' file BEFORE the Secondary Output Filter runs? As written above...it would not load anything. Maybe the process above is changed so that ALL files are loaded initially (as is current usage)...and only when the Secondary Output Filter with "Load on Completion" is called that it simply replaces the file in memory with a new one?

Not sure if this would cause flow issues...if so, perhaps the "Load on Completion" option in the Secondary Output Filter would only be active if it falls as a sub filter of the T-Filter?

I am guessing this would be less efficient, but the user could simple be warned of the issues in the help file.

Would something like that work?

Thanks,

Brent

Hi Brent,

It's a good thought, but I think it would create all sorts of flow control problems.

What is really needed here is a higher level 'Filter Control' object that takes care of triggering filters in the right order.

Also, most users simply do not read the manual, so any non-obvious processing can create support problems.

Hi Simon,

Yeah, I thought if this option was under the T-Filter that it would be independent and yet work...maybe not.

If this option was part of the Secondary Output Filter...it wouldn't be selected by default...and a person would have to read the help to actually figure out what it does.

Anyways, thanks for entertaining my suggestions...
I still think this would be an extremely powerful option, if it could be figured out how to work it in seamlessly.

Thanks,

Brent

The biggest thing to me is to avoid exceptions such as 'if a filter is part of a filter B, then do action 1 instead of action 2'. That makes it harder to code and support and explain. And there will always be exceptions where it doesn't work quite as planned