Command line: Tell TPipe to run all filters in folder

[pecosbill]

I need to be able to tell TextPipe to run all the filters found in a folder. I'm thinking of putting them on the TPipe server with a mounted folder.

I tried this command file
cd /d "%~d0%~p0\Files to Process\"
"C:\Program Files\DataMystic\TextPipe\textpipe.exe" "/F=..\Filters\*" /G
which didn't work.

[DataMystic Support]

Why would you do this? - You need to control the order in which filters are run.

If you really want to do this, you could use:

Code: Select all

FOR %%A IN (*.fll) DO "C:\Program Files\DataMystic\TextPipe\textpipe.exe" "/F=%%~nA" /G

[pecosbill]

Thanks. I didn't think about that though it won't be nice having all those instances popping up.

Another idea that just hit me would be to create a main filter that links into all the known filters. Not quite as novice friendly but it saves the GUI instances. (I am eagerly awaiting my company's purchase of the Run Time versions but alas things move VERY slowly here)

[DataMystic Support]

No worries.

You also might be able to use the 'start /w' command in conjunction with the FOR command to get it to wait before running the next instance.

[pecosbill]

Another idea that just hit me would be to create a main filter that links into all the known filters.

Alas, this fails miserably. I'd put this in the bug section but I'm not completely sure it's a bug and the thread started here.

Since the outermost filter file determines which files to process, I set my command file to set the working dir where the files are (subfolder) then started it with "/F=k:\Runs Known Filters.ffl" /G . That works. Inside that, I have the files to process (tab 2) set to *.rep and then have filters for each linked filter that restrict by filename to target the correct files that the linked file can handle as it's file selector is ignored when linked in. In Theory, all of this should work but when I run the command file, it munges (messes up) the output -- generally at the top of the file. If I run the individual file on its own against the same files, they are fine.

When I did my initial test with only one active linked file, it seemed to work. In this test, the only files to process were the _001_ files (the last link). Maybe my perl pattern is off? Should be near to DOS *_001_*.rep (I prefer + as there should always be chars on either side and it is more readable for the person I'm handing this off to.) If you have a solution, that would be great, otherwise, I'm okay with making them suffer with five instances so it's encouragement to buy the Run Time version.

I have five linked filter files as can be seen in the runner code:

Code: Select all

Restrict to filenames matching perl pattern [.+_016L_.+\.rep]
|
+--Link filter K:\Anchor no Display BC 16L.fll    
Restrict to filenames matching perl pattern [.+_003_.+\.rep]
|
+--Link filter K:\Check No Print BC 3.fll    
Restrict to filenames matching perl pattern [.+_014L_.+\.rep]
|
+--Link filter K:\Out of Place Heading BC 14L.fll    
Restrict to filenames matching perl pattern [.+_002_.+\.rep]
|
+--Link filter K:\Pending Lockout BC 2.fll    
Restrict to filenames matching perl pattern [.+_001_.+\.rep]
|
+--Link filter K:\Unhandled Contracts BC 1.fll

[DataMystic Support]

By any chance could you send us one sample file and your filters (the overall link + the individual ones)? We'd like to see if there is a bug or not.

[pecosbill]

Not really. The data files are confidential. I don't have time to munge the data. Would it still help to have the filter files?

[DataMystic Support]

Yes - the filters would definitely help. Thank you.

[pecosbill]

Just sent. Sorry for the delay.

[DataMystic Support]

The trick with the headers is that you need to be using an Add Header filter, not an Add File Header filter. The difference?

An Add Header filter outputs text both at the start of a file, and the start of a Restriction – which is why it is working when run separately but not when run together.

I think this is the problem with both of your header issues below.

As for debugging, use Filters\Special\Debug filter so that you can see what text flows past a certain point.

You can also right-click to disable as set of filters.

[pecosbill]

Well, that certainly helped but alas and alack, it didn't fix it. I could always run the sub-filter files individually without problems but when linked in, something would go wrong with the pattern matching. Not having the time to see what was wrong, I went back to the for loop:

Code: Select all

FOR %%A IN (k:\{*.fll) DO (
echo "C:\Program Files\DataMystic\TextPipe\textpipe.exe" "/F=%%~fA" /G /Q | start /b
)

I added a little bit to it (this will be run using the Run Time version). The production filters will be stored on a server that will be mounted as drive K: and as there are some reused modules in their own linked in filters, those couldn't be run so I added a { to the ones I did want to be run. I liked the idea of using the start command to get multiple instances running locally in parallel. Since I'm waiting on the Run Time purchase, I can only guess this will work in parallel. I'll probably change the Run Time install path to ensure it's used instead of the Pro version too. (I've found using echo to send the text to start works more reliably.) I hope this helps someone else

[DataMystic Support]

Hi Bill,

I'm pretty confident that the issue relates to the use of linked filters inside restrictions. Text inside a restriction is not treated identically to text inside a file.

If you have a simplified set of filters (yours are pretty amazing!) with a test file that you can share I would love to look at it.

[pecosbill]

Hi Bill,
If you have a simplified set of filters (yours are pretty amazing!) with .....

Wow! Thanks for the compliment People here are pretty happy with what I've done and there's still more to do. Meanwhile, I came up with this to build my list of filters to run. Since I don't want every filter to run (linked in modules shouldn't run by themselves), I prepend a { to indicate that it runs in the filename. (MUCH easier than looking for filenames without it). I also put the production filters on a shared folder and set the local client to mount that as K: That gave me this (I rem'd out the test statements but they might be handy for some):

Code: Select all

echo off
setlocal enabledelayedexpansion

for /f "usebackq delims=" %%C in (`dir /b  /x k:\{*.fll`) do (
	rem echo c is %%C
	set filters="/F=K:\%%C" /G !filters!
	rem echo filters are !filters!
)
echo Starting TextPipe with the following filters
echo final %filters%
echo ...
echo Remember, there is a maximum command length so at some point this might fail.
"C:\Program Files\DataMystic\TextPipe\textpipe.exe" !filters!

.... a test file that you can share I would love to look at it.

You have everything except test files. Feel free to simplify where you want. It appears the challenge lies in the easypattern when massive (ish) amounts of text are linked in. Increasing the max didn't fix it where it had when problems running separate happened. As for the test file, I think I can get away with sending what I have to you after I -- using TextPipe, of course -- munge the letters to all be z and numbers to all be 0. That should keep the privates, private. I'll send it via email shortly to limit risk (Are companies down under as sue happy as ours? Ugh.)

[DataMystic Support]

Our legal is taking the lead from the US unfortunately!