TextPipe 8.4 software bug for an established filter

[dfhtextpipe]

I have already emailed about this issue to DataMystic Support.
My email to Simon Carter included some more details.
I am recording this here too, just in case my message get's stuck in cyberspace.

These lines in my filter did not work in 8.4, but they always worked from the earliest days of my filter, including 8.3.7.
(unchanged since 18 Jan 2008)

Code: Select all

Comment...
|  Remove the umatched </p> </div2> </div3> tags near the start of the file
|
+--Restrict lines:Line 1 .. line 3
   |
   +--Remove matching lines [.*]
         [ ] Include line numbers
         [ ] Include filename
         [ ] Match case
         [ ] Count matches
         Pattern type: 0
         [ ] UTF8 Support
         Context before: 0
         Context after: 0

I therefore suspect there is a software bug in the new version !!!!! :-(

Best regards,
David Haslam

[DataMystic Support]

Thanks David,

We will check it out, but in the meantime, make your filter faster by replacing the Remove matching lines [.*] with a simpler 'Remove\Remove All' filter.

[dfhtextpipe]

Dear Simon,

My license is only for the Standard edition.

In TextPipe Standard, I cannot find a filter called Remove All.

This a feature only available in TextPipe Pro. Hence I had to resort to the method using remove matching lines.

Of course, if you would consider making this filter available in Standard in a future upgrade, then I would be glad to take your advice.

Best regards,
David Haslam

[dfhtextpipe]

Dear Simon,

As a solution for my VPL to ThML filter, I have updated it as follows:

I changed the matching pattern from [.*] to [^(.*)$]

See below. This was successful.

Code: Select all

Comment...
|  Remove the umatched </p> </div2> </div3> tags near the start of the file
|
+--Restrict lines:Line 1 .. line 3
   |
   +--Remove matching lines [^(.*)$]
         [ ] Include line numbers
         [ ] Include filename
         [ ] Match case
         [ ] Count matches
         Pattern type: 0
         [ ] UTF8 Support
         Context before: 0
         Context after: 0

Are there any lessons to be drawn from this experience?

Why did my simpler pattern always work right up to version 8.3.7 but fail with version 8.4 ?

Best regards,
David Haslam

[DataMystic Support]

Possibly because 8.4 did this:
* Grep filter has now been changed so that it does NOT match empty lines.

Still need to investigate on Monday

[DataMystic Support]

Just change that regex to

Code: Select all

^(.+)$

instead of .* - that way the match has some length, and hence the line will get removed.

The 8.4 change here was that:
" Grep filter has now been changed so that it does NOT match empty lines.

[dfhtextpipe]

Dear Simon,

Thanks for your email.

IMHO, the way this change in 8.4 is described is quite misleading.

The three lines my filter was to remove were certainly not empty, so I had no reason to expect that the change would have been relevant!

The release notes do NOT explain that if the grep filter is capable of matching an empty line, it will not be actioned even for a non-empty line!

To me this is somewhat illogical behaviour, as it is very counter-intuitive.

I expect I will not be the only user who encounters this problem.

Also, your latest reply did not really explain why ^(.*)$ actually worked!
As far as I understand things, it works merely because the pattern ^$ is not a valid regexp.

Best regards,

David

[DataMystic Support]

Hi David,

We will need to add an option to 8.4.1 to enable the new behaviour, and keep the old behaviour by default.

Your original match allows the pattern to match 0 or more characters (using *), whereas + forces 1 or more characters.

Look out for this new version shortly.