Page 1 of 1

Filters work individually, not as a group

Posted: Sat Oct 10, 2015 10:52 am
by dynalt
These filters get the correct counts when run individually.
When run together in any order, the counts are off.
Objective is to extract Purchases to TSV format and remove Additions and Other Withdrawals.
Counts: (When run singly)
Purchases 52
Additions 6
Withdrawals 3

I am thoroughly mystified.
File upload objects to every extension, so I am pasting the data.

--------------------
Filters
TextPipe Single User Edition 9.9.2
Purchased by: DYNAMIC Alternatives, DYNAMIC Alternatives

Filter Title: J:\DYNALT\ProgramData\TextPipeData\Household_1.fll

Filter List
-----------
Filter options
| [X] Log to file
| [ ] Append to logfile
| Log filename: %USERPROFILE%\textpipe.log
| Threshold 500
|
|--Input from file(s)
| [ ] Confirm before processing each file
| [ ] Confirm before processing read/only files
| [ ] Delete input files after processing
| Skip binary files
| Sample size 100 characters
|
|--Comment...
| Extract Purchases to TSV
|
|--Perl pattern [(.*)\r\n\t\r\n(.*)\r\n\t\r\nPurchases\r\n\t\r\n-(.*)\r\n\t\r\n(.*)\r\n] with [$1\t$2\t$3\r\n]
| | [ ] Match case
| | [ ] Whole words only
| | [ ] Case sensitive replace
| | [ ] Prompt on replace
| | [ ] Skip prompt if identical
| | [ ] First only
| | [ ] Extract matches
| | Maximum text buffer size 4096
| | [ ] Maximum match (greedy)
| | [ ] Allow comments
| | [X] '.' matches newline
| | [ ] UTF-8 Support
| |
| |--Comment...
| | Change description to Title Case
| |
| +--Perl pattern [(.*)\t(.*)\t(.*)\r\n] with []
| | [ ] Match case
| | [ ] Whole words only
| | [ ] Case sensitive replace
| | [ ] Prompt on replace
| | [ ] Skip prompt if identical
| | [ ] First only
| | [ ] Extract matches
| | Maximum text buffer size 4096
| | [ ] Maximum match (greedy)
| | [ ] Allow comments
| | [X] '.' matches newline
| | [ ] UTF-8 Support
| |
| +--Convert to Title Case
|
|--Comment...
| Remove Other Withdrawals
|
|--Perl pattern [(.*)\r\n\t\r\n(.*)\r\n\t\r\nOther Withdrawals\r\n\t\r\n(.*)\r\n\t\r\n(.*)\r\n] with []
| [ ] Match case
| [ ] Whole words only
| [ ] Case sensitive replace
| [ ] Prompt on replace
| [ ] Skip prompt if identical
| [ ] First only
| [ ] Extract matches
| Maximum text buffer size 4096
| [ ] Maximum match (greedy)
| [ ] Allow comments
| [X] '.' matches newline
| [ ] UTF-8 Support
|
|--Comment...
| Remove Additions
|
|--Perl pattern [(.*)\r\n\t\r\n(.*)\r\n\t\r\nAdditions\r\n\t\r\n(.*)\r\n\t\r\n(.*)\r\n] with []
| [ ] Match case
| [ ] Whole words only
| [ ] Case sensitive replace
| [ ] Prompt on replace
| [ ] Skip prompt if identical
| [ ] First only
| [ ] Extract matches
| Maximum text buffer size 4096
| [ ] Maximum match (greedy)
| [ ] Allow comments
| [X] '.' matches newline
| [ ] UTF-8 Support
|
|--Comment...
| Click >>Here<< or press F1 for help
|
| TextPipe uses 'filters' to perform changes to each file. This 'filter list' is a sequence of changes, like 3 different search and replace operations.
|
| You can add filters from the Filter menu above - Convert, Add, Remove, Unicode, Replace, Special, Maps, Email, Restrict.
|
| For:
| * Search/replace - look in the Replace Menu
| * Unix/DOS/Mainframe End-of-line conversion - look in the Convert Menu
| * Extract email addresses - look in the Email Menu
| * Remove HTML codes - look in the Remove Menu
| * Join (merge) files together - look in the Special Menu
| * Split files apart - look in the Special Menu
| * Join files and then sort them - add a Special Menu\Merge filter followed by a Special Menu\Sort filter
|
| Just click the menu item to add the filter to the list.
|
| You'll find many examples just by clicking File\Open, or you can use File\Search to locate a filter.
|
| You can also ask questions in the TextPipe Discussion Forums -
| http://www.datamystic.com/forums
|
+--Output to file(s)
[ ] Only update date on changed files
[ ] Append mode
[ ] Change extension to: .txt
[ ] Open output file
Only output modified files
Backup mode
[ ] Remove empty output files

Files List
----------

Use the line below to remove common non-text files from website processing
.[ 'gif' or 'png' or 'jpg' or 'bmp' or 'avi' or 'ico' or 'mp3', lineEnd ]
Use the line below to remove common non-text folders from website processing
_vti
P:\Graphics\All Financial Data\Finances\Household.txt

--------------------
Data

10/08/2015

COSTCO WHS E #07 LA HABRA CA

Purchases

-$54.36

$139.27
10/08/2015

99-CENTS- 14540 LE LA MIRADA CA

Purchases

-$28.89

$193.63
10/08/2015

99 CENTS 12301 NO NORWALK CA

Purchases

-$24.52

$222.52
10/08/2015

GOODWILL I 14535 TELEGRAPH LA MIRADA CA

Purchases

-$5.38

$247.04
10/08/2015

COSTCO WHS E #07 LA HABRA CA

Purchases

-$3.62

$252.42
10/02/2015

99 CENTS O NORWALK CA NORWALK CA

Purchases

-$10.87

$256.04
09/29/2015

SPROUTS FA 1431 W IMPERIAL LA HABRA CA

Purchases

-$60.58

$266.91
09/29/2015

99-CENTS- 14540 LE LA MIRADA CA

Purchases

-$17.02

$327.49
09/28/2015

99-CENTS-O LA MIRADA CA LA MIRADA CA

Purchases

-$7.60

$344.51
09/28/2015

UB CHECKING TRANSFER 150928 0431240282 0101

Additions

$200.00

$352.11
09/25/2015

COSTCO WHS E #07 LA HABRA CA

Purchases

-$47.67

$152.11
09/25/2015

PETSMART I 15618 WHITTWOOD WHITTER CA

Purchases

-$32.53

$199.78
09/25/2015

VALLARTA S 8510 PAINTER AV WHITTIER CA

Purchases

-$6.49

$232.31
09/24/2015

SERVICE CHARGE W/ONLINE STATEMENT

Other Withdrawals

-$8.00

$238.80
09/24/2015

99 CENTS O NORWALK CA NORWALK CA

Purchases

-$9.63

$246.80
09/23/2015

99-CENTS- 14540 LE LA MIRADA CA

Purchases

-$19.77

$256.43
09/22/2015

ANAHEIM FE ED & PET ANAHEIM CA

Purchases

-$98.66

$276.20
09/22/2015

TRADER JOE 'S # WHITTIER CA

Purchases

-$77.22

$374.86
09/22/2015

ALTAYEBET 1217 S BROOKHUR ANAHEIM CA

Purchases

-$37.63

$452.08
09/22/2015

CA DMV WHI TTIER FO WHITTIER CA

Purchases

-$33.00

$489.71
09/22/2015

SPROUTS FA 15801 WHITTIER WHITTIER CA

Purchases

-$25.14

$522.71
09/17/2015

COSTCO WHS E #07 LA HABRA CA

Purchases

-$11.97

$547.85
09/17/2015

CHECKING TRANSFER 150917 0431240282 0000

Additions

$300.00

$559.82
09/11/2015

UB CHECKING TRANSFER 150911 0431240282 0101

Additions

$200.00

$259.82
09/10/2015

COSTCO WHS E #07 LA HABRA CA

Purchases

-$65.63

$59.82
09/10/2015

99-CENTS- 14540 LE LA MIRADA CA

Purchases

-$25.58

$125.45
09/04/2015

COSTCO WHS E #07 LA HABRA CA

Purchases

-$19.48

$151.03
09/04/2015

BIG LOTS # 13942 E IMPERIA LA MIRADA CA

Purchases

-$3.29

$170.51
09/03/2015

99-CENTS- 14540 LE LA MIRADA CA

Purchases

-$38.21

$173.80
09/03/2015

97 CENTS P 12719 VALLEY VI LA MIRADA CA

Purchases

-$15.73

$212.01
09/01/2015

BIG LOTS # 13942 E IMPERIA LA MIRADA CA

Purchases

-$15.30

$227.74
08/27/2015

SPROUTS FA 1431 W IMPERIAL LA HABRA CA

Purchases

-$55.36

$243.04
08/26/2015

99-CENTS-O LA MIRADA CA LA MIRADA CA

Purchases

-$14.58

$298.40
08/25/2015

SERVICE CHARGE W/ONLINE STATEMENT

Other Withdrawals

-$8.00

$312.98
08/25/2015

COSTCO WHS E #07 LA HABRA CA

Purchases

-$11.97

$320.98
08/24/2015

COSTCO WHS E #07 LA HABRA CA

Purchases

-$63.56

$332.95
08/24/2015

TRADER JOE 'S # CERRITOS CA

Purchases

-$49.74

$396.51
08/24/2015

SMART AND FINAL WHITTIER CA

Purchases

-$30.04

$446.25
08/24/2015

99-CENTS-O LA MIRADA CA LA MIRADA CA

Purchases

-$14.09

$476.29
08/20/2015

UB CHECKING TRANSFER 150820 0431240282 0101

Additions

$300.00

$490.38
08/18/2015

HARBOR FRE 14345 FIRESTONE LA MIRADA CA

Purchases

-$10.55

$190.38
08/17/2015

99-CENTS-O WHITTIER CA WHITTIER CA

Purchases

-$13.08

$200.93
08/17/2015

99 CENTS O NORWALK CA NORWALK CA

Purchases

-$11.45

$214.01
08/14/2015

DOLLARTREE 13936 IMPERIAL LA MIRADA CA

Purchases

-$2.60

$225.46
08/10/2015

GOODWILL I 14535 TELEGRAPH LA MIRADA CA

Purchases

-$6.00

$228.06
08/10/2015

99-CENTS-O LA MIRADA CA LA MIRADA CA

Purchases

-$7.69

$234.06
08/10/2015

UB CHECKING TRANSFER 150810 0431240282 0101

Additions

$200.00

$241.75
08/07/2015

TRADER JOE 'S # CERRITOS CA

Purchases

-$69.25

$41.75
07/31/2015

99 CENTS O NORWALK CA NORWALK CA

Purchases

-$13.97

$111.00
07/31/2015

99-CENTS-O LA MIRADA CA LA MIRADA CA

Purchases

-$13.16

$124.97
07/30/2015

COSTCO WHS E #07 LA HABRA CA

Purchases

-$149.88

$138.13
07/30/2015

TRADER JOE 'S # CERRITOS CA

Purchases

-$83.07

$288.01
07/27/2015

SERVICE CHARGE W/ONLINE STATEMENT

Other Withdrawals

-$8.00

$371.08
07/23/2015

ALTAYEBET 1217 S B ANAHEIM CA

Purchases

-$29.98

$379.08
07/22/2015

SPROUTS FA 15801 WHITTIER WHITTIER CA

Purchases

-$37.67

$409.06
07/20/2015

99-CENTS-O LA MIRADA CA LA MIRADA CA

Purchases

-$6.58

$446.73
07/20/2015

99 CENTS O NORWALK CA NORWALK CA

Purchases

-$4.18

$453.31
07/17/2015

TRADER JOE 'S # CERRITOS CA

Purchases

-$62.12

$457.49
07/16/2015

UB CHECKING TRANSFER 150716 0431240282 0101

Additions

$500.00

$519.61
07/14/2015

COSTCO WHS E #07 LA HABRA CA

Purchases

-$11.97

$19.61
07/13/2015

141SUPERI 12721 VA LA MIRADA CA

Purchases

-$13.01

$31.58

--------------------
Log

TextPipe Lite Log File

datetime,level,message
2015-10-09 17:21:00,Info,Log file started
2015-10-09 17:21:00,Info,Filter name [J:\DYNALT\ProgramData\TextPipeData\Household_1.fll]
2015-10-09 17:21:00,Info,Job started by [Owner]
2015-10-09 17:21:00,Info,Input file [P:\Graphics\All Financial Data\Finances\Household.txt] Size: 3,591 bytes
2015-10-09 17:21:00,Info,0 replace(s) performed for pattern match [(.*)\r\n\t\r\n(.*)\r\n\t\r\nPurchases\r\n\t\r\n-(.*)\r\n\t\r\n(.*)\r\n]
2015-10-09 17:21:00,Info, 0 replace(s) performed for pattern match [(.*)\t(.*)\t(.*)\r\n]
2015-10-09 17:21:00,Info,3 replace(s) performed for pattern match [(.*)\r\n\t\r\n(.*)\r\n\t\r\nOther Withdrawals\r\n\t\r\n(.*)\r\n\t\r\n(.*)\r\n]
2015-10-09 17:21:00,Info,1 replace(s) performed for pattern match [(.*)\r\n\t\r\n(.*)\r\n\t\r\nAdditions\r\n\t\r\n(.*)\r\n\t\r\n(.*)\r\n]
2015-10-09 17:21:00,Info,Output file [P:\Graphics\All Financial Data\Finances\Household.txt] Size: 105 bytes
2015-10-09 17:21:00,Info,Job finished. 1 of 1 files processed, 0 binary files skipped, 0 non-binary files skipped

--------------------

Where did I go wrong?

Re: Filters work individually, not as a group

Posted: Mon Oct 12, 2015 8:05 am
by DataMystic Support
This is a common misconception about TextPipe.

After TextPipe processes the first replace, the text stream is altered, and so the second and subsequent replaces will not work, as the text they are looking for no longer exists in the stream. The Help Menu\Tutorial helps you with these advanced concepts.

The way to do this is to capture the result into a variable e.g. with EasyPatterns, or the capture filter, and then either convert each matching line into an output line, or have one line at the end of the file that is converted to a replacement containing all the matched data.