Page 1 of 1
create separate files using keywords from a list.
Posted: Wed Jun 02, 2010 12:27 am
by ramin2000
Dear Simon
I am having a very difficult time accomplishing the following.
I have a very long list of keywords on a text file which I want to use to create separate files with.
The list:
Persian-rugs
Indian-rugs
Mashad
Tabriz
Isfahan
Bidjar
Gabbeh
Shiraz
And so on…….
The filter needs to look at rugs.txt file which can be over 10gig and each time it see any of the above keywords any place in the line(not only second column as I told you before),to take that line and put it in a corresponding text file with the same name (with File Header "123456789" for each file):
Persian-rugs.txt
Indian-rugs.txt
Mashad.txt
Tabriz.txt
Isfahan.txt
Bidjar.txt
Gabbeh.txt
Shiraz.txt
And so on…….
I tried to use “Extract lines from list” filter but I am not even sure this is the right way to go.
am a beginner so any instructions must be detailed please or even better I love an attachment filter emailed to me
many thanks
Ramin
Re: create separate files using keywords from a list.
Posted: Wed Jun 02, 2010 8:29 am
by DataMystic Support
We're having an email discussion on this exact topic - where are you getting stuck? I sent you a sample filter (attached) to divert each line type to a new file - you just need to copy the section for each file type - just click on the Restrict to lines matching filter, then click x2 in the toolbar to copy the whole section.
Then adjust the matching text, and the Merge to Filename.
Re: create separate files using keywords from a list.
Posted: Wed Jun 02, 2010 8:42 am
by ramin2000
That is correct, we did talk about this in the email but at the time i was looking at 100 deferent categories.
If I had only 20 of them or 50 or even 100 I could do what you said…the problem is I have 3000+ deferent types/names at the moment and the number change from update to update (I plan on having a public are for people to post rugs for sale so if the guys last name is smith …. His rugs will go under smith.txt…
So……..
I need to be able to upload a list of all the names and send the rugs of Mr. smith to smith.txt.
Does that make sense?
by the way...did not mean to upset you if i did...i am sorry.
Ramin
Re: create separate files using keywords from a list.
Posted: Wed Jun 02, 2010 10:02 am
by DataMystic Support
Not upset
just wondering why you left off email.
Anyway, here is the answer. The attached filter will take a file like this:
Code: Select all
3|Persian-rugs|test
3|Indian-rugs
3|Mashad
3|Tabriz
3|Isfahan|fsdh fds|ds ffs
3|Bidjar
3|Gabbeh
3|Shiraz
and generates JScript code to c:\build_filter.js that looks a bit like this:
Code: Select all
//block for Bidjar
f1 = TPWindow.addGrepFilter( 0, "Bidjar", false, false, true, false, 0, true, false );
TPWindow.startSubfilters();
f2 = TPWindow.addStringFilter( 19, "123456789\r\n", false );
f2 = TPWindow.addSecondaryOutputFilter( 2, 0, false, false, 1, "", "", true, "c:\\Bidjar.txt", false, false, false );
f2 = TPWindow.addSimpleFilter( 48, false );
TPWindow.endSubfilters();
//block for Gabbeh
f1 = TPWindow.addGrepFilter( 0, "Gabbeh", false, false, true, false, 0, true, false );
TPWindow.startSubfilters();
f2 = TPWindow.addStringFilter( 19, "123456789\r\n", false );
f2 = TPWindow.addSecondaryOutputFilter( 2, 0, false, false, 1, "", "", true, "c:\\Gabbeh.txt", false, false, false );
f2 = TPWindow.addSimpleFilter( 48, false );
TPWindow.endSubfilters();
....
1. Run this filter acrosss your 10GB data file to produce c:\build_filter.js - the JavaScript to build the 'real' filter
2. Double-click c:\build_filter.js to build the 'real' filter
3. Run the new filter across your 10GB file.
Re: create separate files using keywords from a list.
Posted: Wed Jun 02, 2010 11:03 am
by ramin2000
when i run the filter i get the following error:
line: 25
char: 44
error: unfermented string constant
code: 800A03F7
Source: microsoft jscript compilation error
Re: create separate files using keywords from a list.
Posted: Wed Jun 02, 2010 11:09 am
by DataMystic Support
Can you paste in the line, along with the 2 lines on either side?
Re: create separate files using keywords from a list.
Posted: Wed Jun 02, 2010 11:27 am
by ramin2000
I have to be honest..i am way over my head here with this….
Lets simplify!
I was trying to do the same thing which is to generate a new filter but the filter kept coming out corrupt.
I can still do this.
what I need from you is a template which I can use to generate this filter and the locations which I can repeat to create a category areas.
For example I will use the example of the rugs-2.zip
Would the following be it?
TGrepObject enabled
GrepObject
[Category]
TStringObject enabled StringObject 123456789
TOutputObject enabled OutputObject .txtC:\ c:\[Category].txt TDescriptionObject enabled DescriptionObject 0
[Category] will be replaced each and every time with the next item on the list as in smith,india-rugs and so on….
Re: create separate files using keywords from a list.
Posted: Wed Jun 02, 2010 11:48 am
by DataMystic Support
Can you please attach here (or email to me) the generated .js code?
What you pasted below is just binary from the filter file and is not useable at all.
Re: create separate files using keywords from a list.
Posted: Wed Jun 02, 2010 11:57 am
by ramin2000
sent to your email....
thanks for your help.
Re: create separate files using keywords from a list.
Posted: Wed Jun 02, 2010 12:48 pm
by DataMystic Support
New filter attached. It is exepcting pipe-delimited data, with rugs in column 2.