create separate files using keywords from a list.

Get help with installation and running here.

Moderators: DataMystic Support, Moderators, DataMystic Support, Moderators, DataMystic Support, Moderators

Post Reply
ramin2000
Posts: 11
Joined: Wed Jun 02, 2010 12:02 am

create separate files using keywords from a list.

Post by ramin2000 »

Dear Simon
I am having a very difficult time accomplishing the following.
I have a very long list of keywords on a text file which I want to use to create separate files with.

The list:

Persian-rugs
Indian-rugs
Mashad
Tabriz
Isfahan
Bidjar
Gabbeh
Shiraz
And so on…….

The filter needs to look at rugs.txt file which can be over 10gig and each time it see any of the above keywords any place in the line(not only second column as I told you before),to take that line and put it in a corresponding text file with the same name (with File Header "123456789" for each file):

Persian-rugs.txt
Indian-rugs.txt
Mashad.txt
Tabriz.txt
Isfahan.txt
Bidjar.txt
Gabbeh.txt
Shiraz.txt
And so on…….

I tried to use “Extract lines from list” filter but I am not even sure this is the right way to go.
am a beginner so any instructions must be detailed please or even better I love an attachment filter emailed to me :D
many thanks
Ramin
User avatar
DataMystic Support
Site Admin
Posts: 2227
Joined: Mon Jun 30, 2003 12:32 pm
Location: Melbourne, Australia
Contact:

Re: create separate files using keywords from a list.

Post by DataMystic Support »

We're having an email discussion on this exact topic - where are you getting stuck? I sent you a sample filter (attached) to divert each line type to a new file - you just need to copy the section for each file type - just click on the Restrict to lines matching filter, then click x2 in the toolbar to copy the whole section.

Then adjust the matching text, and the Merge to Filename.
Attachments
rugs-2.zip
(1.01 KiB) Downloaded 480 times
ramin2000
Posts: 11
Joined: Wed Jun 02, 2010 12:02 am

Re: create separate files using keywords from a list.

Post by ramin2000 »

That is correct, we did talk about this in the email but at the time i was looking at 100 deferent categories.
If I had only 20 of them or 50 or even 100 I could do what you said…the problem is I have 3000+ deferent types/names at the moment and the number change from update to update (I plan on having a public are for people to post rugs for sale so if the guys last name is smith …. His rugs will go under smith.txt…
So……..
I need to be able to upload a list of all the names and send the rugs of Mr. smith to smith.txt.

Does that make sense?
by the way...did not mean to upset you if i did...i am sorry.

Ramin
User avatar
DataMystic Support
Site Admin
Posts: 2227
Joined: Mon Jun 30, 2003 12:32 pm
Location: Melbourne, Australia
Contact:

Re: create separate files using keywords from a list.

Post by DataMystic Support »

Not upset :-) just wondering why you left off email.

Anyway, here is the answer. The attached filter will take a file like this:

Code: Select all

3|Persian-rugs|test
3|Indian-rugs
3|Mashad
3|Tabriz
3|Isfahan|fsdh fds|ds ffs
3|Bidjar
3|Gabbeh
3|Shiraz
and generates JScript code to c:\build_filter.js that looks a bit like this:

Code: Select all

//block for Bidjar
f1 = TPWindow.addGrepFilter( 0, "Bidjar", false, false, true, false, 0, true, false );
TPWindow.startSubfilters();
f2 = TPWindow.addStringFilter( 19, "123456789\r\n", false );
f2 = TPWindow.addSecondaryOutputFilter( 2, 0, false, false, 1, "", "", true, "c:\\Bidjar.txt", false, false, false );
f2 = TPWindow.addSimpleFilter( 48, false );
TPWindow.endSubfilters();

//block for Gabbeh
f1 = TPWindow.addGrepFilter( 0, "Gabbeh", false, false, true, false, 0, true, false );
TPWindow.startSubfilters();
f2 = TPWindow.addStringFilter( 19, "123456789\r\n", false );
f2 = TPWindow.addSecondaryOutputFilter( 2, 0, false, false, 1, "", "", true, "c:\\Gabbeh.txt", false, false, false );
f2 = TPWindow.addSimpleFilter( 48, false );
TPWindow.endSubfilters();

....
1. Run this filter acrosss your 10GB data file to produce c:\build_filter.js - the JavaScript to build the 'real' filter
2. Double-click c:\build_filter.js to build the 'real' filter
3. Run the new filter across your 10GB file.
Attachments
rugs.zip
(1.93 KiB) Downloaded 453 times
ramin2000
Posts: 11
Joined: Wed Jun 02, 2010 12:02 am

Re: create separate files using keywords from a list.

Post by ramin2000 »

when i run the filter i get the following error:
line: 25
char: 44
error: unfermented string constant
code: 800A03F7
Source: microsoft jscript compilation error
:(
User avatar
DataMystic Support
Site Admin
Posts: 2227
Joined: Mon Jun 30, 2003 12:32 pm
Location: Melbourne, Australia
Contact:

Re: create separate files using keywords from a list.

Post by DataMystic Support »

Can you paste in the line, along with the 2 lines on either side?
ramin2000
Posts: 11
Joined: Wed Jun 02, 2010 12:02 am

Re: create separate files using keywords from a list.

Post by ramin2000 »

I have to be honest..i am way over my head here with this….
Lets simplify!
I was trying to do the same thing which is to generate a new filter but the filter kept coming out corrupt.
I can still do this.
what I need from you is a template which I can use to generate this filter and the locations which I can repeat to create a category areas.
For example I will use the example of the rugs-2.zip
Would the following be it?



TGrepObject enabled
GrepObject
[Category]
TStringObject enabled StringObject 123456789

TOutputObject enabled OutputObject .txtC:\ c:\[Category].txt TDescriptionObject enabled DescriptionObject 0


[Category] will be replaced each and every time with the next item on the list as in smith,india-rugs and so on….
User avatar
DataMystic Support
Site Admin
Posts: 2227
Joined: Mon Jun 30, 2003 12:32 pm
Location: Melbourne, Australia
Contact:

Re: create separate files using keywords from a list.

Post by DataMystic Support »

Can you please attach here (or email to me) the generated .js code?

What you pasted below is just binary from the filter file and is not useable at all.
ramin2000
Posts: 11
Joined: Wed Jun 02, 2010 12:02 am

Re: create separate files using keywords from a list.

Post by ramin2000 »

sent to your email....
thanks for your help.
User avatar
DataMystic Support
Site Admin
Posts: 2227
Joined: Mon Jun 30, 2003 12:32 pm
Location: Melbourne, Australia
Contact:

Re: create separate files using keywords from a list.

Post by DataMystic Support »

New filter attached. It is exepcting pipe-delimited data, with rugs in column 2.
Attachments
rugs.zip
(1.93 KiB) Downloaded 464 times
Post Reply