Beta Feedback - Split File by Pattern

Bernard Toplak · Post by **Bernard Toplak** » Fri Jun 16, 2006 12:51 am

Hi to Simon and Datamystic Crew,

I am testing this new pattern function in Split File filter. It seems to work OK with text files, but on binaries it fails, making thousands of 0 byte files, and I can't find logical pattern why this happens.

I am testing it for extraction of PNG file fragments from one big binary file. When I simulate PNG start and end strings in plain textual file it splits perfectly, but in binary results as I said above.

I can send you my test files to see by yourself...

Post by **DataMystic Support** » Fri Jun 16, 2006 3:57 pm

Hi Bernard,

Ahem! We actually assumed that no-one would use it one binary files, hence we coded this in two parts
- a search replace for the pattern, which inserts a null character
- a normal split on nulls.

You can workaround this by removing the nulls first, which sounds like it would be useful anyway.

Bernard Toplak · Post by **Bernard Toplak** » Fri Jun 16, 2006 4:47 pm

Simon Carter wrote:... we coded this in two parts
- a search replace for the pattern, which inserts a null character
- a normal split on nulls.

Well, with it's extreme pattern matching power TextPipe is ideal candidate for such jobs of splitting all kinds of files at pattern-principle.

I'm afraid that none of above methods is good for bitmap data as those files use full range of hex chars, so no split-on-char could be performed.

Simon Carter wrote:You can workaround this by removing the nulls first, which sounds like it would be useful anyway.

Hmm, you mean removing \x00 chars? I'm not sure that is a good solution when bitmap data is on the table. If the split on pattern would work with bins just like on text ones, extracting pattern match and ignoring any other unmatching data, without any additional character insertion it would work perfectly!

Bernard Toplak · Post by **Bernard Toplak** » Tue Jun 27, 2006 5:32 am

Hi Simon & Crew!

I just wanted to ask if there will be any moves in this direction?

I have solved my task with one freeware tool that can identify more than 110 different file formats (and much more) in hex files. This can be very usefull for data forensic tasks like mine. If it is permited I can post info for other users that may need it because I see this is out of the TextPipe's main domain, right?

Best regards
Bernard

Post by **DataMystic Support** » Tue Jun 27, 2006 9:28 am

Hi Bernard,

Thanks for asking. Please email the details to me first.

You'll be please to know that we *will* be adding the full split on pattern function in the next day or so.

Bernard Toplak · Post by **Bernard Toplak** » Tue Jun 27, 2006 9:33 am

Great news, Simon!

... I will gladly test it on those hex files I have.

Mail on the way!