Largest amount of data processed?

Get help with installation and running here.

Moderators: DataMystic Support, Moderators, DataMystic Support, Moderators, DataMystic Support, Moderators

Post Reply
User avatar
DataMystic Support
Site Admin
Posts: 2227
Joined: Mon Jun 30, 2003 12:32 pm
Location: Melbourne, Australia
Contact:

Largest amount of data processed?

Post by DataMystic Support »

What's the largest amount of data you've processed in one hit with TextPipe?

The current record is 350GB, looking for new contenders!

Also helpful would be the time took, and the general data processing area (e.g. insurance, banking, mainframe, data extraction etc).
thelifeofapanca
Posts: 2
Joined: Sat Mar 25, 2017 11:17 pm
Contact:

Re: Largest amount of data processed?

Post by thelifeofapanca »

Wow 350!

My record is 300 :(
twaterloo
Posts: 3
Joined: Thu Jan 05, 2012 7:21 am

Re: Largest amount of data processed?

Post by twaterloo »

In the past six months I've created 311,000 online listings in Amazon with TPP. So far I've been able to accomplish everything I can imagine using TPP with CSV files.

I've also parsed millions of xml records with TPP.

I keep thinking I should learn a real programming language but whenever I have a problem to solve I find a way to do it with TPP.

TPP + pearl regular expressions = everything possible
User avatar
DataMystic Support
Site Admin
Posts: 2227
Joined: Mon Jun 30, 2003 12:32 pm
Location: Melbourne, Australia
Contact:

Re: Largest amount of data processed?

Post by DataMystic Support »

Newest record is held by https://www.sciohealthanalytics.com/

9GB mainframe input file, generating an 11GB windows output file in 36 minutes
Post Reply