What's the largest amount of data you've processed in one hit with TextPipe?
The current record is 350GB, looking for new contenders!
Also helpful would be the time took, and the general data processing area (e.g. insurance, banking, mainframe, data extraction etc).
Largest amount of data processed?
Moderators: DataMystic Support, Moderators, DataMystic Support, Moderators, DataMystic Support, Moderators
-
DataMystic Support
- Site Admin
- Posts: 2227
- Joined: Mon Jun 30, 2003 12:32 pm
- Location: Melbourne, Australia
- Contact:
-
thelifeofapanca
- Posts: 2
- Joined: Sat Mar 25, 2017 11:17 pm
- Contact:
Re: Largest amount of data processed?
Wow 350!
My record is 300
My record is 300
Re: Largest amount of data processed?
In the past six months I've created 311,000 online listings in Amazon with TPP. So far I've been able to accomplish everything I can imagine using TPP with CSV files.
I've also parsed millions of xml records with TPP.
I keep thinking I should learn a real programming language but whenever I have a problem to solve I find a way to do it with TPP.
TPP + pearl regular expressions = everything possible
I've also parsed millions of xml records with TPP.
I keep thinking I should learn a real programming language but whenever I have a problem to solve I find a way to do it with TPP.
TPP + pearl regular expressions = everything possible
-
DataMystic Support
- Site Admin
- Posts: 2227
- Joined: Mon Jun 30, 2003 12:32 pm
- Location: Melbourne, Australia
- Contact:
Re: Largest amount of data processed?
Newest record is held by https://www.sciohealthanalytics.com/
9GB mainframe input file, generating an 11GB windows output file in 36 minutes
9GB mainframe input file, generating an 11GB windows output file in 36 minutes