Improvement suggestion for copybook filter.

Get help with installation and running here.

Moderators: DataMystic Support, Moderators, DataMystic Support, Moderators, DataMystic Support, Moderators

Post Reply
mauros64
Posts: 11
Joined: Tue Oct 13, 2009 7:24 pm

Improvement suggestion for copybook filter.

Post by mauros64 »

I have many situation in which I need to convert a very big and complex mainframe file using the copybook filter.
It work fine but when I need to convert only a limited subset of the available fields is far more faster to delete the unneeded byte range and then use the filter on a copybook modified and restricted to match only the remaining fields.

It will be very useful to let the user specify in a copybook only the needed subset of the fields (for example putting an X character in the first column of the unneeded fields in the copybook, or in some other way) and do the conversion work only for the specified subset.

Thanks
User avatar
DataMystic Support
Site Admin
Posts: 2227
Joined: Mon Jun 30, 2003 12:32 pm
Location: Melbourne, Australia
Contact:

Re: Improvement suggestion for copybook filter.

Post by DataMystic Support »

That sounds like a lot of manual effort to edit the copybook! - and what if it changes?

Why not a filter that removes fields based on column number or by field name, with commas or tabs between each name?

Would that work for you?
mauros64
Posts: 11
Joined: Tue Oct 13, 2009 7:24 pm

Re: Improvement suggestion for copybook filter.

Post by mauros64 »

It sound better to remove field by field name or number.
But ...

If it is after the Copybook that file has already converted 60+ fields ( EBCDIC, Packed Decimal , etc...) just to pass it to the discad filter that keep 8 fields.
If it is before the Copybook it must know the Copybook format.
so to have a good performance (whithout the same knoledge of the copybook filter) it must be inside the Copybook filter.

I usually manage 1-2 GB data files, and discard not needed field is a big performance improvement.
Now I'm using a discard first ( with manual calculation) approach, it works but is a tedious work and error prone.

It's just a suggestion for the future, maybe other people will like it to.

Regards.
User avatar
DataMystic Support
Site Admin
Posts: 2227
Joined: Mon Jun 30, 2003 12:32 pm
Location: Melbourne, Australia
Contact:

Re: Improvement suggestion for copybook filter.

Post by DataMystic Support »

Thanks, I understand what you mean.

Ok, how about a field within the copybook filter that has options to

1. Only keep named fields
2. Discard named fields
3. Only keep numbered fields
4. Discard numbered fields

- so you can specify fields by name or number.

Then this is followed by a field containing space, tab or newline separated field names or numbers. How does that sound?
mauros64
Posts: 11
Joined: Tue Oct 13, 2009 7:24 pm

Re: Improvement suggestion for copybook filter.

Post by mauros64 »

It sound very complete and usefull.
It will be easy to use and it can have grat performance too.
Post Reply