Page 2 of 2

Re: Suggestion: Provide filter Remove all diacritics

Posted: Fri May 08, 2020 8:23 am
by DataMystic Support
Hi David,

I understand the value, and it might be possible with come caveats.

My concern is that TextPipe has tried hard never to assume the format of incoming files, because auto-detection is generally inprecise. But knowing internally what one filter expects vs one another filter is giving it is do-able.

I can see some potential issues. If two filters were separated by a search/replace, then there is no way of knowing if the search/replace modified the file format. But equally, this might be the express intention of the user, to modify the file format.

Re: Suggestion: Provide filter Remove all diacritics

Posted: Sat May 09, 2020 6:15 pm
by dfhtextpipe
See also my recent email (sent yesterday) about grouping the Unicode filters that are UTF-16LE only.

Aside: hasn't the official Unicode terrminology for UTF-16LE been changed to UCS-2 BOM ?

David

Re: Suggestion: Provide filter Remove all diacritics

Posted: Mon May 11, 2020 8:26 am
by DataMystic Support
Thanks David - had a look but did not find any reference to this terminology change. Do you have a reference?

Re: Suggestion: Provide filter Remove all diacritics

Posted: Tue May 12, 2020 7:24 am
by dfhtextpipe
I was mistaken. UCS-2 was an earlier form that preceded UTF-16.

I was confused by Notepad++ having changed menu options from UTF-16 LE to UCS-2.

It may be because it doesn't fully support the former.

Ah well.

https://en.wikipedia.org/wiki/Universal_Coded_Character_Set

Re: Suggestion: Provide filter Remove all diacritics

Posted: Tue May 12, 2020 7:49 am
by DataMystic Support
Ok, thanks for the clarification!

Re: Suggestion: Provide filter Remove all diacritics

Posted: Sat May 16, 2020 4:54 am
by dfhtextpipe
Please add Help page for the new Remove diacritics filter.

Please add Help page to explain Filter Library\Unicode\UTF-16LE only

Please add See also links in these existing filter help pages to the new page.

David

Re: Suggestion: Provide filter Remove all diacritics

Posted: Sat May 16, 2020 8:02 pm
by DataMystic Support
Remove diacritics is ready for TP 11.6. It will handle UTF16-LE - any other format can be converted using other TP filters.