The suggestion above is excellent, but in this very moment it would not work because of an issue between Notepad++ and "Notepad++ Python Script". Since some months Notepad++ plug in manager downloads an old Python Script version that won't work with the editor. To fix that:
- Exit Notepad++
- Download the compatible version from SourceForge.
- Run the downloaded installer by double clicking it. On newer Windows it'll ask to switch to Administrator privileges.
- Make sure to pick the correct install drive at the beginning of the install process. It won't detect the Notepad++ installation disk correctly. I had to reinstall it again because by default it installs on C:\ even if Notepad++ is on another disk.
- Follow the wizard instructions.
- Once finished with the install process, (re)start Notepad++. Now go and open the Plugin menu. You should see a new "Python Script" item inside it. If it appears then you have force-installed the correct version well. You may also double check by opening the Plug in manager, going to the "Installed" tab and looking for an entry showing version 1.0.8 (at this time) of the Python plug in being present.
You are almost done. Go to the Plugins => Python Script => Show console menu. A pane shall appear at the bottom of Notepad++. It MUST show a prompt like the following:
Python 2.7.6-notepad++ r2 (default, Apr 21 2014, 19:26:54) [MSC v.1600 32 bit (Intel)] Initialisation took 156ms Ready.
The various version numbers are current as of today, of course they shall change as time goes on. If the bottom pane shows an exception stating an exception occurred or (and) it stays blank, then you have installed a wrong Python plug in version.
Now, let's apply the script in the correct way:
- Open two new, blank tabs/files.
- Paste your accented text in the first.
- Right click the tab of the second and select the 'Move to Other View' menu. The Notepad++ windows will split.
- Open the Python Script console as explained above (Plugins => Python Script => Show console menu).
- Go to the console line at the bottom of the Python pane, it has a ">>>" marking at its beginning.
- Type:
from Npp import *and then press Enter (from now assume you'll always press Enter at the end of the commands). - Enter:
import unicodedatain the same input text. - Click (select) the tab containing the accented text (this is important!).
Enter the following commands, one line at a time, in the Python prompt and then press Enter after each line:
eText = editor.getText() uText = unicode(eText, "UTF-8") nText = unicodedata.normalize( "NFKD", uText )
If you want to be sure Python "really got the text in": after you typed eText = editor.getText() (+ Enter key), enter: print eText + Enter. You should see your accented text dumped in the Python console output pane.
- Click (select) the empty tab (this is important!).
- Enter:
editor.addText( nText.encode('ASCII', 'ignore') )in the usual Python console command input text box. - The empty tab shall fill in with the converted, accent-less text. Make sure to follow this to-do list carefully because it's easy to miss a step (expecially clicking the tabs) and then you'll have to restart from scratch.
The suggestion above is excellent, but in this very moment it would not work because of an issue between Notepad++ and "Notepad++ Python Script". Since some months Notepad++ plug in manager downloads an old Python Script version that won't work with the editor. To fix that:
- Exit Notepad++
- Download the compatible version from SourceForge.
- Run the downloaded installer by double clicking it. On newer Windows it'll ask to switch to Administrator privileges.
- Make sure to pick the correct install drive at the beginning of the install process. It won't detect the Notepad++ installation disk correctly. I had to reinstall it again because by default it installs on C:\ even if Notepad++ is on another disk.
- Follow the wizard instructions.
- Once finished with the install process, (re)start Notepad++. Now go and open the Plugin menu. You should see a new "Python Script" item inside it. If it appears then you have force-installed the correct version well. You may also double check by opening the Plug in manager, going to the "Installed" tab and looking for an entry showing version 1.0.8 (at this time) of the Python plug in being present.
You are almost done. Go to the Plugins => Python Script => Show console menu. A pane shall appear at the bottom of Notepad++. It MUST show a prompt like the following:
Python 2.7.6-notepad++ r2 (default, Apr 21 2014, 19:26:54) [MSC v.1600 32 bit (Intel)] Initialisation took 156ms Ready.
The various version numbers are current as of today, of course they shall change as time goes on. If the bottom pane shows an exception stating an exception occurred or (and) it stays blank, then you have installed a wrong Python plug in version.
Now, let's apply the script in the correct way:
- Open two new, blank tabs/files.
- Paste your accented text in the first.
- Right click the tab of the second and select the 'Move to Other View' menu. The Notepad++ windows will split.
- Open the Python Script console as explained above (Plugins => Python Script => Show console menu).
- Go to the console line at the bottom of the Python pane, it has a ">>>" marking at its beginning.
- Type:
from Npp import *and then press Enter (from now assume you'll always press Enter at the end of the commands). - Enter:
import unicodedatain the same input text. - Click (select) the tab containing the accented text (this is important!).
Enter the following commands, one line at a time, in the Python prompt and then press Enter after each line:
eText = editor.getText() uText = unicode(eText, "UTF-8") nText = unicodedata.normalize( "NFKD", uText )
If you want to be sure Python "really got the text in": after you typed eText = editor.getText() (+ Enter key), enter: print eText + Enter. You should see your accented text dumped in the Python console output pane.
- Click (select) the empty tab (this is important!).
- Enter:
editor.addText( nText.encode('ASCII', 'ignore') )in the usual Python console command input text box. - The empty tab shall fill in with the converted, accent-less text. Make sure to follow this to-do list carefully because it's easy to miss a step (expecially clicking the tabs) and then you'll have to restart from scratch.
Don't be restricted by what you see as being available. We have python available from within N++, and that means a quick SO search [python] [unicode] remove accents reveals this highly voted question dealing with exactly that!
We can test easily enough in N++ to see how it works::
- Open two new buffers/tabs/files or whatever you like calling them.
- Copy and paste these Latin Extended UTF-8 Characters into the first.
- Right click the tab of the second and 'Move to Other View'.
Open the Python Script console and try the following commands::
from Npp import *
import unicodedata
<< Select tab in view 1. >>
eText = editor.getText() << Select tab in view 2. >>
uText = unicode(eText, "UTF-8")
nText = unicodedata.normalize( "NFKD", uText )
editor.addText( nText.encode('ASCII', 'ignore') )
From looking around a bit it seems there are lots of ways to accomplish removing accents, the question is which works best for you. And now that you can see how easy it is to 'try' these solutions on your text, go forth and give it a shot. Once you like a particular method add it (using the plugin menu) as a script and it will be there whenever you need it.
Have fun!
BTW - if you don't have Python Script installed you can install it using the Plugins -> Plugin Manager.
how to replace all accented characters with English equivalents
How to replace accents character with regular expressions - Looker Studio Community
Replace accented and special characters from a string
Replace accents with normal letters
The correct terminology for such accents is Diacritics. After Googling this term, I found this function which is part of backbone.paginator. It has a very complete collection of Diacritics and replaces them with their most intuitive ascii character. I found this to be the most complete Javascript solution available today.
The full function for future reference:
function removeDiacritics (str) {
var defaultDiacriticsRemovalMap = [
{'base':'A', 'letters':/[\u0041\u24B6\uFF21\u00C0\u00C1\u00C2\u1EA6\u1EA4\u1EAA\u1EA8\u00C3\u0100\u0102\u1EB0\u1EAE\u1EB4\u1EB2\u0226\u01E0\u00C4\u01DE\u1EA2\u00C5\u01FA\u01CD\u0200\u0202\u1EA0\u1EAC\u1EB6\u1E00\u0104\u023A\u2C6F]/g},
{'base':'AA','letters':/[\uA732]/g},
{'base':'AE','letters':/[\u00C6\u01FC\u01E2]/g},
{'base':'AO','letters':/[\uA734]/g},
{'base':'AU','letters':/[\uA736]/g},
{'base':'AV','letters':/[\uA738\uA73A]/g},
{'base':'AY','letters':/[\uA73C]/g},
{'base':'B', 'letters':/[\u0042\u24B7\uFF22\u1E02\u1E04\u1E06\u0243\u0182\u0181]/g},
{'base':'C', 'letters':/[\u0043\u24B8\uFF23\u0106\u0108\u010A\u010C\u00C7\u1E08\u0187\u023B\uA73E]/g},
{'base':'D', 'letters':/[\u0044\u24B9\uFF24\u1E0A\u010E\u1E0C\u1E10\u1E12\u1E0E\u0110\u018B\u018A\u0189\uA779]/g},
{'base':'DZ','letters':/[\u01F1\u01C4]/g},
{'base':'Dz','letters':/[\u01F2\u01C5]/g},
{'base':'E', 'letters':/[\u0045\u24BA\uFF25\u00C8\u00C9\u00CA\u1EC0\u1EBE\u1EC4\u1EC2\u1EBC\u0112\u1E14\u1E16\u0114\u0116\u00CB\u1EBA\u011A\u0204\u0206\u1EB8\u1EC6\u0228\u1E1C\u0118\u1E18\u1E1A\u0190\u018E]/g},
{'base':'F', 'letters':/[\u0046\u24BB\uFF26\u1E1E\u0191\uA77B]/g},
{'base':'G', 'letters':/[\u0047\u24BC\uFF27\u01F4\u011C\u1E20\u011E\u0120\u01E6\u0122\u01E4\u0193\uA7A0\uA77D\uA77E]/g},
{'base':'H', 'letters':/[\u0048\u24BD\uFF28\u0124\u1E22\u1E26\u021E\u1E24\u1E28\u1E2A\u0126\u2C67\u2C75\uA78D]/g},
{'base':'I', 'letters':/[\u0049\u24BE\uFF29\u00CC\u00CD\u00CE\u0128\u012A\u012C\u0130\u00CF\u1E2E\u1EC8\u01CF\u0208\u020A\u1ECA\u012E\u1E2C\u0197]/g},
{'base':'J', 'letters':/[\u004A\u24BF\uFF2A\u0134\u0248]/g},
{'base':'K', 'letters':/[\u004B\u24C0\uFF2B\u1E30\u01E8\u1E32\u0136\u1E34\u0198\u2C69\uA740\uA742\uA744\uA7A2]/g},
{'base':'L', 'letters':/[\u004C\u24C1\uFF2C\u013F\u0139\u013D\u1E36\u1E38\u013B\u1E3C\u1E3A\u0141\u023D\u2C62\u2C60\uA748\uA746\uA780]/g},
{'base':'LJ','letters':/[\u01C7]/g},
{'base':'Lj','letters':/[\u01C8]/g},
{'base':'M', 'letters':/[\u004D\u24C2\uFF2D\u1E3E\u1E40\u1E42\u2C6E\u019C]/g},
{'base':'N', 'letters':/[\u004E\u24C3\uFF2E\u01F8\u0143\u00D1\u1E44\u0147\u1E46\u0145\u1E4A\u1E48\u0220\u019D\uA790\uA7A4]/g},
{'base':'NJ','letters':/[\u01CA]/g},
{'base':'Nj','letters':/[\u01CB]/g},
{'base':'O', 'letters':/[\u004F\u24C4\uFF2F\u00D2\u00D3\u00D4\u1ED2\u1ED0\u1ED6\u1ED4\u00D5\u1E4C\u022C\u1E4E\u014C\u1E50\u1E52\u014E\u022E\u0230\u00D6\u022A\u1ECE\u0150\u01D1\u020C\u020E\u01A0\u1EDC\u1EDA\u1EE0\u1EDE\u1EE2\u1ECC\u1ED8\u01EA\u01EC\u00D8\u01FE\u0186\u019F\uA74A\uA74C]/g},
{'base':'OI','letters':/[\u01A2]/g},
{'base':'OO','letters':/[\uA74E]/g},
{'base':'OU','letters':/[\u0222]/g},
{'base':'P', 'letters':/[\u0050\u24C5\uFF30\u1E54\u1E56\u01A4\u2C63\uA750\uA752\uA754]/g},
{'base':'Q', 'letters':/[\u0051\u24C6\uFF31\uA756\uA758\u024A]/g},
{'base':'R', 'letters':/[\u0052\u24C7\uFF32\u0154\u1E58\u0158\u0210\u0212\u1E5A\u1E5C\u0156\u1E5E\u024C\u2C64\uA75A\uA7A6\uA782]/g},
{'base':'S', 'letters':/[\u0053\u24C8\uFF33\u1E9E\u015A\u1E64\u015C\u1E60\u0160\u1E66\u1E62\u1E68\u0218\u015E\u2C7E\uA7A8\uA784]/g},
{'base':'T', 'letters':/[\u0054\u24C9\uFF34\u1E6A\u0164\u1E6C\u021A\u0162\u1E70\u1E6E\u0166\u01AC\u01AE\u023E\uA786]/g},
{'base':'TZ','letters':/[\uA728]/g},
{'base':'U', 'letters':/[\u0055\u24CA\uFF35\u00D9\u00DA\u00DB\u0168\u1E78\u016A\u1E7A\u016C\u00DC\u01DB\u01D7\u01D5\u01D9\u1EE6\u016E\u0170\u01D3\u0214\u0216\u01AF\u1EEA\u1EE8\u1EEE\u1EEC\u1EF0\u1EE4\u1E72\u0172\u1E76\u1E74\u0244]/g},
{'base':'V', 'letters':/[\u0056\u24CB\uFF36\u1E7C\u1E7E\u01B2\uA75E\u0245]/g},
{'base':'VY','letters':/[\uA760]/g},
{'base':'W', 'letters':/[\u0057\u24CC\uFF37\u1E80\u1E82\u0174\u1E86\u1E84\u1E88\u2C72]/g},
{'base':'X', 'letters':/[\u0058\u24CD\uFF38\u1E8A\u1E8C]/g},
{'base':'Y', 'letters':/[\u0059\u24CE\uFF39\u1EF2\u00DD\u0176\u1EF8\u0232\u1E8E\u0178\u1EF6\u1EF4\u01B3\u024E\u1EFE]/g},
{'base':'Z', 'letters':/[\u005A\u24CF\uFF3A\u0179\u1E90\u017B\u017D\u1E92\u1E94\u01B5\u0224\u2C7F\u2C6B\uA762]/g},
{'base':'a', 'letters':/[\u0061\u24D0\uFF41\u1E9A\u00E0\u00E1\u00E2\u1EA7\u1EA5\u1EAB\u1EA9\u00E3\u0101\u0103\u1EB1\u1EAF\u1EB5\u1EB3\u0227\u01E1\u00E4\u01DF\u1EA3\u00E5\u01FB\u01CE\u0201\u0203\u1EA1\u1EAD\u1EB7\u1E01\u0105\u2C65\u0250]/g},
{'base':'aa','letters':/[\uA733]/g},
{'base':'ae','letters':/[\u00E6\u01FD\u01E3]/g},
{'base':'ao','letters':/[\uA735]/g},
{'base':'au','letters':/[\uA737]/g},
{'base':'av','letters':/[\uA739\uA73B]/g},
{'base':'ay','letters':/[\uA73D]/g},
{'base':'b', 'letters':/[\u0062\u24D1\uFF42\u1E03\u1E05\u1E07\u0180\u0183\u0253]/g},
{'base':'c', 'letters':/[\u0063\u24D2\uFF43\u0107\u0109\u010B\u010D\u00E7\u1E09\u0188\u023C\uA73F\u2184]/g},
{'base':'d', 'letters':/[\u0064\u24D3\uFF44\u1E0B\u010F\u1E0D\u1E11\u1E13\u1E0F\u0111\u018C\u0256\u0257\uA77A]/g},
{'base':'dz','letters':/[\u01F3\u01C6]/g},
{'base':'e', 'letters':/[\u0065\u24D4\uFF45\u00E8\u00E9\u00EA\u1EC1\u1EBF\u1EC5\u1EC3\u1EBD\u0113\u1E15\u1E17\u0115\u0117\u00EB\u1EBB\u011B\u0205\u0207\u1EB9\u1EC7\u0229\u1E1D\u0119\u1E19\u1E1B\u0247\u025B\u01DD]/g},
{'base':'f', 'letters':/[\u0066\u24D5\uFF46\u1E1F\u0192\uA77C]/g},
{'base':'g', 'letters':/[\u0067\u24D6\uFF47\u01F5\u011D\u1E21\u011F\u0121\u01E7\u0123\u01E5\u0260\uA7A1\u1D79\uA77F]/g},
{'base':'h', 'letters':/[\u0068\u24D7\uFF48\u0125\u1E23\u1E27\u021F\u1E25\u1E29\u1E2B\u1E96\u0127\u2C68\u2C76\u0265]/g},
{'base':'hv','letters':/[\u0195]/g},
{'base':'i', 'letters':/[\u0069\u24D8\uFF49\u00EC\u00ED\u00EE\u0129\u012B\u012D\u00EF\u1E2F\u1EC9\u01D0\u0209\u020B\u1ECB\u012F\u1E2D\u0268\u0131]/g},
{'base':'j', 'letters':/[\u006A\u24D9\uFF4A\u0135\u01F0\u0249]/g},
{'base':'k', 'letters':/[\u006B\u24DA\uFF4B\u1E31\u01E9\u1E33\u0137\u1E35\u0199\u2C6A\uA741\uA743\uA745\uA7A3]/g},
{'base':'l', 'letters':/[\u006C\u24DB\uFF4C\u0140\u013A\u013E\u1E37\u1E39\u013C\u1E3D\u1E3B\u017F\u0142\u019A\u026B\u2C61\uA749\uA781\uA747]/g},
{'base':'lj','letters':/[\u01C9]/g},
{'base':'m', 'letters':/[\u006D\u24DC\uFF4D\u1E3F\u1E41\u1E43\u0271\u026F]/g},
{'base':'n', 'letters':/[\u006E\u24DD\uFF4E\u01F9\u0144\u00F1\u1E45\u0148\u1E47\u0146\u1E4B\u1E49\u019E\u0272\u0149\uA791\uA7A5]/g},
{'base':'nj','letters':/[\u01CC]/g},
{'base':'o', 'letters':/[\u006F\u24DE\uFF4F\u00F2\u00F3\u00F4\u1ED3\u1ED1\u1ED7\u1ED5\u00F5\u1E4D\u022D\u1E4F\u014D\u1E51\u1E53\u014F\u022F\u0231\u00F6\u022B\u1ECF\u0151\u01D2\u020D\u020F\u01A1\u1EDD\u1EDB\u1EE1\u1EDF\u1EE3\u1ECD\u1ED9\u01EB\u01ED\u00F8\u01FF\u0254\uA74B\uA74D\u0275]/g},
{'base':'oi','letters':/[\u01A3]/g},
{'base':'ou','letters':/[\u0223]/g},
{'base':'oo','letters':/[\uA74F]/g},
{'base':'p','letters':/[\u0070\u24DF\uFF50\u1E55\u1E57\u01A5\u1D7D\uA751\uA753\uA755]/g},
{'base':'q','letters':/[\u0071\u24E0\uFF51\u024B\uA757\uA759]/g},
{'base':'r','letters':/[\u0072\u24E1\uFF52\u0155\u1E59\u0159\u0211\u0213\u1E5B\u1E5D\u0157\u1E5F\u024D\u027D\uA75B\uA7A7\uA783]/g},
{'base':'s','letters':/[\u0073\u24E2\uFF53\u00DF\u015B\u1E65\u015D\u1E61\u0161\u1E67\u1E63\u1E69\u0219\u015F\u023F\uA7A9\uA785\u1E9B]/g},
{'base':'t','letters':/[\u0074\u24E3\uFF54\u1E6B\u1E97\u0165\u1E6D\u021B\u0163\u1E71\u1E6F\u0167\u01AD\u0288\u2C66\uA787]/g},
{'base':'tz','letters':/[\uA729]/g},
{'base':'u','letters':/[\u0075\u24E4\uFF55\u00F9\u00FA\u00FB\u0169\u1E79\u016B\u1E7B\u016D\u00FC\u01DC\u01D8\u01D6\u01DA\u1EE7\u016F\u0171\u01D4\u0215\u0217\u01B0\u1EEB\u1EE9\u1EEF\u1EED\u1EF1\u1EE5\u1E73\u0173\u1E77\u1E75\u0289]/g},
{'base':'v','letters':/[\u0076\u24E5\uFF56\u1E7D\u1E7F\u028B\uA75F\u028C]/g},
{'base':'vy','letters':/[\uA761]/g},
{'base':'w','letters':/[\u0077\u24E6\uFF57\u1E81\u1E83\u0175\u1E87\u1E85\u1E98\u1E89\u2C73]/g},
{'base':'x','letters':/[\u0078\u24E7\uFF58\u1E8B\u1E8D]/g},
{'base':'y','letters':/[\u0079\u24E8\uFF59\u1EF3\u00FD\u0177\u1EF9\u0233\u1E8F\u00FF\u1EF7\u1E99\u1EF5\u01B4\u024F\u1EFF]/g},
{'base':'z','letters':/[\u007A\u24E9\uFF5A\u017A\u1E91\u017C\u017E\u1E93\u1E95\u01B6\u0225\u0240\u2C6C\uA763]/g}
];
for(var i=0; i<defaultDiacriticsRemovalMap.length; i++) {
str = str.replace(defaultDiacriticsRemovalMap[i].letters, defaultDiacriticsRemovalMap[i].base);
}
return str;
}
Since those characters have no mathematical relation to their 'plain equivalents' in the unicode table you will have to replace them manually using something like this:
function cleanUpSpecialChars(str)
{
return str
.replace(/[ÀÁÂÃÄÅ]/g,"A")
.replace(/[àáâãäå]/g,"a")
.replace(/[ÈÉÊË]/g,"E")
//.... all the rest
.replace(/[^a-z0-9]/gi,''); // final clean up
}
The case-insensitve option doesn't work on those characters, so you have to do it for the lower and upper case variants of them.
I am trying to find a way to replace all accented characters. I currently have a iOS shortcut that uses this regex that matches all the accented characters this I believe uses pcre2
[\u00E0-\u00FC]
I then use a replace for each letter Eg
Match (à)|(á)|(â)|(ä)|(ã)|(À)|(Á)|(Â)|(Ä)|(Ã)+ Replace with a
Etc etc for each accented character
Is there a regex that will only find the accented character and replace with it’s English equivalent in one go ?? Other than lopping through each letter replacing each letter separately
Here’s the example shortcut to show what I mean
https://www.icloud.com/shortcuts/2d7142ca0c9b48c39fc380ac30449d38
I do not see the option to run the macro in my macros list. The macro name is not appearing in the list to select. I have macros enabled and I have a bunch of others I use so I do not understand why it's not showing. – BvilleBullet 4 mins ago
Please see the comment in the above code.
'-- Usage: StripAccent Sheet1.Range("A1:C20")
You have to call it like this
Option Explicit
'-- Add more chars to these 2 string as you want
'-- You may have problem with unicode chars that has code > 255
'-- such as some Vietnamese characters that are outside of ASCII code (0-255)
Const AccChars = "ŠŽšžŸÀÁÂÃÄÅÇÈÉÊËÌÍÎÏÐÑÒÓÔÕÖÙÚÛÜÝàáâãäåçèéêëìíîïðñòóôõöùúûüýÿ"
Const RegChars = "SZszYAAAAAACEEEEIIIIDNOOOOOUUUUYaaaaaaceeeeiiiidnooooouuuuyy"
'~~> This is how you have to call it. Now You can see the macro "Sample" in the list
Sub Sample()
StripAccent Sheet1.Range("A1:C20")
End Sub
Sub StripAccent(aRange As Range)
'-- Usage: StripAccent Sheet1.Range("A1:C20")
Dim A As String * 1
Dim B As String * 1
Dim i As Integer
For i = 1 To Len(AccChars)
A = Mid(AccChars, i, 1)
B = Mid(RegChars, i, 1)
aRange.Replace What:=A, _
Replacement:=B, _
LookAt:=xlPart, _
MatchCase:=True
Next
End Sub
Function stripAccent(Text As String) As String
Const AccChars = "ŠŽšžŸÀÁÂÃÄÅÇÈÉÊËÌÍÎÏÐÑÒÓÔÕÖÙÚÛÜÝàáâãäåçèéêëìíîïðñòóôõöùúûüýÿ"
Const RegChars = "SZszYAAAAAACEEEEIIIIDNOOOOOUUUUYaaaaaaceeeeiiiidnooooouuuuyy"
Dim A As String * 1
Dim B As String * 1
Dim i As Integer
For i = 1 To Len(AccChars)
A = Mid(AccChars, i, 1)
B = Mid(RegChars, i, 1)
Text = Replace(Text, A, B)
Next
stripAccent = Text
End Function
If you want to match just a single character that has an accent, you can simply include it in a search:
:%s/café/restaurant/g
You can type the accented character either using your operating system's input features, or with Vim's digraphs feature (see :help c_CTRL-k)
If you want to ignore the accents, you can use an equivalence class (See :help /[[=).
e.g. Replace both cafe and café with restaurant:
:%s/caf[[=e=]]/restaurant/g
Replace all e-like characters with ASCII e characters:
:%s/[[=e=]]/e/g
The accents are part of the word's spelling. You're asking us how to use vi to take a document with correctly-spelled words and introduce spelling errors.
The days where you could get away with ignoring Unicode are long past.
There are languages without a J, such as classical Latin. Would you not be annoyed if people called you "Iason335" on this site?
Of the vi clones, Vim has by far the best Unicode support. Vim likes UTF-8 best, though on Windows UTF-16 is sometimes a better idea. See the unicode section of the Vim manual for more information.
If you absolutely must crush Unicode back to a legacy encoding such as the ancient 7-bit ASCII standard, there are several options I'd choose before resorting to a blind vi substitution command:
Replace the words on a case-by-case basis using human intelligence, since accented letters don't have the same meaning in all human languages, so there is no dumb repetitive rule you can run that works in all cases:
The Germanic name Jörg is best spelled Joerg in ASCII, not "Jorg".
But please beware: the "ö" → "oe" rule only applies to German. It doesn't apply even to closely-related languages like Dutch, though the same process exists with different orthography.
English is largely German-derived, but in properly-spelled English, the two-dot accent on top of a letter is almost always a diaeresis, not an umlaut.¹
For instance, the English word "cooperate" is sometimes spelled "coöperate" to indicate that the first syllable is "co," not "coop," as in "chicken coop." The proper ASCII-ization is therefore not "cooeperate".²
For another example, the English surname Brontë uses a diaeresis to indicate that the "e" isn't silent, as an English speaker may well expect, causing it to be pronounced as if it were spelled "Bront". The diaeresis tells us that the final "e" is a separate syllable, so the name must be properly pronounced "BRONT-ay". The proper ASCII-ization is "Bronte", not "Brontee" or something else.³
In the Swedish alphabet, "ä" and "ö" are considered separate letters in their own right rather than accented versions of other letters; there is no umlaut as such in Swedish. The ASCII transliteration rule for German is simply incorrect for Swedish. For example, the Swedish name Ångström is best ASCII-ized as "Angstrom" not "Angstroem".
Use
iconv(1)to convert the file automatically.This is likely to do less damage to your file than a blind
visubstitution because the conversion routines were written by experts in the topic, who have thought through issues like the above and more.If your target is plain ASCII, you will have to use the
//translitmodifier, the//ignoremodifier, or the-cflag to make this work. I recommend trying them in that order. For example, on the input text "Please coöperate with Jörg.",iconv -f utf-8 -t ascii//translitgivesPlease coo"perate with Jo"rg.as output. It is doing its best to show that the input text once had dots over the letters.
A better plan would be to use an 8-bit extension to ASCII that also works on your system, such as ISO 8859-1, which can encode the examples I've given so far directly:
iconv -f utf-8 -t iso-8851-1.You can use
iconvfrom withinviusing the various pipe commands. For example, to read a UTF-8-encoded filexas ASCII into the current buffer::r !iconv -t ascii//translit xOr, you could open a file as Unicode, then mark a section of it using normal
vicommands (e.g. Vim'sVblock selection command) then convert it in place usingiconv::'<,'>!iconv -t ascii//translitYou don't have to type the first part if starting from a Vim block selection: the first character you type is the
!, and the rest is filled in automatically. You do have to type it all if using other types ofvimarks.Use the Perl module
Text::Unidecodeinstead oficonv. Because it isn't part of the Perl core, you'll have to install it by hand.Beware that it merely provides a different sort of "wrong" than
iconv //translit. Its documentation is well worth reading, particularly the section "When You Don't Like What Unidecode Does".You can use that from within Vim using this mapping in your
~/.vimrcfile:map <F6> !}perl -C -MText::Unidecode -ne 'print unidecode( $_)'<CR>Pressing F6 will convert the current paragraph.
Asides:
Excepting loanwords, of course.
Some people spell the word "co-operate" to side-step this problem.
There is no "ë" in German, so an "ö" → "oe" type of rule could not properly come into effect here anyway.
You can try iconv, with the //TRANSLIT (transliteration) option
Ex. given
$ cat file
ë
ê
Ý,text
Ò
É
then
$ iconv -t ASCII//TRANSLIT file
e
e
Y,text
O
E
The GNU recode package is very useful to convert between character encodings, and it has a special case that does exactly this with the "flat" encoding:
recode -f utf8..flat <textin.txt >flattext.out