As mentioned in the comments, your question isn't very specific, so I'll try to give you some hints about character encodings, see if you can apply those to your specific case!

Unicode and Encoding

Here's a small primer about encoding. Basically, there are two ways to represent text in Python:

  • unicode. You can consider that unicode is the ultimate encoding, you should strive to use it everywhere. In Python 2.x source files, unicode strings look like u'some unicode'.
  • str. This is encoded text - to be able to read it, you need to know the encoding (or guess it). In Python 2.x, those strings look like 'some str'.

This changed in Python 3 (unicode is now str and str is now bytes).

How does that play out?

Usually, it's pretty straightforward to ensure that you code uses unicode for its execution, and uses str for I/O:

  • Everything you receive is encoded, so you do input_string.decode('encoding') to convert it to unicode.
  • Everything you need to output is unicode but needs to be encoded, so you do output_string.encode('encoding').

The most common encodings are cp-1252 on Windows (on US or EU systems), and utf-8 on Linux.

Applying this to your case

I DO have to write äöü in a path, or it will not work

Windows natively uses unicode for file paths and names, so you should actually always use unicode for those.

It DOES have to be an ANSI-"encoded" file, or it will not work

When you write to the file, be sure to always run your output through output.encode('cp1252') (or whatever encoding ANSI would be on your system).

Things like line.write(str.decode('utf-8')) break the funktion of the file

By now you probably realized that:

  • If str as indeed an str instance, Python will try to convert it to unicode using the utf-8 encoding, but then try to encode it again (likely in ascii) to write it to the file
  • If str is actually an unicode instance, Python will first encode it (likely in ascii, and that will probably crash) to then be able to decode it.

Bottom line is, you need to know if str is unicode, you should encode it. If it's already encoded, don't touch it (or decode it then encode it if the encoding is not the one you want!).

A magical comment at the beginning of the script like # -- coding: iso-8859-1 -- does nothing here (though it is helpful when it comes to the mentioned Metadata and allowed characters in it...)

Not a surprise, this only tells Python what encoding should be used to read your source file so that non-ascii characters are properly recognized.

Oh, and i'm using Python 2.7.3. Third-Party modules dependencies, you know...

Python 3 probably is a big update in terms of unicode and encoding, but that doesn't mean Python 2.x can't make it work!

Will that solve your issue?

You can't be sure, it's possible that the problem lies in the player you're using, not in your code.

Once you output it, you should make sure that your script's output is readable using reference tools (such as Windows Explorer). If it is, but the player still can't open it, you should consider updating to a newer version.

Answer from Thomas Orozco on Stack Overflow
Top answer
1 of 3
30

As mentioned in the comments, your question isn't very specific, so I'll try to give you some hints about character encodings, see if you can apply those to your specific case!

Unicode and Encoding

Here's a small primer about encoding. Basically, there are two ways to represent text in Python:

  • unicode. You can consider that unicode is the ultimate encoding, you should strive to use it everywhere. In Python 2.x source files, unicode strings look like u'some unicode'.
  • str. This is encoded text - to be able to read it, you need to know the encoding (or guess it). In Python 2.x, those strings look like 'some str'.

This changed in Python 3 (unicode is now str and str is now bytes).

How does that play out?

Usually, it's pretty straightforward to ensure that you code uses unicode for its execution, and uses str for I/O:

  • Everything you receive is encoded, so you do input_string.decode('encoding') to convert it to unicode.
  • Everything you need to output is unicode but needs to be encoded, so you do output_string.encode('encoding').

The most common encodings are cp-1252 on Windows (on US or EU systems), and utf-8 on Linux.

Applying this to your case

I DO have to write äöü in a path, or it will not work

Windows natively uses unicode for file paths and names, so you should actually always use unicode for those.

It DOES have to be an ANSI-"encoded" file, or it will not work

When you write to the file, be sure to always run your output through output.encode('cp1252') (or whatever encoding ANSI would be on your system).

Things like line.write(str.decode('utf-8')) break the funktion of the file

By now you probably realized that:

  • If str as indeed an str instance, Python will try to convert it to unicode using the utf-8 encoding, but then try to encode it again (likely in ascii) to write it to the file
  • If str is actually an unicode instance, Python will first encode it (likely in ascii, and that will probably crash) to then be able to decode it.

Bottom line is, you need to know if str is unicode, you should encode it. If it's already encoded, don't touch it (or decode it then encode it if the encoding is not the one you want!).

A magical comment at the beginning of the script like # -- coding: iso-8859-1 -- does nothing here (though it is helpful when it comes to the mentioned Metadata and allowed characters in it...)

Not a surprise, this only tells Python what encoding should be used to read your source file so that non-ascii characters are properly recognized.

Oh, and i'm using Python 2.7.3. Third-Party modules dependencies, you know...

Python 3 probably is a big update in terms of unicode and encoding, but that doesn't mean Python 2.x can't make it work!

Will that solve your issue?

You can't be sure, it's possible that the problem lies in the player you're using, not in your code.

Once you output it, you should make sure that your script's output is readable using reference tools (such as Windows Explorer). If it is, but the player still can't open it, you should consider updating to a newer version.

2 of 3
6

On Windows there is special encoding available called mbcs, it converts between current default ANSI codepage and UNICODE. For example on a Spanish Language PC:

u'ñ'.encode('mbcs') -> '\xf1'
'\xf1'.decode('mbcs') -> u'ñ'

On Windows ANSI means current default multi-byte code page. For western European languages Windows ISO-8859-1, for eastern European languages windows ISO-8859-2) encoded byte string and other encodings for other languages as appropriate.

More info available at:

https://docs.python.org/2.4/lib/standard-encodings.html

See also:

https://docs.python.org/2/library/sys.html#sys.getfilesystemencoding

🌐
Python.org
discuss.python.org › python help
UTF-8 and ANSI encoding issue - Python Help - Discussions on Python.org
November 23, 2023 - Hi all, i have a code that writes to a file with utf-8 encoding. But when i try to open the same file i created in the same script i get an error message saying it can’t read it because the file is in ANSI. Here is the code of creating the file: with open(new_file, 'w', encoding='utf-8') as f: for item in items: f.write('%s\n' % item) with open(new_file, 'a', encoding='utf-8') as f: for line in lines: f.write('%s\n' % line) This is the code which supposed to open the ...
🌐
Python
docs.python.org › 3 › library › codecs.html
codecs — Codec registry and base classes
This module implements the ANSI codepage (CP_ACP). Availability: Windows. Changed in version 3.2: Before 3.2, the errors argument was ignored; 'replace' was always used to encode, and 'ignore' to decode.
🌐
Example Code
example-code.com › python › charset_convert_file_from_utf8_to_ansi.asp
CkPython Convert a File from utf-8 to ANSI (such as Windows-1252)
Chilkat • HOME • Android™ • AutoIt • C • C# • C++ • Chilkat2-Python • CkPython • Classic ASP • DataFlex • Delphi DLL • Go • Java • Node.js • Objective-C • PHP Extension • Perl • PowerBuilder • PowerShell • PureBasic • Ruby • SQL Server • Swift • Tcl • Unicode C • Unicode C++ • VB.NET • VBScript • Visual Basic 6.0 • Visual FoxPro • Xojo Plugin
🌐
PyPI
pypi.org › project › ansipants
ansipants · PyPI
A Python module and command-line utility for converting .ANS format ANSI art to HTML.
      » pip install ansipants
    
Published: Dec 25, 2021
Version: 0.2
🌐
DaniWeb
daniweb.com › programming › software-development › threads › 318059 › how-to-change-text-file-encoding-with-python
tkinter - How to change text file encoding with python? [SOLVED] | DaniWeb
The key is to open the output file in text mode and tell Python which encoding to use. If you open the file with the platform default (on Windows, often cp1252), your Unicode text will be re-encoded as ANSI.
Find elsewhere
🌐
Esri Community
community.esri.com › t5 › python-questions › python-script-file-not-encoded-correctly-with-ansi › td-p › 705853
Solved: Python script file not encoded correctly with ANSI... - Esri Community
December 12, 2021 - input = 'C:/temp/test.py' output = 'C:/temp/test_edited.py' input = open(input).read() with open(output, 'w') as outf: outf.write(input.encode('ascii', 'ignore').decode()) print(repr(input)) print(repr(open(output).read())) ‍‍‍‍‍‍‍‍‍
🌐
Notepad++ Community
community.notepad-plus-plus.org › topic › 24214 › python-multiple-files-ansi-to-utf-8-converter
Python: Multiple files ANSI to utf-8 converter | Notepad++ Community
March 7, 2023 - utf-16 utf-8 *.txt changing encoding of example2.txt from utf-16 to utf-8 changing encoding of example4.txt from utf-16 to utf-8 changing encoding of example3.txt from utf-16 to utf-8 changing encoding of example1.txt from utf-16 to utf-8 changing encoding of example5.txt from utf-16 to utf-8 >python -m encoding_conversion "example directory" utf-8 utf-16 *.md changing encoding of new 2.md from utf-8 to utf-16 changing encoding of new 3.md from utf-8 to utf-16 changing encoding of new 1.md from utf-8 to utf-16 ''' import os def change_encoding(fname, from_encoding, to_encoding='utf-8') -> None: ''' Read the file at path fname with its original encoding (from_encoding) and rewrites it with to_encoding.
🌐
Improve & Repeat
improveandrepeat.com › 2022 › 07 › python-friday-130-different-file-encodings-between-windows-and-linux
#130: Different File Encodings Between Windows and Linux - Python Friday
July 8, 2022 - But if we run the same code on Linux or in WSL, Python uses UTF-8 to encode the content of the file: That difference is the problem. ANSI does not support the right arrow and our Python application crashes on Windows when we try to write unsupported characters.
🌐
7-Zip Documentation
documentation.help › PyScripter › encodedsourcefiles.htm
Encoded Source Files - PyScripter - Documentation & Help
PEP 263 fully. The editor internally uses Unicode strings. When saved, Python files can be encoded in either utf-8 or ansi encoding.
🌐
GitHub
gist.github.com › 3e7e43ab85717e81925656f70f5bae8d
A guide to character encoding aware development · GitHub
December 15, 2021 - In general the name is just cp followed by the code page number (e.g. "cp1251"). The codecs module documentation has a complete list14. If running on a Windows system, Python aliases "mbcs" to the system ANSI code page for convenience.
🌐
Medium
medium.com › towardsdev › mastering-ansi-escape-codes-in-python-parsing-and-processing-text-4d7fc9645bf5
Mastering ANSI Escape Codes in Python: Parsing and Processing Text | by Py-Core Python Programming | Towards Dev
January 15, 2025 - Parsing and processing text files containing these ANSI escape characters using Python can help automate tasks, analyze terminal outputs, and capture data.
🌐
Programmersought
programmersought.com › article › 11051642134
UTF8 to ANSI encoding using Python - Programmer Sought
ANSI == Windows native encoding In Simplified Chinese Windows: ansi == gbk : >>> u'hello'.encode('mbcs') '\xc4\xe3\xba\xc3' >>> u'Hello'.encode('mbcs').decode('gbk') u'\u4f60\u597d' ... #!/usr/bin/env python # -*- coding: utf-8 -*- import os import codecs #oldfile:path to UTF8 file #newfile:path to the ANSI file to be saved def convertUTF8ToANSI(oldfile,newfile): Open UTF8 text file f = codecs.open(oldfile,'r','utf8') utfstr = f.read() f.close() #Transcode UTF8 strings into ANSI strings outansestr = utfstr.encode('mbcs') #Save the transcoded text in binary format f = open(newfile,'wb') f.write(outansestr) f.close()