by this module you can correct your text shape an direction. just install pips and use it.
# install: pip install --upgrade arabic-reshaper
import arabic_reshaper
# install: pip install python-bidi
from bidi.algorithm import get_display
text = "ذهب الطالب الى المدرسة"
reshaped_text = arabic_reshaper.reshape(text) # correct its shape
bidi_text = get_display(reshaped_text) # correct its direction
Answer from Jalal Razavi on Stack Overflowby this module you can correct your text shape an direction. just install pips and use it.
# install: pip install --upgrade arabic-reshaper
import arabic_reshaper
# install: pip install python-bidi
from bidi.algorithm import get_display
text = "ذهب الطالب الى المدرسة"
reshaped_text = arabic_reshaper.reshape(text) # correct its shape
bidi_text = get_display(reshaped_text) # correct its direction
The following code works:
import arabic_reshaper
text_to_be_reshaped = 'اللغة العربية رائعة'
reshaped_text = arabic_reshaper.reshape(text_to_be_reshaped)
rev_text = reshaped_text[::-1] # slice backwards
print(rev_text)
Your code is correct as it works on my computer with both Python 2 and 3 (I'm on OS X):
~$ python -c 'print "تست"'
تست
~$ python3 -c 'print("تست")'
تست
The problem is with your terminal that can not output unicode characters. You could verify it by redirecting your output to a file like python3 my_file.py > test.txt and open the file using an editor.
If you are on Windows you could use a terminal like Console2 or ConEmu that renders unicode better than Windows prompt.
You may encounter errors with these terminals too because of wrong code-pages/encodings of Windows. There is a small python package that fixes them (sets them correctly):
1- Install this pip install win-unicode-console
2- Put this at the top of your python file:
try:
# Fix UTF8 output issues on Windows console.
# Does nothing if package is not installed
from win_unicode_console import enable
enable()
except ImportError:
pass
If you got errors when redirecting to a file, you may fix it by settings io encoding:
On Windows command line:
SET PYTHONIOENCODING=utf-8
On Linux/OS X terminal:
export PYTHONIOENCODING=utf-8
Some points
- There is no need to use
u"aaa"syntax in python 3. Strings literals are unicode by default. - Default coding of files is UTF8 in python 3 so coding declaration comment (e.g.
# -*- coding: utf-8 -*-) is not needed.
The output will depend basically on which platform&terminal you run your code. Let's examine the below snippet for different windows terminals running either with 2.x or 3.x:
# -*- coding: utf-8 -*-
import sys
def case1(text):
print(text)
def case2(text):
print(text.encode("utf-8"))
def case3(text):
sys.stdout.buffer.write(text.encode("utf-8"))
if __name__ == "__main__":
text = "چرا کار نمیکنی؟"
for case in [case1, case2, case3]:
try:
print("Running {0}".format(case.__name__))
case(text)
except Exception as e:
print(e)
print('-'*80)
Results
Python 2.x
Sublime Text 3 3122
Running case1
'charmap' codec can't encode characters in position 0-2: character maps to <undefined>
--------------------------------------------------------------------------------
Running case2
b'\xda\x86\xd8\xb1\xd8\xa7 \xda\xa9\xd8\xa7\xd8\xb1 \xd9\x86\xd9\x85\xdb\x8c\xda\xa9\xd9\x86\xdb\x8c\xd8\x9f'
--------------------------------------------------------------------------------
Running case3
چرا کار نمیکنی؟--------------------------------------------------------------------------------
ConEmu v151205
Running case1
┌åÏ▒Ϻ ┌®ÏºÏ▒ ┘å┘à█î┌®┘å█îσ
--------------------------------------------------------------------------------
Running case2
'ascii' codec can't decode byte 0xda in position 0: ordinal not in range(128)
--------------------------------------------------------------------------------
Running case3
'file' object has no attribute 'buffer'
--------------------------------------------------------------------------------
Windows Command Prompt
Running case1
┌åÏ▒Ϻ ┌®ÏºÏ▒ ┘å┘à█î┌®┘å█îσ
--------------------------------------------------------------------------------
Running case2
'ascii' codec can't decode byte 0xda in position 0: ordinal not in range(128)
--------------------------------------------------------------------------------
Running case3
'file' object has no attribute 'buffer'
--------------------------------------------------------------------------------
Python 3.x
Sublime Text 3 3122
Running case1
'charmap' codec can't encode characters in position 0-2: character maps to <undefined>
--------------------------------------------------------------------------------
Running case2
b'\xda\x86\xd8\xb1\xd8\xa7 \xda\xa9\xd8\xa7\xd8\xb1 \xd9\x86\xd9\x85\xdb\x8c\xda\xa9\xd9\x86\xdb\x8c\xd8\x9f'
--------------------------------------------------------------------------------
Running case3
چرا کار نمیکنی؟--------------------------------------------------------------------------------
ConEmu v151205
Running case1
'charmap' codec can't encode characters in position 0-2: character maps to <undefined>
--------------------------------------------------------------------------------
Running case2
b'\xda\x86\xd8\xb1\xd8\xa7 \xda\xa9\xd8\xa7\xd8\xb1 \xd9\x86\xd9\x85\xdb\x8c\xda\xa9\xd9\x86\xdb\x8c\xd8\x9f'
--------------------------------------------------------------------------------
Running case3
┌åÏ▒Ϻ ┌®ÏºÏ▒ ┘å┘à█î┌®┘å█îσ--------------------------------------------------------------------------------
Windows Command Prompt
Running case1
'charmap' codec can't encode characters in position 0-2: character maps to <unde
fined>
--------------------------------------------------------------------------------
Running case2
b'\xda\x86\xd8\xb1\xd8\xa7 \xda\xa9\xd8\xa7\xd8\xb1 \xd9\x86\xd9\x85\xdb\x8c\xda
\xa9\xd9\x86\xdb\x8c\xd8\x9f'
--------------------------------------------------------------------------------
Running case3
┌åÏ▒Ϻ ┌®ÏºÏ▒ ┘å┘à█î┌®┘å█îσ----------------------------------------------------
----------------------------
As you can see just using sublime text3 terminal (case3) worked alright. The other terminals didn't support persian. The main point here is, it depends which terminal & platform you're using.
Solution (ConEmu specific)
Modern terminals like ConEmu allows you to work with UTF8-Encoding as explained here, so, let's try:
chcp 65001 & cmd
And then running again the script against 2.x & 3.x:
Python2.x
Running case1
��را کار نمیکنی؟[Errno 0] Error
--------------------------------------------------------------------------------
Running case2
'ascii' codec can't decode byte 0xda in position 0: ordinal not in range(128)
--------------------------------------------------------------------------------
Running case3
'file' object has no attribute 'buffer'
--------------------------------------------------------------------------------
Python3.x
Running case1
چرا کار نمیکنی؟
--------------------------------------------------------------------------------
Running case2
b'\xda\x86\xd8\xb1\xd8\xa7 \xda\xa9\xd8\xa7\xd8\xb1 \xd9\x86\xd9\x85\xdb\x8c\xda\xa9\xd9\x86\xdb\x8c\xd8\x9f'
--------------------------------------------------------------------------------
Running case3
چرا کار نمیکنی؟--------------------------------------------------------------------------------
As you can see, now the output was succesfull with python3 case1 (print). So... moral of a fable... learn more about your tools and how to configure them properly for your use-cases ;-)

