Best way to combine Python and Java?
Calling Python in Java? - Stack Overflow
Calling Java from Python - Stack Overflow
Using Python from within Java - Stack Overflow
Videos
My project uses some packages that are available only in Python and heavily rely on C libraries. The project also greatly benefits from Java libraries and the JVM. What's the optimal way to call Python functions from Java?
I tried:
-
Small web-services: overhead to serialize data, start and stop the services. Also debugging is harder and implementing each new function is now double the effort.
-
Jpy: a library that runs an interpreter in the JVM. Spare the service start/stop, but: isn't really feasible for more than a single-liner, data translation between Java and Python is cumbersome, and I also encountered runtime segmentation fault errors.
Any other options?
The project is in the machine-learning domain, so involves exchanging large numeric arrays and text. In some cases the execution switches back and forth between the platforms.
Jython: Python for the Java Platform - http://www.jython.org/index.html
You can easily call python functions from Java code with Jython. That is as long as your python code itself runs under jython, i.e. doesn't use some c-extensions that aren't supported.
If that works for you, it's certainly the simplest solution you can get. Otherwise you can use org.python.util.PythonInterpreter from the new Java6 interpreter support.
A simple example from the top of my head - but should work I hope: (no error checking done for brevity)
PythonInterpreter interpreter = new PythonInterpreter();
interpreter.exec("import sys\nsys.path.append('pathToModules if they are not there by default')\nimport yourModule");
// execute a function that takes a string and returns a string
PyObject someFunc = interpreter.get("funcName");
PyObject result = someFunc.__call__(new PyString("Test!"));
String realResult = (String) result.__tojava__(String.class);
As of 2021, Jython does not support Python 3.x
I think there are some important things to consider first with how strong you wish to have the linking between Java and Python.
Firstly Do you only want to call functions or do you actually want Python code to change the data in your java objects? This is very important. If you only want to call some Python code with or without arguments, then that is not very difficult. If your arguments are primitives it makes it even more easy. However, if you want to have Java class implement member functions in Python, which change the data of the Java object, then this is not so easy or straight forward.
Secondly are we talking CPython, or will Jython do? I would say CPython is where its at! I would advocate this is why Python is so kool! Having such high abstractions however access to C or C++ when needed. Imagine if you could have that in Java. This question is not even worth asking if Jython is ok because then it is easy anyway.
So I have played with the following methods, and listed them from easy to difficult:
Java to Jython
Advantages: Trivially easy. Have actual references to Java objects
Disadvantages: No CPython, Extremely Slow!
Jython from Java is so easy, and if this is really enough then great. However it is very slow and no CPython! Is life worth living without CPython? I don't think so! You can easily have Python code implementing your member functions for you Java objects.
Java to Jython to CPython via Pyro
Pyro is the remote object module for Python. You have some object on a CPython interpreter, and you can send it objects which are transferred via serialization and it can also return objects via this method. Note that if you send a serialized Python object from Jython and then call some functions which change the data in its members, then you will not see those changes in Java. You just need to remember to send back the data which you want from Pyro. This, I believe, is the easiest way to get to CPython! You do not need any JNI or JNA or SWIG or .... You don't need to know any C, or C++. Kool huh?
Advantages:
- Access to CPython
- Not as difficult as following methods
Disadvantages:
- Cannot change the member data of Java objects directly from Python
- Is somewhat indirect (Jython is middle man)
Java to C/C++ via JNI/JNA/SWIG to Python via Embedded interpreter (maybe using BOOST Libraries?)
OMG this method is not for the faint of heart. And I can tell you it has taken me very long to achieve this in with a decent method. Main reason you would want to do this is so that you can run CPython code which as full rein over you java object. There are major things to consider before deciding to try and breed Java (which is like a chimp) with Python (which is like a horse). Firstly if you crash the interpreter, that's lights out for you program! And don't get me started on concurrency issues! In addition, there is a lot of boiler, I believe I have found the best configuration to minimize this boiler but it is still a lot! So how to go about this:
Consider that C++ is your middle man, your objects are actually C++ objects! Good that you know that now. Just write your object as if your program is in C++ and not Java, with the data you want to access from both worlds. Then you can use the wrapper generator called SWIG to make this accessible to java and compile a dll which you call (System.load(dllNameHere)) in Java. Get this working first, then move on to the hard part!
To get to Python you need to embed an interpreter. Firstly I suggest doing some hello interpreter programs or this tutorial Embedding Python in C/C. Once you have that working, its time to make the horse and the monkey dance! You can send you C++ object to Python via [boost][3] . I know I have not given you the fish, merely told you where to find the fish. Some pointers to note for this when compiling.
When you compile boost you will need to compile a shared library. And you need to include and link to the stuff you need from jdk, ie jawt.lib, jvm.lib, (you will also need the client jvm.dll in your path when launching the application) As well as the python27.lib or whatever and the boost_python-vc100-mt-1_55.lib. Then include Python/include, jdk/include, boost and only use shared libraries (dlls) otherwise boost has a teary. And yeah full on I know. There are so many ways in which this can go sour. So make sure you get each thing done block by block. Then put them together.
You could also use Py4J. There is an example on the frontpage and lots of documentation, but essentially, you just call Java methods from your python code as if they were python methods:
from py4j.java_gateway import JavaGateway
gateway = JavaGateway() # connect to the JVM
java_object = gateway.jvm.mypackage.MyClass() # invoke constructor
other_object = java_object.doThat()
other_object.doThis(1,'abc')
gateway.jvm.java.lang.System.out.println('Hello World!') # call a static method
As opposed to Jython, one part of Py4J runs in the Python VM so it is always "up to date" with the latest version of Python and you can use libraries that do not run well on Jython (e.g., lxml). The other part runs in the Java VM you want to call.
The communication is done through sockets instead of JNI and Py4J has its own protocol (to optimize certain cases, to manage memory, etc.)
Disclaimer: I am the author of Py4J
Here is my summary of this problem: 5 Ways of Calling Java from Python
http://baojie.org/blog/2014/06/16/call-java-from-python/ (cached)
Short answer: Jpype works pretty well and is proven in many projects (such as python-boilerpipe), but Pyjnius is faster and simpler than JPype
I have tried Pyjnius/Jnius, JCC, javabridge, Jpype and Py4j.
Py4j is a bit hard to use, as you need to start a gateway, adding another layer of fragility.
I'm aware of the Jython project, but it looks like this represents a way to use Java and its libraries from within Python, rather than the other way round - am I wrong about this?
Yes, you are wrong. You can either call a command line interpreter to run python code using Jyton or use python code from Java. In the past there was also a python-to-Java compiler, but it got discontinued with Jython 2.2
I would write a Python module to handle the text and language processing, and then build a small bridge in jython that your java program can interact with. The jython bridge will be a very simple one, that's really only responsible for forwarding calls to the python module, and return the answer from the python module to the java module. Jython is really easy to use, and setup shouldn't take you more than 15 minutes.
Best of luck!