You have a number of options. Since you are using Linux you could use UNIX domain sockets. Or, you could serialise the data as ASCII or JSon or some other format and feed it through a pipe, SHM (shared memory segment), message queue, DBUS or similar. It's worth thinking about what sort of data you have, as these IPC mechanisms have different performance characteristics. There's a draft USENIX paper with a good analysis of the various trade-offs that is worth reading.

Since you say (in the comments to this answer) that you prefer to use SHM, here are some code samples to start you off. Using the Python posix_ipc library:

import posix_ipc # POSIX-specific IPC
import mmap      # From Python stdlib

class SharedMemory(object):
    """Python interface to shared memory. 
    The create argument tells the object to create a new SHM object,
    rather than attaching to an existing one.
    """

    def __init__(self, name, size=posix_ipc.PAGE_SIZE, create=True):
        self.name = name
        self.size = size
        if create:
            memory = posix_ipc.SharedMemory(self.name, posix_ipc.O_CREX,
                                            size=self.size)
        else:
            memory = posix_ipc.SharedMemory(self.name)
        self.mapfile = mmap.mmap(memory.fd, memory.size)
        os.close(memory.fd)
        return

    def put(self, item):
        """Put item in shared memory.
        """
        # TODO: Deal with the case where len(item) > size(self.mapfile)
        # TODO: Guard this method with a named semaphore
        self.mapfile.seek(0)
        pickle.dump(item, self.mapfile, protocol=2)
        return

    def get(self):
        """Get a Python object from shared memory.
        """
        # TODO: Deal with the case where len(item) > size(self.mapfile)
        # TODO: Guard this method with a named semaphore
        self.mapfile.seek(0)
        return pickle.load(self.mapfile)

    def __del__(self):
        try:
            self.mapfile.close()
            memory = posix_ipc.SharedMemory(self.name)
            memory.unlink()
        except:
            pass
        return    

For the Java side you want to create the same class, despite what I said in the comments JTux seems to provide the equivalent functionality and the API you need is in UPosixIPC class.

The code below is an outline of the sort of thing you need to implement. However, there are several things missing -- exception handling is the obvious one, also some flags (find them in UConstant), and you'll want to add in a semaphore to guard the put / get methods. However, this should set you on the right track. Remember that an mmap or memory-mapped file is a file-like interface to a segment of RAM. So, you can use its file descriptor as if it were the fd of a normal file.

import jtux.*;

class SHM {

    private String name;
    private int size;
    private long semaphore;
    private long mapfile; // File descriptor for mmap file

    /* Lookup flags and perms in your system docs */
    public SHM(String name, int size, boolean create, int flags, int perms) {
        this.name = name;
        this.size = size;
        int shm;
        if (create) {
            flags = flags | UConstant.O_CREAT;
            shm = UPosixIPC.shm_open(name, flags, UConstant.O_RDWR);
        } else {
            shm = UPosixIPC.shm_open(name, flags, UConstant.O_RDWR);
        }
        this.mapfile = UPosixIPC.mmap(..., this.size, ..., flags, shm, 0);
        return;
    }


    public void put(String item) {
        UFile.lseek(this.mapfile(this.mapfile, 0, 0));
        UFile.write(item.getBytes(), this.mapfile);
        return;
    }


    public String get() {    
        UFile.lseek(this.mapfile(this.mapfile, 0, 0));
        byte[] buffer = new byte[this.size];
        UFile.read(this.mapfile, buffer, buffer.length);
        return new String(buffer);
    }


    public void finalize() {
        UPosix.shm_unlink(this.name);
        UPosix.munmap(this.mapfile, this.size);
    }

}

Well I'm not sure what exactly was the cause of the issue, it very well could be the use of a port number < 1024. Also, I eventually came across a great set of encapsulating readers/writers, the DataOutputStream and DataInputStream classes. These provided a much easier solution by providing specific read and write ops based on the object; such as dis.readInt() and dos.writeInt(int n).

Here is the completed and successfully running source code:

package socket_ipc;
import java.net.*;
import java.io.*;
/** Socket Inter-Process Communication.
 * Implements the Producer-Consumer Problem, using Sockets for IPC.
 * @author William Hatfield: CEG-4350-01 Fall 2015
 */
public class Socket_IPC {
    static final int NUMBER_OF_MESSAGES = 100; // number of messages to pass
    static int[] PRODUCED_MSSG = new int[NUMBER_OF_MESSAGES]; // for comparing
    static int[] CONSUMED_MSSG = new int[NUMBER_OF_MESSAGES]; // for comparing
    static final String HOST = "127.0.0.1"; // IP address of loopback device
    static final int PORT = 1234;           // arbitrary port number (local)
    static ServerSocket serverSocket;       // the shared server socket
    private static class Producer extends Thread {
        @Override
        public void run() {
            try {
                Socket toClient = serverSocket.accept();
                DataOutputStream dos = new DataOutputStream(toClient.getOutputStream());
                for (int i = 0; i < NUMBER_OF_MESSAGES; i++) {
                PRODUCED_MSSG[i] = 
                        (int)((Math.random() - .5) * Integer.MAX_VALUE);
                dos.writeInt(PRODUCED_MSSG[i]);
            }
            } catch (IOException ex) {
                System.err.println("Producer Error: " + ex.toString());
            }
        }
    }
    private static class Consumer extends Thread {
        @Override
        public void run() {
            try {
                Socket fromServer = new Socket(HOST, PORT);
                DataInputStream dis = new DataInputStream(fromServer.getInputStream());
                for (int i = 0; i < NUMBER_OF_MESSAGES; i++) {
                CONSUMED_MSSG[i] = dis.readInt();
            }
            } catch (IOException ex) {
                System.err.println("Consumer Error: " + ex.toString());
            }    
        }
    }
    public static void main(String[] args) {
        try {
            serverSocket = new ServerSocket(PORT);
            Producer p = new Producer();    // create the producer thread
            Consumer c = new Consumer();    // create the consumer thread
            p.start();                      // start the producer thread
            c.start();                      // start the consumer thread
            p.join();                       // wait for producer thread
            c.join();                       // wait for consumer thread
        } catch (InterruptedException | IOException ex) { /* handle later */ }
         /* compare produced and consumed data, exit if any match fails */
        for (int i = 0; i < NUMBER_OF_MESSAGES; i++) {
            System.out.print("[" + i + "]: " + PRODUCED_MSSG[i]);
            System.out.println(" == " + CONSUMED_MSSG[i]);
            if (PRODUCED_MSSG[i] != CONSUMED_MSSG[i]) {
                System.out.println("PROCESS SYNC ERROR AT INDEX: " + i);
                System.exit(0);
            }
        }
        /* inform the user that synchroniozation was succesful, then exit */
        System.out.println("SYNC SUCCESFUL!");
    }
}

If there are any improvements you would like to offer, I'm happy to listen. Especially to the die-hard Java coders, I'm sure there's something that goes against the standard Java practice in here!

Multiple options for IPC:

Socket-Based (Bare-Bones) Networking

• not necessarily hard, but:
  • might be verbose for not much,
  • might offer more surface for bugs, as you write more code.
• you could rely on existing frameworks, like Netty

RMI

• Technically, that's also network communication, but that's transparent for you.

Fully-fledged Message Passing Architectures

• usually built on either RMI or network communications as well, but with support for complicated conversations and workflows
• might be too heavy-weight for something simple
• frameworks like ActiveMQ or JBoss Messaging

Java Management Extensions (JMX)

• more meant for JVM management and monitoring, but could help to implement what you want if you mostly want to have one process query another for data, or send it some request for an action, if they aren't too complex
• also works over RMI (amongst other possible protocols)
• not so simple to wrap your head around at first, but actually rather simple to use

File-sharing / File-locking

• that's what you're doing right now
• it's doable, but comes with a lot of problems to handle

Signals

• You can simply send signals to your other project
• However, it's fairly limited and requires you to implement a translation layer (it is doable, though, but a rather crazy idea to toy with than anything serious.

Without more details, a bare-bone network-based IPC approach seems the best, as it's the:

• most extensible (in terms of adding new features and workflows to your
• most lightweight (in terms of memory footprint for your app)
• most simple (in terms of design)
• most educative (in terms of learning how to implement IPC). ^{(as you mentioned "socket is hard" in a comment, and it really is not and should be something you work on)}

That being said, based on your example (simply requesting the other process to do an action), JMX could also be good enough for you.

There is never a best way to solve a problem. There is only the way which works best for you.

When you have two separate processes running on different hosts, you need to communicate via network sockets. You could use an existing protocol standard like SOAP. When you are sure that none of the two applications will ever be rewritten in a different programming language, you could also use ObjectOutputStream and ObjectInputStream connected to network sockets and exchange Java objects directly between the two programs. Or you could design your own protocol from scratch which is optimized for your application.

When the processes run on the same host, communicating via network is often still a good option. But there are others like communicating via writing and reading temporary files. You could also go the UNIX way, and have one application write to System.out and pipe its output to the other program which reads it with System.in.

For this, you're best off a file based FIFO queue. Or using Java Native Access/Java Native Interface to write to a NamedPipe or Shared Memory. If you go the JNA/JNI route, you could create a Named Event.

But there's probably no way to do what you want, with any amount of efficiency without going the JNA/JNI route.