you could build a simple server/client, the server running continuously making and updating the plot, and the client just communicating the next file to process.

I wrote a simple server/client example based on the basic example from the socket module docs: http://docs.python.org/2/library/socket.html#example

here is server.py:

# expensive imports
import numpy as np
import matplotlib.pyplot as plt
import matplotlib.animation as animation
import scipy.ndimage
import scipy.signal
import sys
import os

# Echo server program
import socket

HOST = ''                 # Symbolic name meaning all available interfaces
PORT = 50007              # Arbitrary non-privileged port
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind((HOST, PORT))
s.listen(1)
while 1:
    conn, addr = s.accept()
    print 'Connected by', addr
    data = conn.recv(1024)
    if not data: break
    conn.sendall("PLOTTING:" + data)
    # update plot
    conn.close()

and client.py:

# Echo client program
import socket
import sys

HOST = ''    # The remote host
PORT = 50007              # The same port as used by the server
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((HOST, PORT))
s.sendall(sys.argv[1])
data = s.recv(1024)
s.close()
print 'Received', repr(data)

you just run the server:

python server.py

which does the imports, then the client just sends via the socket the filename of the new file to plot:

python client.py mytextfile.txt

then the server updates the plot.

On my machine running your imports take 0.6 seconds, while running client.py 0.03 seconds.

The first things that come to mind are:

• Try a smaller path
• Make sure your modules are pyc's so they'll load faster
• Make sure you don't double import, or import too much

Other than that, are you sure that the disk operations are what's bogging you down? Is your disk/operating system really busy or old and slow?

Maybe a defrag is in order?

I took a look at what happens when you run import sympy, and it imports all of sympy.

https://github.com/sympy/sympy/blob/master/sympy/__init__.py

If you are only using certain parts of sympy, then only import those parts that you need.

It would be nice if you could do this:

import sympy.sets

But (as you point out) that imports sympy and then sets.

One solution is to write your own importer. You can do this with the help of the imp module.

import imp
sets = imp.load_module("sets", open("sympy/sets/__init__.py"), "sympy/sets/__init__.py", ('.py', 'U', 1))

But, even that may not optimize enough. Taking a look at sympy/sets/__init__.py I see that it does this:

from .sets import (Set, Interval, Union, EmptySet, FiniteSet, ProductSet,
    Intersection, imageset, Complement, SymmetricDifference)
from .fancysets import TransformationSet, ImageSet, Range, ComplexRegion
from .contains import Contains
from .conditionset import ConditionSet

Maybe you can import only the sets module from simpy sets namespace?

import imp
sets = imp.load_module("sets", open("sympy/sets/set.py") "sympy/sets/set.py", ('.py', 'U', 1))