Starting with Java 7, you can download a file with built-in features as simple as

Files.copy(
    new URL("http://example.com/update/PubApp_2.0.jar").openStream(),
    Paths.get("C:/PubApp_2.0/update/lib/kitap.jar"));
// specify StandardCopyOption.REPLACE_EXISTING as 3rd argument to enable overwriting

for earlier versions, the solution from Java 1.4 to Java 6 is

try(
  ReadableByteChannel in=Channels.newChannel(
    new URL("http://example.com/update/PubApp_2.0.jar").openStream());
  FileChannel out=new FileOutputStream(
    "C:/PubApp_2.0/update/lib/kitap.jar").getChannel() ) {

  out.transferFrom(in, 0, Long.MAX_VALUE);
}

This code transfers a URL content to a file without any 3rd party library. If it’s still slow, you know that it is not the additional library’s and most probably not Java’s fault. At least there’s nothing you could improve here. So then you should search the reason outside the JVM.

  try {
        // Get the directory and iterate them to get file by file...
        File file = new File(fileName);

        if (!file.exists()) {
            context.addMessage(new ErrorMessage("msg.file.notdownloaded"));
            context.setForwardName("failure");
        } else {
            response.setContentType("APPLICATION/DOWNLOAD");
            response.setHeader("Content-Disposition", "attachment"+ 
                                     "filename=" + file.getName());
            stream = new FileInputStream(file);
            response.setContentLength(stream.available());
            OutputStream os = response.getOutputStream();      
            os.close();
            response.flushBuffer();
        }
    } catch (IOException e) {
        e.printStackTrace();
    } finally {
        if (stream != null) {
            try {
                stream.close();
            } catch (IOException e) {
                e.printStackTrace();
            }
        }
    }

Hope you got some idea...

How can I download a binary file like an mp3 from a server to a client?

To download a file with Java, you can Use URL.openStream();

http://download.oracle.com/javase/tutorial/networking/urls/readingURL.html

Don't print the output to System.out. Write it to a file, instead.

FileOutputStream fos = new FileOutputStreamm(new File("path_to_file.mp3");
int byte;

while ((byte= in.readLine()) != -1)
    fos.write(byte);

Where I have to put my server application? Don't implement a server unless you really have to. Use an http-Server if possile (Tomcat oder Apache HTTPD). Make your file available through HTTP.

If you want to use a Java Server, you should write a Servlet and packkage it into a WAR-File:

http://docstore.mik.ua/orelly/java-ent/servlet/

The loop that you asked for specifically and marked in your post with a comment, seems fine: it reads everything from the input stream and writes everything to the output stream. Nothing really suspicious there to me.

But you have a couple of much bigger problems in this code. A couple of pointers to get you started:

1. Review all your static variables. Take a hard look and check if they really need to be static (hint: most of them don't)

2. Try to make the content of all the try blocks smaller. Don't cover large blocks of code with a single catch. When many things can go wrong at many points, it's impossible to know the failure point, and it's impossible to recover gracefully. Try to handle each failure point individually.

3. Reorganize the code to not exit from anywhere other than the main method. Code with exit statements in constructors or non-toplevel methods is a very bad sign.

4. Avoid unnecessary operations. For example, the file exists method will iterate over all items, even after it already found a match. A better way is to return true immediately once you found a match. If the loop reaches the end, return false. No need for a boolean state variable.

5. The convention is to use camelCase for variable names, which the "Connected" variable violates.