You extend the Authenticator class and register it. The javadocs at the link explain how.

I don't know if this works with the nio method that got the accepted answer to the question, but it for sure works for the old fashioned way that was the answer under that one.

Within the authenticator class implementation, you are probably going to use a PasswordAuthentication and override the getPasswordAuthentication() method of your Authenticator implementation to return it. That will be the class which is passed the user name and password you need.

Per your request, here is some sample code:

public static final String USERNAME_KEY = "username";
public static final String PASSWORD_KEY = "password";
private final PasswordAuthentication authentication;

public MyAuthenticator(Properties properties) {
    String userName = properties.getProperty(USERNAME_KEY);
    String password = properties.getProperty(PASSWORD_KEY);
    if (userName == null || password == null) {
        authentication = null;
    } else {
        authentication = new PasswordAuthentication(userName, password.toCharArray());
    }
}

protected PasswordAuthentication getPasswordAuthentication() {
    return authentication;
}

And you register it in the main method (or somewhere along the line before you call the URL):

Authenticator.setDefault(new MyAuthenticator(properties));

The usage is simple, but I find the API convoluted and kind of backwards for how you typically think about these things. Pretty typical of singleton design.

Answer from Yishai on Stack Overflow
🌐
Stack Overflow
stackoverflow.com › questions › 23765282 › download-file-from-url-with-authentication
java - download file from url with authentication - Stack Overflow
url is third party, i dont want to ask them to change their code, currently if i can download pdf from browser (a pop up comes and i give credentials, then download), i believe its possible with java code as well ... Don't ask them anything, just include the username:password in the URL string. Have a read through this en.wikipedia.org/wiki/URL#Syntax ... I solved this issue. I used a customized authenticator before connecting the URL, and it authenticates and downloads the document.
Top answer
1 of 6
17

You extend the Authenticator class and register it. The javadocs at the link explain how.

I don't know if this works with the nio method that got the accepted answer to the question, but it for sure works for the old fashioned way that was the answer under that one.

Within the authenticator class implementation, you are probably going to use a PasswordAuthentication and override the getPasswordAuthentication() method of your Authenticator implementation to return it. That will be the class which is passed the user name and password you need.

Per your request, here is some sample code:

public static final String USERNAME_KEY = "username";
public static final String PASSWORD_KEY = "password";
private final PasswordAuthentication authentication;

public MyAuthenticator(Properties properties) {
    String userName = properties.getProperty(USERNAME_KEY);
    String password = properties.getProperty(PASSWORD_KEY);
    if (userName == null || password == null) {
        authentication = null;
    } else {
        authentication = new PasswordAuthentication(userName, password.toCharArray());
    }
}

protected PasswordAuthentication getPasswordAuthentication() {
    return authentication;
}

And you register it in the main method (or somewhere along the line before you call the URL):

Authenticator.setDefault(new MyAuthenticator(properties));

The usage is simple, but I find the API convoluted and kind of backwards for how you typically think about these things. Pretty typical of singleton design.

2 of 6
9

This is some code I wrote that fetches a website and displays the contents to System.out. It uses Basic authentication:

import java.net.*;
import java.io.*;

public class foo {
    public static void main(String[] args) throws Exception {

   URL yahoo = new URL("http://www.MY_URL.com");

   String passwdstring = "USERNAME:PASSWORD";
   String encoding = new 
          sun.misc.BASE64Encoder().encode(passwdstring.getBytes());

   URLConnection uc = yahoo.openConnection();
   uc.setRequestProperty("Authorization", "Basic " + encoding);

   InputStream content = (InputStream)uc.getInputStream();
   BufferedReader in   =   
            new BufferedReader (new InputStreamReader (content));

   String line;
   while ((line = in.readLine()) != null) {
      System.out.println (line);
   }   

   in.close();
}

Problems with the above code:

  1. This code isn't production-ready (but it gets the point across.)

  2. The code yields this compiler warning:

foo.java:11: warning: sun.misc.BASE64Encoder is Sun proprietary API and may be removed in a future release
      sun.misc.BASE64Encoder().encode(passwdstring.getBytes());
              ^ 1 warning

One really should use the Authenticator class, but for the life of me, I could not figure out how and I couldn't find any examples either, which just goes to show that the Java people don't actually like it when you use their language to do cool things. :-P

So the above isn't a good solution, but it does work and could easily be modified later.

Discussions

Java: Download file from URL when URL requests authentication - Stack Overflow
The server runs apache httpd and ... login/authentication first. Then when I put this URL in a browser I get the download prompt to download this zip file. How can I do this in Java? I am learning Java and I am from a Python background. Any help is greatly appreciated. Edit: Server runs on HTTPS auth. ... It uses HTTPS instead of HTTP basic. ... HTTPS is orthogonal to the question I asked. Both of those options can be used over HTTP or HTTPS. (although they should not be used with ... More on stackoverflow.com
🌐 stackoverflow.com
JAVA: Trying to download file from URL with authentication - Stack Overflow
I have a problem with downloading a file from a URL with authentication. I am pretty new to this topic and I need help from you guys! :( I want to download a docx document from a url. If you would ... More on stackoverflow.com
🌐 stackoverflow.com
java - File download involving HTTP Basic Authentication - Stack Overflow
Connect and share knowledge within a single location that is structured and easy to search. Learn more about Teams · Get early access and see previews of new features. Learn more about Labs ... In my Java program i want to download a file from a server that uses Http Basic Authentication. Can i use the URL ... More on stackoverflow.com
🌐 stackoverflow.com
authentication - Download large files with java from authenticated server - Stack Overflow
I need to implement the functionality to download multiple files from an authenticated server. But I'm receiving the following result, even though accessing via browser makes the download starts no... More on stackoverflow.com
🌐 stackoverflow.com
🌐
Mysamplecode
mysamplecode.com › 2019 › 04 › java-download-image-from-url-authentication.html
Programmers Sample Guide: java download image from url with basic authentication
private boolean downloadImage(String url, String username, String password, String fileName) { boolean success = true; InputStream in = null; FileOutputStream out = null; try{ URL myUrl = new URL(url); HttpURLConnection conn = (HttpURLConnection) myUrl.openConnection(); conn.setDoOutput(true); conn.setReadTimeout(30000); conn.setConnectTimeout(30000); conn.setUseCaches(false); conn.setAllowUserInteraction(false); conn.setRequestProperty("Content-Type", "application/json"); conn.setRequestProperty("Accept-Charset", "UTF-8"); conn.setRequestMethod("GET"); String userCredentials = username.trim()
🌐
Coderanch
coderanch.com › t › 430792 › java › Pulling-file-HTTP-URL-Authentication
Pulling file via HTTP URL - Uses Authentication (Java in General forum at Coderanch)
February 11, 2009 - I can download the file easily using HTTPconnection or using URL Connection class in JAVA. My problem is that when somebody access this URL they require authentication. Is there anyway in HTTPConnection or URLConnection class to pass authentication parameters ?
🌐
Stack Overflow
stackoverflow.com › questions › 69883585 › java-trying-to-download-file-from-url-with-authentication
JAVA: Trying to download file from URL with authentication - Stack Overflow
IF you could open the Developer Toolbar in your Browser -- Go to Network --> Clear everything --> Paste your URL and click Enter --> Copy the Request from your Network Tab. I dont know how the request looks like, so I cant help with it.
🌐
Hedleyproctor
hedleyproctor.com › 2012 › 01 › using-java-to-download-a-file-that-needs-authentication
Using Java to download a file that needs authentication | Hedley Proctor
def getFileViaSelenium() { println("Logging in via Selenium") val driver = new FirefoxDriver() driver.get("https://www.someurl.com/login") driver.findElement(By.id("username")).clear(); driver.findElement(By.id("username")).sendKeys("John Smith"); driver.findElement(By.id("password")).clear(); driver.findElement(By.id("password")).sendKeys("password"); driver.findElement(By.name("commit")).click(); // now get the cookies val seleniumCookies = driver.manage().getCookies().asScala val cookieString = new StringBuilder() for (cookie <- seleniumCookies) { println("Cookie value: " + cookie.getValue(
🌐
Google Groups
groups.google.com › a › runmyprocess.com › g › supportforum › c › ZHx4Otag7r8
Downloading a file using http basic authentication
I need to download a file (a pdf ... email addresses permission to view the original message ... Yes, you can. Just request a GET request to the file URL and you should be able to download....
Find elsewhere
🌐
My Memory
putridparrot.com › blog › downloading-a-file-from-url-using-basic-authentication
Downloading a file from URL using basic authentication | My Memory
This code assumes that the url is supplied to this code along with a filename for where to save the downloaded file. We use a proxy, hence the proxy is supplied, and then we supply the NetworkCredential which will handle basic authentication.
🌐
Stack Overflow
stackoverflow.com › questions › 50417288 › download-large-files-with-java-from-authenticated-server
authentication - Download large files with java from authenticated server - Stack Overflow
But I'm receiving the following result, even though accessing via browser makes the download starts normally (if authenticated). ... import org.apache.commons.io.FileUtils; import java.io.File; import java.io.IOException; import java.net.Authenticator; import java.net.PasswordAuthentication; import java.net.URL; public class Example { private static final String URL = "totally_a_valid_url"; private static final String PATH = "D:\\___Automation\\_TEST"; public static void main(String[] args) throws IOException { Authenticator.setDefault(new MyAuthenticator()); URL url = new URL(URL); File desti
🌐
Stack Overflow
stackoverflow.com › questions › 16540506 › java-android-download-file-http-authentication
java android - download file http authentication - Stack Overflow
I want to download the content of html and json files from a webserver with http authentication on android. On the browsers I always used http://username:[email protected]/path/to/something, which worked fine. But in Java on Android it doesn't work (it worked fine before adding HTTP Authoriziation). I can show a html file in the webview using this. But how to download the content? I always get FileNotFoundException. Code to download html file: String url = "http://username:[email protected]/path/to/something"; //or: "http://example.com/path/to/something" URL oracle = new URL(url); URLConnectio
Top answer
1 of 2
4

This is what I came up with helped by S201's answer plus a lot of googling. The code is simplified and without try-catch constructions.

HttpClient client = new DefaultHttpClient();
HttpPost post = new HttpPost("https://example.com/login");
HttpResponse response = null;
List<NameValuePair> postFields = new ArrayList<NameValuePair>(2);

// Set the post fields
postFields.add(new BasicNameValuePair("username", "myusername"));
postFields.add(new BasicNameValuePair("password", "mypassword"));
post.setEntity(new UrlEncodedFormEntity(postFields, HTTP.UTF_8));

// Execute the POST request
response = client.execute(post);

// Now GET the file
HttpGet get = new HttpGet("http://example.com/files/myfile.mp3");
response = client.execute(get);

HttpEntity entity = response.getEntity();
InputStream in = entity.getContent();

// Save the file to SD
File path = Environment.getExternalStoragePublicDirectory(Environment.DIRECTORY_DOWNLOADS);
path.mkdirs();
File file = new File(path, "myfile.mp3");
FileOutputStream fos = new FileOutputStream(file);

byte[] buffer = new byte[1024];
int len1 = 0;
while ((len1 = in.read(buffer)) > 0) {
        fos.write(buffer, 0, len1);
}

fos.close();
2 of 2
2

You going to want to use an HttpClient object in combination with HttpPost, HttpGet, and HttpResponse objects. It is probably easier to just look at an example.

HttpClient client = new DefaultHttpClient();
HttpPost post = new HttpPost(context.getString(R.string.loginURL));
HttpResponse response = null;
List<NameValuePair> postFields = new ArrayList<NameValuePair>(2);  

// Set the post fields
postFields.add(new BasicNameValuePair("username", settings.getString("username", "null")));
postFields.add(new BasicNameValuePair("password", settings.getString("password", "null")));
post.setEntity(new UrlEncodedFormEntity(postFields, HTTP.UTF_8));

// Execute the POST request
response = client.execute(post);

Assuming the login was successful, you can now execute GET and POST requests as an authenticated user as long as you execute them though the HttpClient that you executed the login through. It is this object that manages the cookies. Hope this helps!

EDIT: Forgot to mention that you can of course use the HttpRespose object to perform error checking.

🌐
DEV Community
dev.to › thokuest › downloading-files-with-groovy-and-authentication-6bb
Downloading Files With Groovy and Authentication - DEV Community
August 30, 2019 - Downloading a file from a URL requiring user authentication with Groovy is as easy as: java.net.Authenticator.setDefault (new Authenticator() { protected PasswordAuthentication getPasswordAuthentication() { return new PasswordAuthentication...
🌐
CodeJava
codejava.net › java-se › networking › use-httpurlconnection-to-download-file-from-an-http-url
Java HttpURLConnection to download file from an HTTP URL
July 18, 2019 - [Master REST API Development and ... from web server programmatically.You know, in Java, we can use the classes URLand HttpURLConnection in the package java.net to programmatically download a file from a given URL by following these steps:...
Top answer
1 of 3
3

You may be having certificate issues. This is typically the problem I have encountered in the past when working with HTTPS connections in Java.

First, you should check and see if the URL to which you are attempting to connect has a signed certificate by a well-known trusted root CA, and is valid (not expired).

I would recommend opening the URL in your browser and checking the certificate information.

Just a FYI, there may be a disconnect between the Trusted Root CAs recognized by your browser and those recognized by Java. Here is another Stackoverflow question about how to get those recognized by Java: How can I get a list of trusted root certificates in Java?

If this is a self-signed certificate, then there are hoops you will need to jump through regarding importing it into and using a local Keystore. There are numerous sites and blogs that guide you through doing this, here is one such blog (not mine): Adding self-signed https certificates to java keystore

Also, while you are testing with the browser, this will help you verify that there are no proxy issues. You should definitely check your browser's settings to determine whether or not you are going through a proxy server.

You should definitely look into using HttpClient instead of java.net.URL. Here is the Apache page for HttpClient 4.2.1.

Finally, if you are looking to do a file transfer via HTTP or HTTPS, you may want to consider WebDAV.

I have used Jakarta Slide WebDAV Client for this in the past. Unfortunately, it looks like Slide is retired at this point, but there are alternatives you can find with a little bit of searching.

ADDITION

I copied down your source code sample and looked at it more closely. Looks like you set properties for http, but not https.

HTTPS has separate properties:

  • https.proxyHost
  • https.proxyPort

Try setting:

System.setProperty("https.proxyHost","trproxy.rwe.com") ; 
System.setProperty("https.proxyPort", "443") ; 

Look at section 2.2 on Oracle's Java Networking and Proxies.

http://docs.oracle.com/javase/6/docs/technotes/guides/net/proxies.html

2 of 3
1

Looks like your problem could be with the proxy. It should also support https. Also, you should use HttpClient 4 for downloading the files, its a mature library for making http connections.

🌐
Baeldung
baeldung.com › home › java › java io › download a file from an url in java
Download a File From an URL in Java | Baeldung
January 8, 2024 - The most basic API we can use to download a file is Java IO. We can use the URL class to open a connection to the file we want to download.
🌐
Google
docs.google.com › viewer
Google
Sign in · Use your Google Account · Email or phone · Forgot email · Type the text you hear or see · Create account