Access an HTTPS url with Java is the same then access an HTTP url. You can always use the
URL url = new URL("https://hostname:port/file.txt");
URLConnection connection = url.openConnection();
InputStream is = connection.getInputStream();
// .. then download the file
But, you can have some problem when the server's certificate chain cannot be validated. So you may need to disable the validation of certificates for testing purposes and trust all certificates.
To do that write:
// Create a new trust manager that trust all certificates
TrustManager[] trustAllCerts = new TrustManager[]{
new X509TrustManager() {
public java.security.cert.X509Certificate[] getAcceptedIssuers() {
return null;
}
public void checkClientTrusted(
java.security.cert.X509Certificate[] certs, String authType) {
}
public void checkServerTrusted(
java.security.cert.X509Certificate[] certs, String authType) {
}
}
};
// Activate the new trust manager
try {
SSLContext sc = SSLContext.getInstance("SSL");
sc.init(null, trustAllCerts, new java.security.SecureRandom());
HttpsURLConnection.setDefaultSSLSocketFactory(sc.getSocketFactory());
} catch (Exception e) {
}
// And as before now you can use URL and URLConnection
URL url = new URL("https://hostname:port/file.txt");
URLConnection connection = url.openConnection();
InputStream is = connection.getInputStream();
// .. then download the file
Access an HTTPS url with Java is the same then access an HTTP url. You can always use the
URL url = new URL("https://hostname:port/file.txt");
URLConnection connection = url.openConnection();
InputStream is = connection.getInputStream();
// .. then download the file
But, you can have some problem when the server's certificate chain cannot be validated. So you may need to disable the validation of certificates for testing purposes and trust all certificates.
To do that write:
// Create a new trust manager that trust all certificates
TrustManager[] trustAllCerts = new TrustManager[]{
new X509TrustManager() {
public java.security.cert.X509Certificate[] getAcceptedIssuers() {
return null;
}
public void checkClientTrusted(
java.security.cert.X509Certificate[] certs, String authType) {
}
public void checkServerTrusted(
java.security.cert.X509Certificate[] certs, String authType) {
}
}
};
// Activate the new trust manager
try {
SSLContext sc = SSLContext.getInstance("SSL");
sc.init(null, trustAllCerts, new java.security.SecureRandom());
HttpsURLConnection.setDefaultSSLSocketFactory(sc.getSocketFactory());
} catch (Exception e) {
}
// And as before now you can use URL and URLConnection
URL url = new URL("https://hostname:port/file.txt");
URLConnection connection = url.openConnection();
InputStream is = connection.getInputStream();
// .. then download the file
Actually I had the similar problem. I was unable to download files from HTTPS server. Then I fixed this problem with this solution:
// But are u denied access?
// well here is the solution.
public static void TheKing_DownloadFileFromURL(String search, String path) throws IOException {
// This will get input data from the server
InputStream inputStream = null;
// This will read the data from the server;
OutputStream outputStream = null;
try {
// This will open a socket from client to server
URL url = new URL(search);
// This user agent is for if the server wants real humans to visit
String USER_AGENT = "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36";
// This socket type will allow to set user_agent
URLConnection con = url.openConnection();
// Setting the user agent
con.setRequestProperty("User-Agent", USER_AGENT);
//Getting content Length
int contentLength = con.getContentLength();
System.out.println("File contentLength = " + contentLength + " bytes");
// Requesting input data from server
inputStream = con.getInputStream();
// Open local file writer
outputStream = new FileOutputStream(path);
// Limiting byte written to file per loop
byte[] buffer = new byte[2048];
// Increments file size
int length;
int downloaded = 0;
// Looping until server finishes
while ((length = inputStream.read(buffer)) != -1)
{
// Writing data
outputStream.write(buffer, 0, length);
downloaded+=length;
//System.out.println("Downlad Status: " + (downloaded * 100) / (contentLength * 1.0) + "%");
}
} catch (Exception ex) {
//Logger.getLogger(WebCrawler.class.getName()).log(Level.SEVERE, null, ex);
}
// closing used resources
// The computer will not be able to use the image
// This is a must
outputStream.close();
inputStream.close();
}
Use this function... Hope you will get benefited with this easy solution.
Videos
You may be having certificate issues. This is typically the problem I have encountered in the past when working with HTTPS connections in Java.
First, you should check and see if the URL to which you are attempting to connect has a signed certificate by a well-known trusted root CA, and is valid (not expired).
I would recommend opening the URL in your browser and checking the certificate information.
Just a FYI, there may be a disconnect between the Trusted Root CAs recognized by your browser and those recognized by Java. Here is another Stackoverflow question about how to get those recognized by Java: How can I get a list of trusted root certificates in Java?
If this is a self-signed certificate, then there are hoops you will need to jump through regarding importing it into and using a local Keystore. There are numerous sites and blogs that guide you through doing this, here is one such blog (not mine): Adding self-signed https certificates to java keystore
Also, while you are testing with the browser, this will help you verify that there are no proxy issues. You should definitely check your browser's settings to determine whether or not you are going through a proxy server.
You should definitely look into using HttpClient instead of java.net.URL. Here is the Apache page for HttpClient 4.2.1.
Finally, if you are looking to do a file transfer via HTTP or HTTPS, you may want to consider WebDAV.
I have used Jakarta Slide WebDAV Client for this in the past. Unfortunately, it looks like Slide is retired at this point, but there are alternatives you can find with a little bit of searching.
ADDITION
I copied down your source code sample and looked at it more closely. Looks like you set properties for http, but not https.
HTTPS has separate properties:
- https.proxyHost
- https.proxyPort
Try setting:
System.setProperty("https.proxyHost","trproxy.rwe.com") ;
System.setProperty("https.proxyPort", "443") ;
Look at section 2.2 on Oracle's Java Networking and Proxies.
http://docs.oracle.com/javase/6/docs/technotes/guides/net/proxies.html
Looks like your problem could be with the proxy. It should also support https. Also, you should use HttpClient 4 for downloading the files, its a mature library for making http connections.
Give Java NIO a try:
URL website = new URL("http://www.website.com/information.asp");
ReadableByteChannel rbc = Channels.newChannel(website.openStream());
FileOutputStream fos = new FileOutputStream("information.html");
fos.getChannel().transferFrom(rbc, 0, Long.MAX_VALUE);
Using transferFrom() is potentially much more efficient than a simple loop that reads from the source channel and writes to this channel. Many operating systems can transfer bytes directly from the source channel into the filesystem cache without actually copying them.
Check more about it here.
Note: The third parameter in transferFrom is the maximum number of bytes to transfer. Integer.MAX_VALUE will transfer at most 2^31 bytes, Long.MAX_VALUE will allow at most 2^63 bytes (larger than any file in existence).
Use Apache Commons IO. It is just one line of code:
FileUtils.copyURLToFile(URL, File)