🌐
Tabnine
tabnine.com › home page › code › java › com.amazonaws.services.s3.model.s3objectinputstream
com.amazonaws.services.s3.model.S3ObjectInputStream java code examples | Tabnine
private ObjectMetadata downloadTo0(final OutputStream output, RequestMetricCollector requestMetricCollector) { GetObjectRequest req = new GetObjectRequest(getBucketName(), getKey()) .withRequestMetricCollector(requestMetricCollector); S3Object s3Object = getAmazonS3Client().getObject(req); S3ObjectInputStream objectContent = s3Object.getObjectContent(); try { byte[] buffer = new byte[1024 * 10]; int bytesRead = -1; while ((bytesRead = objectContent.read(buffer)) > -1) { output.write(buffer, 0, bytesRead); } } catch (IOException ioe) { objectContent.abort(); throw new SdkClientException("Unable to transfer content from Amazon S3 to the output stream", ioe); } finally { try { objectContent.close(); } catch (IOException ioe) {} } return s3Object.getObjectMetadata(); }
🌐
AWS
docs.aws.amazon.com › aws sdk for java › developer guide for version 1.x › aws sdk for java code examples › amazon s3 examples using the aws sdk for java › performing operations on amazon s3 objects
Performing Operations on Amazon S3 Objects - AWS SDK for Java 1.x
System.out.format("Downloading %s from S3 bucket %s...\n", key_name, bucket_name); final AmazonS3 s3 = AmazonS3ClientBuilder.standard().withRegion(Regions.DEFAULT_REGION).build(); try { S3Object o = s3.getObject(bucket_name, key_name); S3ObjectInputStream s3is = o.getObjectContent(); ...
Discussions

java - How do I convert String to S3ObjectInputStream - Stack Overflow
In the other API, when I have to convert the string back to S3ObjectInputStream, I try the following steps, but it doesn't seem to work: More on stackoverflow.com
🌐 stackoverflow.com
java - Maping S3ObjectInputStream into input stream - Stack Overflow
However, i would like to map S3ObjectInputStream into normal InputStream, is something like that possible? I would love to create API that hides this implementation. I know if i have small file i can just store bytes in array and create InputStream from that, but big files will be needed too. Thanks for help! ... Look at the documentation. It seems that this is a sub-class of InputStream, so all you have to do is casting. ... https://github.com/awsdocs/aws-doc-sdk-examples... More on stackoverflow.com
🌐 stackoverflow.com
amazon s3 - How to download s3 object directly into memory in java - Stack Overflow
Is it possible to download S3object in Java directly into memory and get it removed when i'm done with the task? More on stackoverflow.com
🌐 stackoverflow.com
java - How to write an S3 object to a file? - Stack Overflow
What's the fastest way to write an S3 object (of which I have the key) to a file? I'm using Java. More on stackoverflow.com
🌐 stackoverflow.com
🌐
Java Tips
javatips.net › api › com.amazonaws.services.s3.model.s3objectinputstream
Java Examples for com.amazonaws.services.s3.model.S3ObjectInputStream
Unable to calculate " + "checksum and verify data integrity.", e); } } else { // Ensures the data received from S3 has the same length as the // expected content-length is = new LengthCheckInputStream(is, // expected length s3Object.getObjectMetadata().getContentLength(), // bytes received from S3 are all included even if skipped INCLUDE_SKIPPED_BYTES); } s3Object.setObjectContent(new S3ObjectInputStream(is, httpRequest, false)); return s3Object; } catch (AmazonS3Exception ase) { if (ase.getStatusCode() == 412 || ase.getStatusCode() == 304) { publishProgress(listener, ProgressEventType.TRANSFER_CANCELED_EVENT); return null; } publishProgress(listener, ProgressEventType.TRANSFER_FAILED_EVENT); throw ase; } }
🌐
Stack Overflow
stackoverflow.com › questions › 73013130 › how-do-i-convert-string-to-s3objectinputstream
java - How do I convert String to S3ObjectInputStream - Stack Overflow
@Test public void testS3ObjectInputStream() throws IOException { String originalString = "Hello World"; InputStream inputStream = new ByteArrayInputStream(originalString.getBytes(StandardCharsets.UTF_8)); // your code String streamString = IOUtils.toString(inputStream, StandardCharsets.UTF_8); S3ObjectInputStream s3ObjectInputStream = new S3ObjectInputStream( new ByteArrayInputStream(streamString.getBytes(StandardCharsets.UTF_8)), null); assertNotNull(s3ObjectInputStream); //PASSES!
🌐
Java2s
java2s.com › example › java-api › com › amazonaws › services › s3 › model › s3objectinputstream › close-0-0.html
Example usage for com.amazonaws.services.s3.model S3ObjectInputStream close
public static void main(String[] args) { final String USAGE = "\n" + "To run this example, supply the name of an S3 bucket and object to\n" + "download from it.\n" + "\n" + "Ex: GetObject <bucketname> <filename>\n"; if (args.length < 2) { System.out.println(USAGE); System.exit(1);//from w w ...
🌐
GitHub
github.com › aws › aws-sdk-java › blob › master › aws-java-sdk-s3 › src › main › java › com › amazonaws › services › s3 › model › S3ObjectInputStream.java
aws-sdk-java/aws-java-sdk-s3/src/main/java/com/amazonaws/services/s3/model/S3ObjectInputStream.java at master · aws/aws-sdk-java
The official AWS SDK for Java 1.x (In Maintenance Mode, End-of-Life on 12/31/2025). The AWS SDK for Java 2.x is available here: https://github.com/aws/aws-sdk-java-v2/ - aws-sdk-java/aws-java-sdk-s3/src/main/java/com/amazonaws/services/s3/model/S3ObjectInputStream.java at master · aws/aws-sdk-java
Author   aws
Find elsewhere
🌐
Alexwlchan
alexwlchan.net › 2019 › streaming-large-s3-objects
Streaming large objects from S3 with ranged GET requests – alexwlchan
September 12, 2019 - import java.util import com.amazonaws.services.s3.model.S3ObjectInputStream val pieceSize: Long val enumeration = new util.Enumeration[S3ObjectInputStream] { var currentPosition = 0L val totalSize: Long override def hasMoreElements: Boolean = currentPosition < totalSize override def nextElement(): InputStream = { // The Range request is inclusive of the `start` and `end` parameters, // so to read `pieceSize` bytes we need to go to `pieceSize - 1`. val getRequest = new GetObjectRequest(bucketName, key) .withRange(currentPosition, currentPosition + pieceSize - 1) currentPosition += pieceSize s3Client.getObject(getRequest).getObjectContent } }
Top answer
1 of 6
41

Since Java 7 (published back in July 2011), there’s a better way: Files.copy() utility from java.util.nio.file.

Copies all bytes from an input stream to a file.

So you need neither an external library nor rolling your own byte array loops. Two examples below, both of which use the input stream from S3Object.getObjectContent().

InputStream in = s3Client.getObject("bucketName", "key").getObjectContent();

1) Write to a new file at specified path:

Files.copy(in, Paths.get("/my/path/file.jpg"));

2) Write to a temp file in system's default tmp location:

File tmp = File.createTempFile("s3test", "");
Files.copy(in, tmp.toPath(), StandardCopyOption.REPLACE_EXISTING);

(Without specifying the option to replace existing file, you'll get a FileAlreadyExistsException.)

Also note that getObjectContent() Javadocs urge you to close the input stream:

If you retrieve an S3Object, you should close this input stream as soon as possible, because the object contents aren't buffered in memory and stream directly from Amazon S3. Further, failure to close this stream can cause the request pool to become blocked.

So it should be safest to wrap everything in try-catch-finally, and do in.close(); in the finally block.

The above assumes that you use the official SDK from Amazon (aws-java-sdk-s3).

2 of 6
20

While IOUtils.copy() and IOUtils.copyLarge() are great, I would prefer the old school way of looping through the inputstream until the inputstream returns -1. Why? I used IOUtils.copy() before but there was a specific use case where if I started downloading a large file from S3 and then for some reason if that thread was interrupted, the download would not stop and it would go on and on until the whole file was downloaded.

Of course, this has nothing to do with S3, just the IOUtils library.

So, I prefer this:

InputStream in = s3Object.getObjectContent();
byte[] buf = new byte[1024];
OutputStream out = new FileOutputStream(file);
while( (count = in.read(buf)) != -1)
{
   if( Thread.interrupted() )
   {
       throw new InterruptedException();
   }
   out.write(buf, 0, count);
}
out.close();
in.close();

Note: This also means you don't need additional libraries

Top answer
1 of 8
73

Because the original question was never answered, and I had to run into this same problem, the solution for the MD5 problem is that S3 doesn't want the Hex encoded MD5 string we normally think about.

Instead, I had to do this.

// content is a passed in InputStream
byte[] resultByte = DigestUtils.md5(content);
String streamMD5 = new String(Base64.encodeBase64(resultByte));
metaData.setContentMD5(streamMD5);

Essentially what they want for the MD5 value is the Base64 encoded raw MD5 byte-array, not the Hex string. When I switched to this it started working great for me.

2 of 8
46

If all you are trying to do is solve the content length error from amazon then you could just read the bytes from the input stream to a Long and add that to the metadata.

/*
 * Obtain the Content length of the Input stream for S3 header
 */
try {
    InputStream is = event.getFile().getInputstream();
    contentBytes = IOUtils.toByteArray(is);
} catch (IOException e) {
    System.err.printf("Failed while reading bytes from %s", e.getMessage());
} 

Long contentLength = Long.valueOf(contentBytes.length);

ObjectMetadata metadata = new ObjectMetadata();
metadata.setContentLength(contentLength);

/*
 * Reobtain the tmp uploaded file as input stream
 */
InputStream inputStream = event.getFile().getInputstream();

/*
 * Put the object in S3
 */
try {

    s3client.putObject(new PutObjectRequest(bucketName, keyName, inputStream, metadata));

} catch (AmazonServiceException ase) {
    System.out.println("Error Message:    " + ase.getMessage());
    System.out.println("HTTP Status Code: " + ase.getStatusCode());
    System.out.println("AWS Error Code:   " + ase.getErrorCode());
    System.out.println("Error Type:       " + ase.getErrorType());
    System.out.println("Request ID:       " + ase.getRequestId());
} catch (AmazonClientException ace) {
    System.out.println("Error Message: " + ace.getMessage());
} finally {
    if (inputStream != null) {
        inputStream.close();
    }
}

You'll need to read the input stream twice using this exact method so if you are uploading a very large file you might need to look at reading it once into an array and then reading it from there.

🌐
Medium
medium.com › @contactnithin21 › working-with-amazon-s3-in-java-a-comprehensive-guide-3a39f48ddc4b
Working with Amazon S3 in Java: A Comprehensive Guide | by Nithin | Medium
February 28, 2025 - import com.amazonaws.services.s3.model.S3Object; import com.amazonaws.services.s3.model.S3ObjectInputStream; import java.io.File; import java.io.FileOutputStream; import java.io.IOException; import java.io.InputStream; public class S3Service { private AmazonS3 s3Client; public S3Service() { BasicAWSCredentials awsCreds = new BasicAWSCredentials("AWS_ACCESS_KEY", "AWS_SECRET_KEY"); this.s3Client = AmazonS3ClientBuilder.standard() .withRegion("us-east-1") // Replace with your region .withCredentials(new AWSStaticCredentialsProvider(awsCreds)) .build(); } public void downloadFile(String bucketNam
🌐
Javadoc.io
javadoc.io › doc › com.amazonaws › aws-java-sdk-s3 › 1.11.37 › com › amazonaws › services › s3 › model › S3ObjectInputStream.html
S3ObjectInputStream (AWS Java SDK for Amazon S3 1.11. ...
Bookmarks · Latest version of com.amazonaws:aws-java-sdk-s3 · https://javadoc.io/doc/com.amazonaws/aws-java-sdk-s3 · Current version 1.11.37 · https://javadoc.io/doc/com.amazonaws/aws-java-sdk-s3/1.11.37 · package-list path (used for javadoc generation -link option) · https://javadoc...
🌐
Coding Nagger
codingnagger.com › home › development › bits of code › s3object.getobjectcontent mocking using mockito
S3Object.getObjectContent mocking using Mockito - Coding Nagger
January 25, 2020 - I chose to manipulate bytes directly since most of the files to handle are binaries, also that’s how they’re all stored. S3Object.getObjectContent is the right place to start. The method returns an S3ObjectInputStream. I could cheat by creating an instance of an S3Object and call setObjectContent to set up my test but we don’t do that here.