There's the javax.json.JSONParser.

An example that prints out a JSON:

import javax.json.Json;
import javax.json.stream.JsonParser;
...
JsonParser parser = Json.createParser(new StringReader(jsonData));
while (parser.hasNext()) {
   JsonParser.Event event = parser.next();
   switch(event) {
      case START_ARRAY:
      case END_ARRAY:
      case START_OBJECT:
      case END_OBJECT:
      case VALUE_FALSE:
      case VALUE_NULL:
      case VALUE_TRUE:
         System.out.println(event.toString());
         break;
      case KEY_NAME:
         System.out.print(event.toString() + " " +
                          parser.getString() + " - ");
         break;
      case VALUE_STRING:
      case VALUE_NUMBER:
         System.out.println(event.toString() + " " +
                            parser.getString());
         break;
   }
}

from The Java EE Tutorial 19.4.1 Reading JSON Data Using a Parser. It's part of the JavaEE 7 API but a standalone jar can be gotten from https://jsonp.java.net/download.html.

Since you're already using Google's Json-Simple library, you can parse the json from an InputStream like this:

InputStream inputStream = ... //Read from a file, or a HttpRequest, or whatever.
JSONParser jsonParser = new JSONParser();
JSONObject jsonObject = (JSONObject)jsonParser.parse(
      new InputStreamReader(inputStream, "UTF-8"));

I would suggest you have to use a Reader to convert your InputStream in.

BufferedReader streamReader = new BufferedReader(new InputStreamReader(in, "UTF-8")); 
StringBuilder responseStrBuilder = new StringBuilder();

String inputStr;
while ((inputStr = streamReader.readLine()) != null)
    responseStrBuilder.append(inputStr);
new JSONObject(responseStrBuilder.toString());

I tried in.toString() but it returns:

getClass().getName() + '@' + Integer.toHexString(hashCode())

(like documentation says it derives to toString from Object)

You can convert a JSONObject into its String representation, and then convert that String into an InputStream.

The code in the question has a JSONObject being cast into File, but I am not sure if that works as intended. The following, however, is something I have done before (currently reproduced from memory):

String str = json.getJSONObject("data").toString();
InputStream is = new ByteArrayInputStream(str.getBytes());

Note that the toString() method for JSONObject overrides the one in java.lang.Object class.

From the Javadoc:

Returns: a printable, displayable, portable, transmittable representation of the object, beginning with { (left brace) and ending with } (right brace).

Yes, you can use the methods that don't load the whole JSON into the memory using JsonParser:

• use JsonParser.readBinaryValue(OutputStream o)
• use JsonParser.getText(Writer w)
  • inherit the relevant factory and JsonParser implementation classes and tweak them to your needs
• inherit the writer/stream class and write your own implementation

Unfortunately, using another library GSON, there's no way to use a writer stream because its interface only provides a nextString method.

Using readBinaryValue

you can use another method that reads base64, decodes it to binary and streams it to your own OutputStream.

parser.readBinaryValue(Base64Variants.MIME, jsonOutputStream);

and then inherit and tweak the implementation of your OutputStream to use it the way you want

public class JsonOutputStream extends OutputStream {
    private final ByteArrayOutputStream buffer = new ByteArrayOutputStream(4);
    private final Base64.Encoder encoder = Base64.getMimeEncoder();

    @Getter
    private int count = 0;

    @Override
    public void write(int b) throws IOException {
        buffer.write(b);
        if (buffer.size() % 4 == 0) { // Process in chunks of 4 Base64 characters
            flushBuffer();
        }
    }

    public void resetCount() {
        count = 0;
    }

    private void flushBuffer() {
        if (buffer.size() < 4) return; // Only decode full Base64 chunks
        byte[] encoded = encoder.encode(buffer.toByteArray());
        buffer.reset();
    }
}

JsonParser uses its own base64 variant, if you want the standard one you have to choose MIME. Another option is also URL.

And the good thing about this option is that you don't have to tweak anything in this case as it doesn't create segments like the getText option. It sets the buffer length configured more or less like private final static int[] BYTE_BUFFER_LENGTHS = new int[] { 8000, 8000, 2000, 2000 }; (because there's a padding in b64, and chunks are always 4 bytes long, there is some tweaking with - 3). Then when it needs more room, it simply flushes the buffer.

Therefore, it doesn't load the whole string into the memory and you don't have to do any custom tweaking.

Using JsonParser.getText(Writer w)

The basic JsonParser.getText(writer) does a .getText which loads everything into memory

Yes, you're right, but you can tweak it. Every implementation like ReaderBasedJsonParser or UTF8DataInputJsonParser calls a helper method inside the getText from the snippet

if (t == JsonToken.VALUE_STRING) {
    if (_tokenIncomplete) {
    _tokenIncomplete = false;
    _finishString(); // only strings can be incomplete
  }

and I debugged it, for the string tokens _tokenIncomplete is always true as intended. Then inside the method _finishedString2 called by the _finishString there is a nice loop that copies the JSON's string input into the buffer that is segmented. Each segment has the size that can be configured from the factory in the context passed to the parser's constructor. The default value configured in TextBuffer is

final static int MAX_SEGMENT_LEN = 0x10000;

and the new segment is created in the code fragment

// Need more room?
if (outPtr >= outBuf.length) {
    outBuf = _textBuffer.finishCurrentSegment();
    outPtr = 0;
}

so instead of finishing the current segment we can simply write the contents to our writer and reset the buffer, releasing the memory each time a new chunk is read. That way we never load the whole string into the memory, only MAX_SEGMENT_LEN at maximum.

For example, you can inherit the ReaderJsonParser and write your own factory method for it analogously to the JsonFactory:

public class ChunkReaderBasedJsonParser extends ReaderBasedJsonParser {
    private boolean finishedStringBuffer = false;

    public ChunkReaderBasedJsonParser(IOContext ctxt, int features, Reader r, ObjectCodec codec, CharsToNameCanonicalizer st, char[] inputBuffer, int start, int end, boolean bufferRecyclable) {
        super(ctxt, features, r, codec, st, inputBuffer, start, end, bufferRecyclable);
    }

    public ChunkReaderBasedJsonParser(IOContext ctxt, int features, Reader r, ObjectCodec codec, CharsToNameCanonicalizer st) {
        super(ctxt, features, r, codec, st);
    }

    @Override
    public int getText(Writer writer) throws IOException {
        JsonToken t = _currToken;
        if (t == JsonToken.VALUE_STRING) {
            int total = 0;
            while (_tokenIncomplete) {
                _tokenIncomplete = getNextStringChunk();
                total += _textBuffer.contentsToWriter(writer);
                _textBuffer.resetWithEmpty();
            }
            return total;
        }
        if (t == JsonToken.FIELD_NAME) {
            String n = _parsingContext.getCurrentName();
            writer.write(n);
            return n.length();
        }
        if (t != null) {
            if (t.isNumeric()) {
                return _textBuffer.contentsToWriter(writer);
            }
            char[] ch = t.asCharArray();
            writer.write(ch);
            return ch.length;
        }
        return 0;
    }

    /**
     *
     * @return true if the token is still incomplete, false if it is complete
     */
    private boolean getNextStringChunk() throws IOException {
        finishedStringBuffer = false;
        _finishString();
        return finishedStringBuffer;
    }

    @Override
    protected void _finishString2() throws IOException {
        char[] outBuf = _textBuffer.getCurrentSegment();
        int outPtr = _textBuffer.getCurrentSegmentSize();
        final int[] codes = INPUT_CODES_LATIN1;
        final int maxCode = codes.length;

        while (true) {
            // we ran out of buffer?
            if (outPtr >= outBuf.length) {
                finishedStringBuffer = true;
                return;
            }

            if (_inputPtr >= _inputEnd) {
                if (!_loadMore()) {
                    _reportInvalidEOF(": was expecting closing quote for a string value",
                            JsonToken.VALUE_STRING);
                }
            }
            char c = _inputBuffer[_inputPtr++];
            int i = c;
            if (i < maxCode && codes[i] != 0) {
                if (i == INT_QUOTE) {
                    break;
                } else if (i == INT_BACKSLASH) {
                    /* Although chars outside of BMP are to be escaped as
                     * an UTF-16 surrogate pair, does that affect decoding?
                     * For now let's assume it does not.
                     */
                    c = _decodeEscaped();
                } else if (i < INT_SPACE) {
                    _throwUnquotedSpace(i, "string value");
                } // anything else?
            }

            // Ok, let's add char to output:
            outBuf[outPtr++] = c;
        }
        _textBuffer.setCurrentLength(outPtr);
    }
}

I debugged it for a big string and it loads it in segments, in a rather unexpected way. There has to be a bug somewhere but it's a good scaffolding to begin with. Also there is some problem with the read string length, it reads about 100kb instead of 350kb, maybe it has something to do with MAX_SEGMENT_LEN but I don't think so. I guess it's an encoding issue. And then you use it like that, write your own factory method

var factory = new ChunkJsonFactory();
try (var parser = factory.createParser(new FileReader("src/main/resources/sample-json.json"))){
        while(parser.nextToken()!=null){
        if(parser.getCurrentToken().isScalarValue()){
            parser.getText(writer);
        }
    }
}

Inheriting Writer

For example, if you want to count the characters inside of the string, you can extend Writer and use the following code

public class JsonWriter extends Writer {

    @Getter
    @Setter
    private int count;

    @Setter
    private char letterToCount = 'a';

    @Override
    public void write(char[] cbuf, int off, int len) throws IOException {
        for (int i = off; i < off + len; i++) {
            if (cbuf[i] == letterToCount) {
                count++;
            }
        }
    }
}

I created a sample repository here

You are passing into the JSONObject constructor an instance of a StringBuilder class.

This is using the JSONObject(Object) constructor, not the JSONObject(String) one.

Your code should be:

JSONObject jsonObj = new JSONObject(jsonString.toString());