I just don't get it. I'm a junior and see it often in the codebase of the company i work at. Documentation says that it helps serialize and deserialize objects, but why does that need to happen using this interface? There are so many classes that do not implement Serializable, so what happens to them?
Head First Java book says that objects need to be serialized when data is sent over the network or saved to a disk. But there is serialization/deserialization happening to JSON objects for example when they're being sent from server to client and vice versa, and those classes do not implement Serializable.
So in which "special" scenario does one need/want to implement Serializable?
xml - What is the need of serialization of objects in Java? - Stack Overflow
Java JSON serialization - best practice - Stack Overflow
json - Java Serialization for long-ish-term storage - Software Engineering Stack Exchange
java - Why does Jackson implement Serializable? - Stack Overflow
Videos
Short story about serialization
After many years of hard work, Earth's scientists developed a robot who can help them in daily work. But this robot had fewer features than the robots developed by the scientists from planet Mars.
After a meeting between both planets' scientists, it is decided that Mars will send their robots to Earth. But a problem occurred. The cost of sending 100 robots to Earth was $100 million. And it takes around 60 days of traveling.
Finally, Mars' scientists decided to share their secret with Earth's scientists. This secret was about the structure of class/robot. Earth's scientists developed the same structure on Earth itself. Mars' scientists serialized the data of each robot and sent it to earth. Earth's scientists deserialized the data and fed it into each robot accordingly.
This process saved them time in communicating a massive amount of data.
Some of the robots were being used in some defensive work on Mars. So their scientists marked some crucial properties of those robots as transient before sending their data to Earth. Note that the transient property is set to null (in case of reference) or to the default value (in case of the primitive type) when the object gets deserialized.
One more point noticed by Earth's scientists is that Mars' scientists asked them to create some static variables to keep details about the environment. These details are used by some robots. But Mars' scientists don't share these details. Because Earth's environment was different from Mars' environment.
Even though knowing about the robot class structure and having serialized data Earth's scientist were not able to deserialize the data which can make robots working.
Exception in thread "main" java.io.InvalidClassException:
SerializeMe; local class incompatible: stream classdesc
:
Mars' scientists were waiting for the complete payment. Once the payment was done Mars' scientists shared the serialversionUID with Earth's scientists. Earth's scientist set it to robot class and everything started working.
Update
Though with the help of serialization, they became able to send data using signals instead of actual spaceship, they realized that sending large set of data was still a challenge. Serialization make the process cheaper and faster but it was still slow. Hence the different scientists came up with different ideas to reduce the data size. Some scientists suggested to compress the data and some suggested to use different mechanism to represent it so it can be deserialized back. Some of the ideas are XML, JSON, msgpack, Nimn
Serialization is usually used When the need arises to send your data over network or stored in files. By data I mean objects and not text.
Now the problem is your Network infrastructure and your Hard disk are hardware components that understand bits and bytes but not JAVA objects.
Serialization is the translation of your Java object's values/states to bytes to send it over network or save it.
This is analogous to how your voice is transmitted over PSTN telephone lines.
Are you tied to this library? Google Gson is very popular. I have myself not used it with Generics but their front page says Gson considers support for Generics very important.
As others have hinted, you should consider dumping org.json's library. It's pretty much obsolete these days, and trying to work around its problems is waste of time.
But to specific question; type variable T just does not have any information to help you, as it is little more than compile-time information. Instead you need to pass actual class (as 'Class cls' argument), and you can then create an instance with 'cls.newInstance()'.
While you do not anticipate changing the nature of your objects now very much - needs change over time.
I would highly suggest you consider using protobuf or Apache Thrift or a similar design instead of relying on default Java serialization.
Their advantages include strong support for avoiding impact during minor version changes, significantly better serialization and deserialization speed and smaller footprint of objects.
When I do this, typically I include the message version as a field on the object. This allows me to implement version-to-version changes during the serialization by wrapping that method, if that need should arise in the future.
I wouldn't store the serialized objects for long term. As the others already stated: When you change a litte bit on your classes, (maybe you don't even notice it) maybe you can't load your data. It will be hell to fix it.
I'm not sure, why you need all the data in the memory. Is it, because it is used everytime and everywhere in your software (master data)? Looks like a "Yes"
What I would do: I would store the data in a inmemory-database (hsqldb,...) dump the file and provide the file on the machine After the startup I would load the file (quite fast) and then use it like a "normal" database.
You can do this also with your serialized objects. But I wouldn't see a sense any more. The "normal" database can be cached and will be extremly fast. So query the object.
Any time you deserialize an object by calling ObjectInputStream.readObject, you have a remote code execution vulnerability: if someone can make you deserialize the wrong bytes, they can run any command on your computer.
The exploit works by creating an object that will run evil code inside its readObject method, which is called during the process of deserialization, then serializing this object and making you deserialize it. At first glance, you'd think this could only happen if the attacker could load an evil class into your program, in which case they've already hacked your program with or without serialization. However, there are several ways to create an "evil object" using only classes from common libraries (example) and in the future one might be found in the standard library, which would work in every program.
If you happen to be familiar with CVE-2010-0840 (escaping the Java applet sandbox using "Trusted Method Chaining") the concept is very similar but the details are completely different.
The exploit occurs during the process of deserialization, inside readObject, so nothing you do after readObject returns can prevent it.
For more details, see the writeup.
You can prevent the exploit in a cumbersome way, by creating a subclass of ObjectInputStream, overriding the resolveClass method so that only the specific classes you want to be deserialized can be resolved, and using this subclass for deserialization, or by calling setObjectInputFilter with an appropriate filter before reading any objects.
"never" is a strong word. However, when the official documentation of a class starts with a bold security warning:
Warning: Deserialization of untrusted data is inherently dangerous and should be avoided. Untrusted data should be carefully validated according to the "Serialization and Deserialization" section of the Secure Coding Guidelines for Java SE. Serialization Filtering describes best practices for defensive use of serial filters.
(emphasis in original)
it's probably a good idea to take that warning seriously.
In a nutshell, the problem with Java Serialization is that the classes to be instantiated are determined by the serialized data. That is, the data controls with objects are created, and can instantiate any (serializable) class available to your program. In addition, classes can declare methods to be invoked upon deserialization, which allows an attacker to invoke any such method with any input. While care should be taken to harden such methods against misuse, their sheer number makes it virtually certain that at least one class in your classpath remains vulnerable, and can be exploited by the attacker.
(Yes, after the object has been instantiated and returned to your code, your code will notice that the object is of the wrong type and raise an error. However, that happens after the attacker-selected code has executed, and is therefore too late to prevent remote code execution, resource exhaustion, or whatever else the vulnerable method did)
More modern serialization libraries prevent this by asking the caller for the expected class, rather than trusting the data. For instance, with a Jackson ObjectMapper, you'd write:
var person = objectMapper.readObject(Person.class)
That puts control of which classes are instantiated into the hands of the receiving rather than the sending program.
To mitigate the risk in cases where we can't switch to a more modern data format, ObjectInputStreams have been extended with filtering options. However, due to backward compatiblity concerns, these protections are disabled by default. They are also somewhat onerous to configure. And filters just restrict which classes can be instantiated - if one of those is vulnerable, you can still be owned.
Overall, the official recommendation that Java Serialization should not be used with untrusted data is entirely reasonable.
Now, one might argue that Java Serialization remains suitable if data comes from a trusted source. I'd disagree, because
- it's quite possible that even though the data is trusted now, it may become untrusted in the future. For instance, suppose you use Java Serialization for session persistence. Sounds harmless, right? But suppose the customer reports some weird error when reading persisted sessions. To diagnose the issue, you ask them for the session file and load it in your dev environment. Oops.
- in the event that a trusted system is compromised, Java Serialization can be used to escalate the compromise to your system
What could possibly justify these risks? The only things in favor of Java Serialization are that it is part of the JDK and easy to use. But more modern serialization libraries are also easy to use, and often have a more readable, text-based wire format such as JSON, which helps greatly with debugging. (One might expect Java Serialization's binary format to be more compact, but it also contains a lot more metadata, which can more than offet the gains from the binary format). And in the age of Maven (or whichever dependency managment tool you prefer), pulling in an additional library is not nearly as hard as it used to be.
Overall, I find the recommendation to "never use Java Serialization" quite sensible. It's one of those early Java technologies that did not anticipate the security challenges we face today.