I have a large number of big json files and I cannot open all of them at the same time. I'm trying to merge them into one json file, but all the solutions I find on SO (e.g., this) require opening the files which causes my machine to crash. I was wondering if there's a way to merge them without opening them
You're using the json module to convert the JSON file into Python objects, but you're not using the module to convert those Python objects back into JSON. Instead of this at the end
textfile_merged.write(str(all_items))
try this:
json.dump({ "items": all_items }, textfile_merged)
(Note that this is also wrapping the all_items array in a dictionary so that you get the output you expect, otherwise the output will be a JSON array, not an object with an "items" key).
If you just want to merge all json files sequentially,
go to the folder where all json files are, select all and rename the first one as "yourchoice", by doing this all will be in sequential order i.e. yourchoice1,yourchoice2 ...
next go to cmd and type : copy *.json "outputfilename".json
All of your json files are merged sequentially into the "outputfilename".json file
Combine multiple json files into one json file
Combine two JSON objects into one
Merging Multiple JSON Files Using Azure Data Factory Copy Activity
How to merge multiple json files into one file in python - Stack Overflow
How do I merge multiple JSON files?
Will the merged JSON file preserve my data structure?
Do I need to create an account to merge JSON files?
Videos
You should use extend instead of append. It will add the items of the passed list to result instead of a new list:
files=['my.json','files.json',...,'name.json']
def merge_JsonFiles(filename):
result = list()
for f1 in filename:
with open(f1, 'r') as infile:
result.extend(json.load(infile))
with open('counseling3.json', 'w') as output_file:
json.dump(result, output_file)
merge_JsonFiles(files)
import json
import pandas as pd
with open('example1.json') as f1: # open the file
data1 = json.load(f1)
with open('example2.json') as f2: # open the file
data2 = json.load(f2)
df1 = pd.DataFrame([data1]) # Creating DataFrames
df2 = pd.DataFrame([data2]) # Creating DataFrames
MergeJson = pd.concat([df1, df2], axis=1) # Concat DataFrames
MergeJson.to_json("MergeJsonDemo.json") # Writing Json