First of all, it depends on what operating system you're using.
On Windows
If you right click and press "save" (Alternatively use ctrl-s) and then chose the file format "json" when saving.
On macOS
You can do this in a few ways, I'd suggest using curl. You'd do this by opening the terminal and navigating to the directory you'd like to download the file to. Then enter the following:
curl https://somedomain.com/somepath.json > fileName.json
This will download the file in its original format.
Answer from Maximilian on Stack OverflowFirst of all, it depends on what operating system you're using.
On Windows
If you right click and press "save" (Alternatively use ctrl-s) and then chose the file format "json" when saving.
On macOS
You can do this in a few ways, I'd suggest using curl. You'd do this by opening the terminal and navigating to the directory you'd like to download the file to. Then enter the following:
curl https://somedomain.com/somepath.json > fileName.json
This will download the file in its original format.
In a Mac, the curl solution proposed above doesn't work if the url isn't public (on my side I got this permissions error: unauthorized permission requested.
So, here's an alternative that worked for me. Using the Chrome browser, go to the url with the json, then right click, then 'Inspect'. That brings up the Chrome devtools ui. From there go to 'Sources' and you will see the json file in the list. Then right click and you will be able to click 'Save as'. This will then download the file to wherever you want!
UPDATE: In case this method doens't work (I realise it doesn't work in all cases), then simply copy-paste the text of the json into a text editor, and save the file with the json extension.
So I've got a JSON file I want to download from a website.
The data comes in a URL that ends with a 6 digit ID number (unfortunately I cannot find a logic to the ID but it's not a single increment system, so I'd like to be able to check like the next 10,000 ID's say once a week).
I'd then like to pull parts of the JSON files into some kind of spreadsheet (dumping to a CSV is fine). There is a lot of data I don't need/want in these files. I know how to pull the data into excel but I feel like there will be a better way to do this automagically.
Videos
I'm trying to download JSON files from a URL. When I've attempted to open the saved JSON files in python, I get the error:
"raise JSONDecodeError("Expecting value", s, err.value) from None".
Whenever I try opening the JSON files in a URL, I see this:
"SyntaxError: JSON.parse: unexpected character at line 1 column 1 of the JSON data"
Below is a simplified version of my code. Is there a way to download JSON files correctly?
def read_url(url):
urls = []
psfiles = []
url = url.replace(" ","%20")
req = Request(url)
a = urlopen(req).read()
soup = BeautifulSoup(a, 'html.parser')
x = (soup.find_all('a'))
for i in x:
file_name = i.extract().get_text()
url_new = url + file_name
url_new = url_new.replace(" ","%20")
if(file_name[-1]=='/' and file_name[0]!='.'):
read_url(url_new)
if url_new.endswith('json'):
urls.append(url_new)
for i in urls:
psfile = i.replace('url','')
psfiles.append(psfile)
for j in range(len(psfiles)):
urllib.request.urlretrieve("url", "path to directory"+psfiles[j])
if __name__ == '__main__':
while True:
read_url("url")
time.sleep(1800)It would be the files listed here. https://open.fda.gov/apis/drug/event/download/
I have been using youtube videos on how to bulk download but I haven't figure it out.
Adding the HTML5 download attribute to the <a> tag should work for you
Anchor Tag doc
download HTML5
This attribute instructs browsers to download a URL instead of navigating to it, so the user will be prompted to save it as a local file
The docs also note some gotchas like it will only work for same-origin URLs and a few others so its worth a read.
Unfortunately the download attribute is not supported in Internet Explorer so for the download to work properly you will have to add Content-Type and Content-Disposition headers on the response from the server
they should look something like:
Content-Type: application/octet-stream
Content-Disposition: attachment;filename=\"filename.json\"
Add header:
header('Content-disposition: attachment; filename=file.json');
header('Content-type: application/json');
echo $json;