Make sure you are downloading from the "raw" content URL, and not the HTML github page. I believe this is the URL for the raw JSON file you want:

https://raw.githubusercontent.com/yasuhisa1984/achieve/master/vendor/bundle/gems/fog-aws-1.2.1/lib/fog/aws/iam/default_policy_versions.json

The following code works for me with that URL:

import requests
url = "https://raw.githubusercontent.com/yasuhisa1984/achieve/master/vendor/bundle/gems/fog-aws-1.2.1/lib/fog/aws/iam/default_policy_versions.json"
resp = requests.get(url)
data = resp.json()
Answer from totalhack on Stack Overflow
🌐
Testengineeringnotes
testengineeringnotes.com › posts › 2023-03-25-python-github-api
Get file content from Github repository — Test Engineering Notes
March 25, 2023 - Personal access token can be generated at Settings / Developer settings / Personal access tokens · To get the file content from Github repository you need to check the Github API for Repository / Content.
🌐
GitHub
docs.github.com › en › rest › repos › contents
REST API endpoints for repository contents - GitHub Docs
Use the REST API to create, modify, and delete Base64 encoded content in a repository. Gets the contents of a file or directory in a repository. Specify the file path or directory with the path parameter.
Top answer
1 of 3
42

As the description (located at http://developer.github.com/v3/repos/contents/) says:

/repos/:owner/:repo/contents/:path

An ajax code will be:

$.ajax({
    url: readme_uri,
    dataType: 'jsonp',
    success: function(results)
    {
        var content = results.data.content;
    });

Replace the readme_uri by the proper /repos/:owner/:repo/contents/:path.

2 of 3
39

This GitHub API page provides the full reference. The API endpoint for reading a file:

https://api.github.com/repos/{username}/{repository_name}/contents/{file_path}
{
  "encoding": "base64",
  "size": 5362,
  "name": "README.md",
  "content": "encoded content ...",
  "sha": "3d21ec53a331a6f037a91c368710b99387d012c1",
  ...
}
  • Consider using a personal access token
    • Rate-limits (up to 60 per-hour for anonymous, up to 5,000 per-hour for authenticated) read more
    • Enable accessing files in private repos
  • The file content in the response is base64 encoded string

Using curl

Reading https://github.com/airbnb/javascript/blob/master/package.json using GitHub's API via curl:

curl -H 'Accept: application/vnd.github.v3.raw' https://api.github.com/repos/airbnb/javascript/contents/package.json
  • Make sure to pass header Accept: application/vnd.github.v3.raw to get raw file response (thanks jakub.g)

Using Python

Reading https://github.com/airbnb/javascript/blob/master/package.json using GitHub's API in Python:

import base64
import json
import requests
import os


def github_read_file(username, repository_name, file_path, github_token=None):
    headers = {}
    if github_token:
        headers['Authorization'] = f"token {github_token}"
        
    url = f'https://api.github.com/repos/{username}/{repository_name}/contents/{file_path}'
    r = requests.get(url, headers=headers)
    r.raise_for_status()
    data = r.json()
    file_content = data['content']
    file_content_encoding = data.get('encoding')
    if file_content_encoding == 'base64':
        file_content = base64.b64decode(file_content).decode()

    return file_content


def main():
    github_token = os.environ['GITHUB_TOKEN']
    username = 'airbnb'
    repository_name = 'javascript'
    file_path = 'package.json'
    file_content = github_read_file(username, repository_name, file_path, github_token=github_token)
    data = json.loads(file_content)
    print(data['name'])


if __name__ == '__main__':
    main()
  • Define an environment variable GITHUB_TOKEN before running
🌐
Readthedocs
pygithub.readthedocs.io › en › latest › examples › Repository.html
Repository — PyGithub 0.1.dev50+ge36ffcbb0 documentation
>>> repo = g.get_repo("PyGithub/PyGithub") >>> contents = repo.get_contents("") >>> while contents: ... file_content = contents.pop(0) ... if file_content.type == "dir": ... contents.extend(repo.get_contents(file_content.path)) ... else: ... print(file_content) ... ContentFile(path=".gitignore") ContentFile(path="CONTRIBUTING.md") ... ContentFile(path="github/tests/ReplayData/Team.testRepoPermission.txt") ContentFile(path="github/tests/ReplayData/Team.testRepos.txt") ContentFile(path="github/tests/ReplayData/UserKey.setUp.txt")
🌐
LONGLOVEMYU
longlovemyu.com › githubapi
Access files using github api | LONGLOVEMYU
March 26, 2020 - My idea is using get_contents for a directory to request sha and download_url keywords, then use requests.get(). Git Blobs API is another solution. ... Notice: update_file need sha parameter, you have to request sha before use it.
🌐
Medium
medium.com › plumbersofdatascience › import-and-export-files-to-and-from-github-via-api-626efd7dd859
Import and Export Files to and from GitHub via API | by Henry Alpert | Plumbers Of Data Science | Medium
January 12, 2023 - Then, use the get_contents() method to grab all the file’s information from that GitHub repo and assign it to a variable. This call creates a PyGithub ContentFile object, but the content is not in the desired form yet, because it is still ...
🌐
Stack Overflow
stackoverflow.com › questions › 65413318 › github-python-api-accessing-file-content-after-get-files
pygithub - Github Python API accessing file content after get_files() - Stack Overflow
from github import Github github = Github('user', 'token') user = github.get_user() repository = user.get_repo('YourRepository') pr = repository.get_pull(30) filelist = [] for file in pr.get_files(): filelist.append(file.filename) for each_file in filelist: print(repository.get_contents(each_file)) ... Sign up to request clarification or add additional context in comments. ... Find the answer to your question by asking. Ask question ... See similar questions with these tags. ... Settle down, nerds. AI is a normal technology ... Stack Overflow chat opening up to all users in January; Stack Exchange chat... Modernizing curation: A proposal for The Workshop and The Archive ... 0 Python API for Github, getting contents in specific directory for specific branch not returning all content
Find elsewhere
🌐
Stack Overflow
stackoverflow.com › questions › 58592214 › how-to-pull-contents-of-a-file-using-python-and-ghe-api
github api - How to pull contents of a file using python and GHE API - Stack Overflow
The above returns {"message":"Not Found","documentation_url":"https://developer.github.com/enterprise/2.16/v3/repos/contents/#get-contents"} in the .txt file ... import requests import json request = requests.get('https://api.github.com/path') if(request.ok): content = json.loads(request.content) # Do something with content
🌐
PyPI
pypi.org › project › github-contents
github-contents · PyPI
June 9, 2019 - Note that file contents is passed and returned as bytestrings, not regular strings. ... You will need a GitHub OAuth token with full repository access. The easiest way to create one of these is using https://github.com/settings/tokens · from github_contents import GithubContents # For repo simonw/disaster-data: github = GithubContents( "simonw", "disaster-data", token=GITHUB_OAUTH_TOKEN, branch="main" ) ... content_sha, commit_sha = github.write( filepath=path_within_repo, content_bytes=contents_in_bytes, sha=previous_sha, # Optional commit_message=commit_message, committer={ "name": COMMITTER_NAME, "email": COMMITTER_EMAIL, }, )
      » pip install github-contents
    
Published   Feb 24, 2021
Version   0.2
Top answer
1 of 2
2

Yes, it is different.

Github REST API download files has two steps.

1, The first step is get the download url.

The url format like this:

https://api.github.com/repos/<Project Name>/<Repository Name>/contents/<File Name>

The response format like this:

2, The second step is using the download url to get the file content.

The url format like this:

https://raw.githubusercontent.com/<Project Name>/<Repository Name>/main/<File Name>?token=<Random Token that related to Revision Version>

Please notice that the first step can't skip, otherwise you will be unable to get the revision token.

I can achieve your requirement using python:

import requests

#Define required information
project_name = "xxx"
repository_name = "xxx"
# repository_name = "xxx"
branch_name = "xxx"
File_name = "xxx"
PAT = "xxx"
url = "https://api.github.com/repos/"+project_name+"/"+repository_name+"/contents/"+File_name

#downoad file from github
payload = {}
headers = {
    'Authorization': 'token '+PAT
}

#download file
file_content = requests.request("GET", ((requests.request("GET", url, headers=headers, data=payload)).json())['download_url'], headers=headers, data=payload)
print(file_content.text)

Successfully get the latest contents(I am also based on private repository):

2 of 2
0

If you want a raw file, you just need to set request header Accept with application/vnd.github.v3.raw. Just one step would be enough.

GitHub API: /repos/{owner}/{repo}/contents/{path}

Refer to:

Official way to access / download github files from scripts?

https://docs.github.com/en/rest/repos/contents

🌐
GitHub
gist.github.com › keithweaver › b0912519d410b7e2ab3c98bf350bcfc2
Get File Content with Python · GitHub
January 6, 2020 - Get File Content with Python · Raw · get-file.py · This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
🌐
GitHub
github.com › PyGithub › PyGithub › issues › 576
Is there a way to get raw format of a ContentFile? · Issue #576 · PyGithub/PyGithub
May 5, 2017 - ContentFile.content returns the base64-encoded content from the API. The API also supports getting the raw format of the file. These API methods let you retrieve the contents of files within a repo...
Author   PyGithub
🌐
GitHub
github.com › PyGithub › PyGithub › issues › 140
Repository.get_contents does not return directory information · Issue #140 · PyGithub/PyGithub
February 12, 2013 - The GitHub API returns a list of all files in a directory when you use Get Contents on a directory. example: https://api.github.com/repos/twitter/bootstrap/contents/js/?ref=d28343dc3ad53a411ae3685e7d6a7866c8c22d6b Currently PyGithub only...
Author   PyGithub
🌐
The Python Code
thepythoncode.com › code › using-github-api-in-python
Code for How to Use Github API in Python - Python Code
from github import Github # your github account credentials username = "username" password = "password" # initialize github object g = Github(username, password) # searching for my repository repo = g.search_repositories("pythoncode tutorials")[0] # create a file and commit n push repo.create_file("test.txt", "commit message", "content of the file") # delete that created file contents = repo.get_contents("test.txt") repo.delete_file(contents.path, "remove test.txt", contents.sha) Ethical Hacking ·
🌐
Stack Overflow
stackoverflow.com › questions › 58902999 › get-python-file-content-from-github-api-and-parse-to-dict
git - Get python file content from github api and parse to dict - Stack Overflow
November 18, 2019 - When i try to access the manifest files i got base64 content which i can't encode to python code. How could i decode these files into .py files or extract the dict from the file. ... I tried an other approach to get the file content as from Github api response,it send the download url which is the file + access token,so i used that url to do requests.get() then i used ast lib to convert the text to python dict
🌐
Python Forum
python-forum.io › thread-26072.html
Read content of GitHub file
Let say , I have repository called "Sample" in my GitHub and inside my repository i have a file named "demo.txt" or any format file. How do I read the content of file which is in GitHub ? Can i able t