You can also use the recently released Gitlab GraphQL API to query groups by name :

{
  group(fullPath: "your_group_here") {
    projects {
      nodes {
        name
        description
        httpUrlToRepo
        nameWithNamespace
        starCount
      }
    }
  }
}

You can go to the following URL : https://[your_gitlab_host]/-/graphql-explorer and past the above query

The Graphql endpoint is a POST on "https://$gitlab_url/api/graphql" An example using curl and jq:

gitlab_url=<your gitlab host>
access_token=<your access token>
group_name=<your group>

curl -s -H "Authorization: Bearer $access_token" \
     -H "Content-Type:application/json" \
     -d '{ 
          "query": "{ group(fullPath: \"'$group_name'\") { projects {nodes { name description httpUrlToRepo nameWithNamespace starCount}}}}"
      }' "https://$gitlab_url/api/graphql" | jq '.'

Use %2f to specify full path to subgroup:

curl -s https://gitlab.com/api/v4/groups/gitlab-org%2fgitter/projects

See this issue.

The API endpoint to list a group's projects has an include_subgroups attribute that will include all of the projects listed under the subgroups, is that what you're trying to do?

So, inspired by the answer of sytech, I found out that it was not working in the first place, as the shared projects were still hidden in the subgroups. So I came up with the following code that digs through all various levels of subgroups to find all shared projects. I assume this can be written way more elegant, but it works for me:

# group definition
main_group_id = 11111

# create empty list that will contain final result
list_subgroups_id_all = []

# create empty list that act as temporal storage of the results outside the function
list_subgroups_id_stored = []



# function to create a list of subgroups of a group (id)
def find_subgroups(group_id):

    # retrieve group object
    group = gl.groups.get(group_id)

    # create empty lists to store id of subgroups
    list_subgroups_id = []

    #iterate through group to find id of all subgroups
    for sub in group.subgroups.list():
        list_subgroups_id.append(sub.id)

    return(list_subgroups_id)


# function to iterate over the various groups for subgroup detection
def iterate_subgroups(group_id, list_subgroups_id_all):

    # for a given id, find existing subgroups (id) and store them in a list
    list_subgroups_id = find_subgroups(group_id)

    # add the found items to the list storage variable, so that the results are not overwritten
    list_subgroups_id_stored.append(list_subgroups_id)

    # for each found subgroup_id, test if it is already part of the total id list
    # if not, keep store it and test for more subgroups
    for test_id in list_subgroups_id:
        if test_id not in list_subgroups_id_all:

            # add it to total subgroup id list (final results list)
            list_subgroups_id_all.append(test_id)

            # check whether test_id contains more subgroups
            list_subgroups_id_tmp = iterate_subgroups(test_id, list_subgroups_id_all)

            #if so, append to stored subgroup list that is currently checked
            list_subgroups_id_stored.append(list_subgroups_id_tmp)

    return(list_subgroups_id_all)

# find all subgroup and subsubgroups, etc... store ids in list
list_subgroups_id_all = iterate_subgroups(main_group_id , list_subgroups_id_all)

print("***ids of all subgroups***")
print(list_subgroups_id_all)
print("")

print("***names of all subgroups***")
list_names = []
for ids in list_subgroups_id_all:
    group = gl.groups.get(ids)
    group_name = group.attributes['name']
    list_names.append(group_name)
print(list_names)
#print(list_subgroups_name_all)
print("")

# print all directly integrated projects of the main group, also those in subgroups
print("***integrated projects***")
group = gl.groups.get(main_group_id)
projects=group.projects.list(include_subgroups=True, all=True)
for prj in projects:
    print(prj.attributes['name'])
print("")

# print all shared projects
print("***shared projects***")
for sub in list_subgroups_id_all:
    group = gl.groups.get(sub)
    for shared_prj in group.shared_projects:
        print(shared_prj['path_with_namespace'])
print("")

One question that remains - at the very beginning I retrieve the main group by its id (here: 11111), but can I actually also get this id by looking for the name of the group? Something like: group_id = gl.group.get(attribute={'name','foo'}) (not working)?

I also have not found any API to help me, and then write a script to do that.

CURRENT_DIR=$(dirname "$0")
GITLAB_SITE="<gitlab site>"
GITLAB_GROUP="<group>"
GITLAB_ACCESS_TOKEN="<access token>"

#### get projects url #####
url_file=$(mktemp)
temp_file=$(mktemp)
for ((page=1; ; page+=1)); do
  # iterate all pages
  url="${GITLAB_SITE}/api/v4/projects?per_page=100&page=${page}"
  data=$(curl  --request GET --header "PRIVATE-TOKEN: ${GITLAB_ACCESS_TOKEN}" $url )
  [ $(jq length <<< "$data") -eq 0 ] && break

  echo "$data" \
  | jq -r --arg GITLAB_GROUP "$GITLAB_GROUP" '.[] | select(.path_with_namespace | startswith("smartcompany")) | [.id] | @csv' \
  | while read -r id; do echo "${GITLAB_SITE}/api/v4/projects/${id}/members/all" >> ${url_file} ; done
done

#### get groups url #####
temp_file=$(mktemp)
for ((page=1; ; page+=1)); do
  # iterate all pages
  url="${GITLAB_SITE}/api/v4/groups?per_page=100&page=${page}"
  data=$(curl  --request GET --header "PRIVATE-TOKEN: ${GITLAB_ACCESS_TOKEN}" $url )
  [ $(jq length <<< "$data") -eq 0 ] && break

  echo "$data" \
  | jq -r --arg GITLAB_GROUP "$GITLAB_GROUP" '.[] | select(.full_path | test("^" + $GITLAB_GROUP + "(/|$)")) | [.id] | @csv' \
  | while read -r id; do echo "${GITLAB_SITE}/api/v4/groups/${id}/members/all" >> ${url_file} ; done
done


#### get menbers ####
tmp_members="$(mktemp)"
while IFS= read -r url
do
  for ((page=1; ; page+=1)); do
    url="${url}?per_page=100&page=${page}"
    data=$(curl  --request GET --header "PRIVATE-TOKEN: ${GITLAB_ACCESS_TOKEN}" $url )
  [ $(jq length <<< "$data") -eq 0 ] && break

    # data
    echo "$data" | jq -r '.[] | [.id,.name] | @csv' >> $tmp_members
  done

done < "${url_file}"

#### remove duplication ####
members_csv="${CURRENT_DIR}/users-$(date '+%Y-%m-%d').csv"
echo "id,name" > "${members_csv}"
sort -n ${tmp_members} | uniq >> "${members_csv}"

It seems it can get only 100 per page so you will have to run it two times - first with page=1 and next with page=2. And for second page you will need >> to append to existing file gitlab.txt

curl --header "..." "https://...&per_page=100&page=1" | ...  > gitlab.txt

curl --header "..." "https://...&per_page=100&page=2" | ...  >> gitlab.txt

Or you will have to write script which first get all pages and later send it to pipe. You may also try to use for-loop in bash

If only your private token is available, you can only use the API:

PROJECTS

Use the following command to request projects:

curl "https://<host>/api/v4/projects?private_token=<your private token>"

This will return you the first 20 entries. To get more you can add the paramater per_page

curl "https://<host>/api/v4/projects?private_token=<your private token>&per_page=100"

with this parameter you can request between 20and 100 entries (see REST API Pagination documentation).

If you now want all projects, you have to loop through the pages. To get to another page add the parameter page.

curl "https://<host>/api/v4/projects?private_token=<your private token>&per_page=100&page=<page_number>"

Now you may want to know how many pages there are. For that, add the curl parameter --head. This will not return the payload, but the header.

The result should look like this:

HTTP/1.1 200 OK
Server: nginx
Date: Thu, 13 Jul 2017 17:43:24 GMT
Content-Type: application/json
Content-Length: 29428
Cache-Control: no-cache
Link: <request link>
Vary: Origin
X-Frame-Options: SAMEORIGIN
X-Next-Page: 2
X-Page: 1
X-Per-Page: 20
X-Prev-Page:
X-Request-Id: 80ecc167-4f3f-4c99-b09d-261e240e7fe9
X-Runtime: 4.117558
X-Total: 312257
X-Total-Pages: 15613
Strict-Transport-Security: max-age=31536000

The two interesting parts here are X-Totaland X-Total-Pages. The first is the count of available entries and the second the count of total pages.

I suggest to use python or some other kind of script to handle the requests and concat the results at the end.

If you want to refine the search, consult this wiki page: https://docs.gitlab.com/api/projects.html#projects-api

GROUPS

For groups simply replace projects with groups in the curls. https://docs.gitlab.com/api/groups.html#list-groups

UPDATE:

Here is the official list of Gitlab API clients/wrappers: https://docs.gitlab.com/api/rest/third_party_clients/

I highly recommend using one of these.