I've had success using the browsable artifacts for this purpose. In your example, you would create a job for your develop branch and set the PUBLIC_URL to the path on gitlab.io where the job's artifacts are published:

develop:
    artifacts:
        paths:
          - public

    environment:
        name: Develop
        url: "https://$CI_PROJECT_NAMESPACE.gitlab.io/-/$CI_PROJECT_NAME/-/jobs/$CI_JOB_ID/artifacts/public/index.html"

    script: |
        # whatever

    stage: deploy

    variables:
        PUBLIC_URL: "/-/$CI_PROJECT_NAME/-/jobs/$CI_JOB_ID/artifacts/public"

Setting the environment as indicated produces a »Review app« link in relevant merge requests, allowing you to get to the artifacts with a single click.

Note: if your repository is in a subgroup, you need to insert the subgroup name in two places above above between /-/ and $CI_PROJECT_NAME for the resulting URLs to work.

Interestingly, it is possible to post from any branch, just not from any job.

To do this, I needed to make two changes:

1. I need to know the current state of the published data
2. I need to change directory based on the current branch

GitLab has the ability to cache folders. Generally this is used to speed up builds by caching downloaded drivers. There is no reasons I could not use this to store the public folder. This way when I make changes to staging, I will remember the state of the root application:

cache:
  paths:
  - public

The next trick would be to publish pages to the appropriate folder, depending on current branch being built. To do this, we can look to GitLab CI/CD Environment Variables; in particular:

• CI_COMMIT_REF_SLUG: The current branch
• CI_DEFAULT_BRANCH: the default branch (master)

Knowing these two values, we can do a bit of bash to determine the correct place to write the content to.

pages:
  stage: deploy
  script:
  - dir="$CI_COMMIT_REF_SLUG"
  - if [ "$CI_COMMIT_REF_SLUG" == "$CI_DEFAULT_BRANCH" ]; then dir=""; fi;
  - dir="public/$dir"
  - echo "Deploying to $dir"
  - mkdir -p $dir
  - cp -r www $dir
  artifacts:
    paths:
    - public
  only:
  - staging
  - master

Don't forget to limit pages to only staging and master.

WARNING

I'm not satisfied with this.

I think it would be better to maintain the cache somewhere completely different and copy them in at a later stage, but completely re-writing the public folder each time.

The current solution will build cruft over time, but the basic idea is sound.

Only one site is supported on these platforms. You cannot have separate sites for separate branches.

The only way you might do this without some sort of cache/artifact retrieval (as you mentioned is another option in your question) is to build all your branches at once when publishing your Pages site.

How exactly you do that depends on a lot of factors, including what tool(s) you're using to build your site and if they are context-dependent -- but it might look something like this in GitLab

pages:
  # fetch the whole repo
  # this logic can change if you're on a detached head, like an MR
  # so we account for that here
  before_script: | 
        if [[ -n "$CI_COMMIT_BRANCH" ]]; then  # branch pipelines
            git remote set-url origin "https://gitlab-ci-token:${CI_JOB_TOKEN}@${CI_SERVER_HOST}/${CI_PROJECT_PATH}.git"
            git fetch origin
            git checkout $CI_COMMIT_BRANCH
        fi
        if [[ -n "$CI_MERGE_REQUEST_IID" ]]; then  # MR pipelines
            git remote set-url origin "https://gitlab-ci-token:${CI_JOB_TOKEN}@${CI_SERVER_HOST}/${CI_MERGE_REQUEST_SOURCE_PROJECT_PATH}.git"
            git fetch origin
        fi
  script: |
        mkdir public
        branches=()
        # ref: https://stackoverflow.com/a/3847586/5747944
        eval "$(git for-each-ref --shell --format='branches+=(%(refname))' refs/heads/)"
        for branch in "${branches[@]}"; do
            git checkout "$branch"
            # build each branch and output to public directory
            # YOU implement this
            make build "$branch" -o "public/${branch}"
        done
  artifacts:
    paths:
      - public
  environment: # ensure outdated jobs are skipped
    name: pages # https://docs.gitlab.com/ee/ci/environments/deployment_safety.html#skip-outdated-deployment-jobs