I've had success using the browsable artifacts for this purpose. In your example, you would create a job for your develop branch and set the PUBLIC_URL to the path on gitlab.io where the job's artifacts are published:
develop:
artifacts:
paths:
- public
environment:
name: Develop
url: "https://$CI_PROJECT_NAMESPACE.gitlab.io/-/$CI_PROJECT_NAME/-/jobs/$CI_JOB_ID/artifacts/public/index.html"
script: |
# whatever
stage: deploy
variables:
PUBLIC_URL: "/-/$CI_PROJECT_NAME/-/jobs/$CI_JOB_ID/artifacts/public"
Setting the environment as indicated produces a »Review app« link in relevant merge requests, allowing you to get to the artifacts with a single click.
Note: if your repository is in a subgroup, you need to insert the subgroup name in two places above above between /-/ and $CI_PROJECT_NAME for the resulting URLs to work.
git - Deploying GitLab pages for different branches - Stack Overflow
continuous integration - GitLab CI/CD removes old artifacts from pages public folder - Stack Overflow
Gitlab Pages: /public cannot be used for the publicly present files.
Downloadable file on Gitlab Pages
I've had success using the browsable artifacts for this purpose. In your example, you would create a job for your develop branch and set the PUBLIC_URL to the path on gitlab.io where the job's artifacts are published:
develop:
artifacts:
paths:
- public
environment:
name: Develop
url: "https://$CI_PROJECT_NAMESPACE.gitlab.io/-/$CI_PROJECT_NAME/-/jobs/$CI_JOB_ID/artifacts/public/index.html"
script: |
# whatever
stage: deploy
variables:
PUBLIC_URL: "/-/$CI_PROJECT_NAME/-/jobs/$CI_JOB_ID/artifacts/public"
Setting the environment as indicated produces a »Review app« link in relevant merge requests, allowing you to get to the artifacts with a single click.
Note: if your repository is in a subgroup, you need to insert the subgroup name in two places above above between /-/ and $CI_PROJECT_NAME for the resulting URLs to work.
It is possible to keep several pages published for different pipelines/branches.
To do that you need to copy your pages content (basically test report, or whatever needs to be published) to specific unique directory in public folder. For example, the name of the directory can be the id of pipeline (CI_PIPELINE_ID). So the path to pages sources would be like public/$CI_PIPELINE_ID/.
Then the whole public folder should be defined as artifacts with specific unique name (here again "$CI_PIPELINE_ID" can be used).
Unique name for artifacts is needed to not override the artifacts with the next pipeline execution (if name is not specified, the default name will be taken https://docs.gitlab.com/ee/ci/yaml/#artifactsname).
Then you can access the published report via the link:
https://yourGitlab/yourNamespace/yourProjectName/{CI_PIPELINE_ID}/index.html
, that means you can access all your saved reports by changing the pipeline id.
My example:
stages:
- publish
cache:
# Required to keep artifacts from old builds, e.g. from master
paths:
- public
pages:
stage: publish
script:
- mkdir -p public/$CI_PIPELINE_ID
- cp target/site/allure-maven-plugin/* public/$CI_PIPELINE_ID/ -R
artifacts:
name: "$CI_PIPELINE_ID"
paths:
- public
expire_in: 5 days
when: always
If you are using managed GitLab (On GitLab.com), then this feature (multiple deployments) is not enabled in the free version (But it is in Premium, Ultimate versions), but if you have self-hosted GitLab then you should enabled this feature by administrator (It's disabled by-default).
Reference:
On self-managed GitLab, by default this feature is not available. To make it available, an administrator can enable the feature flag named pages_multiple_versions_setting. On GitLab.com, this feature is not available. This feature is not ready for production use.
Select the Deploy -> Feature flags on the left side project's menu:

Set the pages_multiple_versions_setting feature flag:

Then you should configure the path prefix within pages.path_prefix scope in your .gitlab-ci.yml file. You can find examples and details on this GitHub page In your case you should extend your pages section with the following part:
pages:
path_prefix: "$DEPLOY_DIR"
Note:
- This feature is not production ready. You can follow the status of the feature in this ticket: https://gitlab.com/gitlab-org/gitlab/-/issues/422145
you can only have one pages site per GitLab project on GitLab.com free tier. Whatever you upload to the public folder in artefacts will override what was previously there hence why you are experiencing this behaviour.
You CAN do this however it is only for certain accounts as referenced here:
Tier: Premium, Ultimate Offering: SaaS, self-managed Status: Experiment
Read in full here.