You basically have two option.

Easy solution: don't merge, Rebase
Rebase your branch on top of the main branch, resolve conflict, and then merge. As you'll have a straight history, you'll be able to fast-forward to merge and this won't create any merge commit.

git checkout feature
git rebase main
# Resolve conflict if there is
git checkout main
git merge feature

Second option: Rebase -i
You can edit your history after your merge (before you push to a remote). You can manage this with rebase interactive mode.

git checkout main
git merge feature
#You merge and resolve conflict
git rebase -i <sha of the commit before the merge>

Then you'll be taken into an interactive shell with the list of the commits, e.g.:

pick 73c991e Create progress bar module
pick b8a0b83 merge branch feature
pick 2120f47 Add user form
pick 70c55e4 Quiz prototype system

You just need to add squash or s instead of pick:

pick 73c991e Create progress bar module
pick b8a0b83 merge branch feature
s 2120f47 Add user form
pick 70c55e4 Quiz prototype system

This command would squash together b8a0b83 and 2120f47. The next step will be a commit text editor where you have both commit message combined, and it's now up to you to edit them correctly to only keep your original message.

The answer marked correct here did not work for me as after you quit the editor with wq or :q!, the file gets saved and the merge happens.

The alternative, I've found for now is to suspend the vim process by using a Ctrl + Z and then aborting the merge with a git merge --abort and then killing the background process (you'll see the PID when you do the Ctrl + Z)