This solution is provided in the Accepted Answer on this SO question, so if the below helps, go and upvote that.
Using the command palette (default ctrl+shift+p/cmd+shift+p), there is an option
Terminal: Create New Integrated Terminal (local)

vs code - How to open local terminal when running VSCode in a Dev Container? - Stack Overflow
Is it possible to open and use Developer Command Prompt for VS inside vscode? And if so, how to do it?
is there a VSCode Web with integrated terminal?
How can I get the "code" terminal command to open VSCode for me?
How do I open the terminal in VSCode?
How do I split the terminal in VSCode?
How do I open the VSCode terminal in a specific folder?
This solution is provided in the Accepted Answer on this SO question, so if the below helps, go and upvote that.
Using the command palette (default ctrl+shift+p/cmd+shift+p), there is an option
Terminal: Create New Integrated Terminal (local)

Provided that you have key-based SSH access from the container to the host, you can add a terminal profile to your remote settings (run command Preferences: Open Remote Settings (JSON) (Dev Container) to open them):
Use with AI{
"terminal.integrated.profiles.linux": {
"bash (Host)": {
"path": "ssh",
"icon": "terminal-bash",
"args": [
"user@host",
],
},
},
}
This will get you an according bash (Host) entry in your "Add Terminal" drop-down menu:
