After searching for a while, I stumbled into the answer for the py update part, at least.

The py command is an executable included with Windows Python distributions starting with Python v3.3. It is called the "Python Launcher for Windows" in the official Python documentation.

• Note: An official py command is not included with Linux Python distributions, though there is the Py Launcher for Unix available for Linux that can be installed separately and works like the Windows version.

The documentation on the Python Launcher is pretty complete, but to answer my original question, I had set the environment variable PY_PYTHON to 3.7 to declare the default Python version for py to use.

I was able to fix the issue by updating this to 3.8 and restarting any terminal and program that uses a terminal to access py/python. There are other Python environment variables so I updated them all:

UPDATE: I was able to completely remove the PYTHONHOME and PYTHONPATH variables and just use PY_PYTHON to control the default version. You can read more about these variables in the Environment Variables section of the Python Documentation

• set PY_PYTHON to 3.8
• set PYTHONHOME to my new Python38 directory
• for PYTHONPATH added the new python38 and python38/scripts directories.

Note: After updating environment variables, you will need to restart some programs, and others might require either a logout or full reboot (was true of VS Code on Windows).

More info on py command and versions

You can get a list of basic py commands using py --help:

$ py --help
Python Launcher for Windows Version 3.8.150.1013

usage:
C:\WINDOWS\py.exe [launcher-args] [python-args] script [script-args]

Launcher arguments:

-2     : Launch the latest Python 2.x version
-3     : Launch the latest Python 3.x version
-X.Y   : Launch the specified Python version
     The above all default to 64 bit if a matching 64 bit python is present.
-X.Y-32: Launch the specified 32bit Python version
-X-32  : Launch the latest 32bit Python X version
-X.Y-64: Launch the specified 64bit Python version
-X-64  : Launch the latest 64bit Python X version
-0  --list       : List the available pythons
-0p --list-paths : List with paths

The version strings above used to launch a particular version of python are what can be put into the PY_PYTHON environment variable. For example, if you just wanted the latest Python3, then use 3, for a particular Python3, use 3.8, or for the 32-bit version, use 3.8-32, etc.

Every minor version of Python, that is any 3.x and 2.x version, will install side-by-side with other versions on your computer. Only patch versions will upgrade existing installations.

So if you want to keep your installed Python 2.7 around, then just let it and install a new version using the installer. If you want to get rid of Python 2.7, you can uninstall it before or after installing a newer version—there is no difference to this.

Current Python 3 installations come with the py.exe launcher, which by default is installed into the system directory. This makes it available from the PATH, so you can automatically run it from any shell just by using py instead of python as the command. This avoids you having to put the current Python installation into PATH yourself. That way, you can easily have multiple Python installations side-by-side without them interfering with each other. When running, just use py script.py instead of python script.py to use the launcher. You can also specify a version using for example py -3 or py -3.6 to launch a specific version, otherwise the launcher will use the current default (which will usually be the latest 3.x).

Using the launcher, you can also run Python 2 scripts (which are often syntax incompatible to Python 3), if you decide to keep your Python 2.7 installation. Just use py -2 script.py to launch a script.

As for PyPI packages, every Python installation comes with its own folder where modules are installed into. So if you install a new version and you want to use modules you installed for a previous version, you will have to install them first for the new version. Current versions of the installer also offer you to install pip; it’s enabled by default, so you already have pip for every installation. Unless you explicitly add a Python installation to the PATH, you cannot just use pip though. Luckily, you can also simply use the py.exe launcher for this: py -m pip runs pip. So for example to install Beautiful Soup for Python 3.6, you could run py -3.6 -m pip install beautifulsoup4.