On Windows, there’s not much you can do other than rely on the Uninstall or remove apps or programs page in the Windows settings. This is generally dependent on things being handled correctly by the developer though, and you can’t really get around that because of how Windows works.

In general though, most of what gets left behind will be either log files or configuration changes. Log files are harmless other than disk usage (and usually easy enough to find and remove). Configuration changes are trickier to clean up and usually require you to know what changed, though there is software that can help with this if you’re proactive about it (such as regshot).

Outside of config changes and odd files here and there though, it’s unlikely anything not actively malicious will leave behind stuff that will survive a reboot.

You can mitigate all this risk by only installing applications from trusted sources, and to a lesser extent by using a proper package manager (for example, it’s a lot harder for something installed from the Microsoft Store to leave behind anything with a lasting impact on your system than it is some random software you installed off the internet).

On other platforms both of those mitigations (using a real package manager, and only installing from trusted sources) are the norm to the point of people looking at you funny for trying something else. Windows (specifically Windows as used by end-users) is very much the abnormal case here in that the norm is to download software form dozens of different sources and run whatever file you downloaded with minimal verification.

I'm adding this answer because, judging from some of the comments, it is not clear to all that Windows 10 offers two separate interfaces for uninstalling programs, and that editing the registry is generally not required to remove an entry from the programs list. When there is a problem uninstalling an application, the newer "Settings" interface does not currently provide the option to remove the entry. However, the older "Control Panel" interface (which is still easily available in Windows 10) does allow removal of an errant entry.

Yes, strictly speaking, this does not answer the OP's question about where the registry entry is, and about using the newer Settings-based interface, but it may offer a simpler solution that more people (including myself) are looking for when they land on this question, and that may not be obvious on a quick reading of previous answers (even though some do explicitly mention Control Panel). And, it DOES directly answer the Subject Line of the OP's question.

Here are screenshots to emphasize the two methods.

The older Control Panel, "Programs and Features" interface (available from the Windows Start menu, Control Panel, then select Programs and Features) allows removal of the item:

The newer Settings, "Apps & Features" interface DOES NOT offer to remove the item:

If you find same universal or provisioned apps are difficult to remove, try the GRID command in Powershell:

PowerShell Commands to Remove Apps in GridView

Just use Out-Gridview to select which applications you want to remove.

Get-AppxPackage | Out-GridView -Passthru | Remove-AppXPackage

Keep in mind the above only removed the apps for the current user. To remove the apps from the computer for all users, run the following:

Get-AppxProvisionedPackage -Online | Out-GridView -PassThru | Remove-AppxProvisionedPackage -Online

This will display a grid of all installed apps. You can SELECT the apps (highlight in blue) you want to remove from the displayed list and click OK. Reboot.

(I found I could only delete a few apps at a time by repeating the above command and selecting a few each time I reran the command)