Try this in the command line:

pip install pip-system-certs

I've been struggling with the CERTIFICATE_VERIFY_FAILED error as well when using the requests module. I tried installing certifi but that didn't help. The only solution to my problem was to install pip-system-certs. For some reason that allowed requests to access a local certificate.

Also: requests.get(url, verify=False) is not recommended for production environment because you basically turn safety off.

It looks like the context needs to be passed into an HTTPSHandler in order to make use of the certifi certificates.

Try patching your local copy of /usr/local/lib/python3.8/dist-packages/hcaptcha_field/fields.py with the following.

To the imports, as you've already done, add:

import certifi
import ssl

In addition, expand the import from urllib.request to include HTTPSHandler:

from urllib.request import build_opener, Request, ProxyHandler, HTTPSHandler

Also as you've already done, create an SSL context that uses the certifi certificates. I would add this right under the import block, though, rather than deep in the module code:

context=ssl.create_default_context(cafile=certifi.where())

Then, immediately below this new context=... line, create a new HTTPSHandler instance using context:

https_handler = HTTPSHandler(context=context)

Then, revise the line where opener is defined using the build_opener(...) call to include the new HTTPSHandler instance:

opener = build_opener(https_handler, ProxyHandler(hcaptcha_settings.PROXIES))

If I'm reading the build_opener docs correctly, the build_opener machinery is already using a default instance of HTTPSHandler as it tries to open the URL. Hopefully, replacing that default instance with this new https_handler instance, which will operate with the SSL context that includes the certifi certificates, will allow the urlopen call to work.

If this does work, then an issue/PR to the upstream project might be a good idea, to integrate this logic and either to provide a default behavior that pulls in the certifi certificates automatically if they're available, or to include a configuration option to let the user wire in the certifi certificates.

UPDATE 2022-04-14: Since this did fix the problem, I'll add some more detail as to why this fix was necessary and how it worked.

HTTPS requires the website you're accessing to provide a certificate that validates its identity. Otherwise, how do you know your traffic hasn't been messed with, man-in-the-middle attacked, etc.? The certificate that a website provides to you when you browse to it is issued to the operators of the website by a certificate issuing authority. This authority serves as a trusted third party, to "vouch for" the website operators and that, "yes, you can trust this certificate from them".

There are several of these issuing authorities out there. Those authorities issue their own certificates ("certificate authority certificates", or "CA certificates"), which are completely separate from the ones that individual websites provide. Your browser (or URL opener) needs to have access to these CA certificates in order to complete the 'trusted third party' verification of the certificate that the website presents. The key error the OP is facing here is the inability of the URL opener to find a good set of CA certificates to use for this third-party validation:

<urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1131)>

The message unable to get local issuer certificate means that urlopen cannot find the needed issuer certificate (CA certificate) on the local system.

So, the solution is to point urlopen to the CA certificates provisioned by the certifi package:

• In order to tell the opener where to find them, you have to provide their location when you create opener from build_opener

• build_opener works based upon a collection of handler objects, so the CA certificate information must be included within a suitable handler

• The only handler (that I could find in the docs) that includes information about the SSL context is the HTTPSHandler

• The way that HTTPSHandler takes in information about SSL-related configuration is through the context keyword argument when creating a new instance, which takes an 'SSL context' object

• ssl.create_default_context is (as best I understand) a factory function for creating SSL context objects, customizeable in various ways

• One of the ways it's customizeable is with the location of a CA certificate store (via the cafile argument)

• certifi provides the location on disk of its curated CA certificate store through the certifi.where() function

If you scan these bullets from bottom to top, they map pretty well to each of the steps in the answer.

The issue I found on my Ubuntu with Python was that I upgraded pip install --upgrade certifi... And then it took me hours trying to understand why things do not work... Only once I had seen different versions here, that I started thinking in the right direction...

vitaliy@vitalnuc01:~/HANAClientsTutorial/python$ dpkg -l | grep python3-certifi
ii  python3-certifi                            2018.1.18-2                                         all          root certificates for validating SSL certs and verifying TLS hosts (python3)
vitaliy@vitalnuc01:~/HANAClientsTutorial/python$ pip3 search certifi
...
certifi (2020.4.5.1)                        - Python package for providing Mozilla's CA Bundle.
  INSTALLED: 2020.4.5.1 (latest)
...

So...

pip3 uninstall certifi
sudo apt-get install --reinstall python3-certifi

...and Python-based programs started working for me again.

And yes, I was close to reinstalling my Ubuntu as well :)

This isn't a solution to your specific problem, but I'm putting it here because this thread is the top Google result for "SSL: CERTIFICATE_VERIFY_FAILED", and it lead me on a wild goose chase.

If you have installed Python 3.6 on OSX and are getting the "SSL: CERTIFICATE_VERIFY_FAILED" error when trying to connect to an https:// site, it's probably because Python 3.6 on OSX has no certificates at all, and can't validate any SSL connections. This is a change for 3.6 on OSX, and requires a post-install step, which installs the certifi package of certificates. This is documented in the file ReadMe.rtf, which you can find at /Applications/Python\ 3.6/ReadMe.rtf (see also the file Conclusion.rtf, and the script build-installer.py that generates the macOS installer).

The ReadMe will have you run the post-install script at

/Applications/Python\ 3.10/Install\ Certificates.command (Terminal App, this command alone should, fix the issue. Be sure to update the file path using your current subversion.)

(its source is install_certificates.command), which:

• first installs the Python package certifi, and
• then creates a symbolic link from the OpenSSL certificates file to the certificates file installed by the package certifi.

Release notes have some more info: https://www.python.org/downloads/release/python-360/

On newer versions of Python, there is more documentation about this:

• https://github.com/python/cpython/blob/e05a703848473b0365886dcc593cbddc46609f29/Mac/BuildScript/resources/ReadMe.rtf#L22-L34
• https://github.com/python/cpython/blob/e05a703848473b0365886dcc593cbddc46609f29/Mac/BuildScript/resources/Conclusion.rtf#L15-L19
• https://github.com/python/cpython/blob/e05a703848473b0365886dcc593cbddc46609f29/Mac/BuildScript/resources/Welcome.rtf#L23-L25
• https://github.com/python/cpython/blob/e05a703848473b0365886dcc593cbddc46609f29/Mac/BuildScript/resources/install_certificates.command
• https://github.com/python/cpython/blob/e05a703848473b0365886dcc593cbddc46609f29/Mac/BuildScript/README.rst
• https://github.com/python/cpython/blob/e05a703848473b0365886dcc593cbddc46609f29/Mac/BuildScript/build-installer.py#L239-L246

TLDR:

hostname=XXX
port=443
trust_cert_file_location=`curl-config --ca`

sudo bash -c "echo -n | openssl s_client -showcerts -connect $hostname:$port -servername $hostname \
    2>/dev/null  | sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p'  \
    >> $trust_cert_file_location"

Warning: as noted in gareththered's excellent answer, this adds all certificates, instead of only the Root CAs.
Blindly adding all (any) certificates to your truststore without due diligence is not the best course of action.

Note: x-yuri adds in the comments:

• Although echo -n (basically EOF) is a valid way to make s_client exit, you can also do echo Q.
• sed doesn't need -e if there's only one script.

Long answer

The basic reason is that your computer doesn't trust the certificate authority that signed the certificate used on the GitLab server. This doesn't mean the certificate is suspicious, but it could be self-signed or signed by an institution/company that isn't in the list of your OS's list of CAs. What you have to do to circumvent the problem on your computer is telling it to trust that certificate — if you don't have any reason to be suspicious about it.

You need to check the web certificate used for your GitLab server, and add it to your </git_installation_folder>/bin/curl-ca-bundle.crt.

To check if at least the clone works without checking said certificate, you can set:

export GIT_SSL_NO_VERIFY=1
#or
git config --global http.sslverify false

But that would be for testing only, as illustrated in "SSL works with browser, wget, and curl, but fails with git", or in this blog post.

Check your GitLab settings.

To get that certificate (that you would need to add to your curl-ca-bundle.crt file), type a:

echo -n | openssl s_client -showcerts -connect yourserver.com:YourHttpsGitlabPort \
  2>/dev/null  | sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p'

(with 'yourserver.com' being your GitLab server name, and YourHttpsGitlabPort is the https port, usually 443)

To check the CA (Certificate Authority issuer), type a:

echo -n | openssl s_client -showcerts -connect yourserver.com:YourHttpsGilabPort \
  2>/dev/null  | sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' \
  | openssl x509 -noout -text | grep "CA Issuers" | head -1

Note: Valeriy Katkov suggests in the comments to add -servername option to the openssl command; otherwise the command isn't showed certificate for www.github.com in Valeriy's case.

openssl s_client -showcerts -servername www.github.com -connect www.github.com:443

Findekano adds in the comments:

to identify the location of curl-ca-bundle.crt, you could use the command

curl-config --ca

Also, see my more recent answer "GitHub: server certificate verification failed": you might have to renistall those certificates:

sudo apt-get install --reinstall ca-certificates
sudo mkdir /usr/local/share/ca-certificates/cacert.org
sudo wget -P /usr/local/share/ca-certificates/cacert.org http://www.cacert.org/certs/root.crt http://www.cacert.org/certs/class3.crt
sudo update-ca-certificates
git config --global http.sslCAinfo /etc/ssl/certs/ca-certificates.crt