For anyone else looking for this like me: contrary to Sheldores comment to the OP, for me adding '_r' to the colorscale name as described here worked perfectly fine. However, I think this was probably changed in plotly since then. Compare the two following figures:

import plotly.graph_objs as go
import plotly.plotly as py
import numpy as np

y_data = np.random.randn(500)
color_data = np.random.randn(500)


fig1 = go.Figure(go.Scatter(
    y = y_data,
    mode='markers',
    marker=dict(
        size=16,
        color = color_data, #set color equal to a variable
        colorscale='Viridis',
        showscale=True
    )
))

fig2 = go.Figure(go.Scatter(
    y = y_data,
    mode='markers',
    marker=dict(
        size=16,
        color = color_data, #set color equal to a variable
        colorscale='Viridis_r',
        showscale=True
    )
))

fig1.show()
fig2.show()

My plotly version is 4.11.0

Adding _r to the palette name now works (notebook).

import plotly.express as px
gapminder = px.data.gapminder()
fig = px.choropleth(gapminder, locations="iso_alpha", 
                    color="lifeExp", hover_name="country", 
                    animation_frame="year", 
                    range_color=[20,80],  
                    color_continuous_scale='RdYlGn_r')
fig.show()