As you've mentioned
The class name has spaces in it, which lead me to use the css_selector
this is right approach, however you should also make sure that one
- One should remove the space and put a
. .represent class in CSS.
So the below code should work:
driver.find_element(By.CSS_SELECTOR, ".btn.btn-alt.see-full-list-btn")
or you can even use it with the tag a
driver.find_element(By.CSS_SELECTOR, "a.btn.btn-alt.see-full-list-btn")
or the recommended solution would be to use with explicit waits:
see_full_list_button = WebDriverWait(driver, 10).until(EC.visibility_of_element_located((By.CSS_SELECTOR, "a.btn.btn-alt.see-full-list-btn")))
see_full_list_button.click()
Imports:
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
Css Selector button click with selenium (python) - Stack Overflow
architecture - Hierachy and API design for a CSS-selector-related Python library - Software Engineering Stack Exchange
python - How to located selenium element by css selector - Stack Overflow
Python: CSS Selector to use inside lxml.cssselect - Stack Overflow
Videos
As you've mentioned
The class name has spaces in it, which lead me to use the css_selector
this is right approach, however you should also make sure that one
- One should remove the space and put a
. .represent class in CSS.
So the below code should work:
driver.find_element(By.CSS_SELECTOR, ".btn.btn-alt.see-full-list-btn")
or you can even use it with the tag a
driver.find_element(By.CSS_SELECTOR, "a.btn.btn-alt.see-full-list-btn")
or the recommended solution would be to use with explicit waits:
see_full_list_button = WebDriverWait(driver, 10).until(EC.visibility_of_element_located((By.CSS_SELECTOR, "a.btn.btn-alt.see-full-list-btn")))
see_full_list_button.click()
Imports:
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
There is no necessity to focus on the element HTML after the click is already invoked.
As per the HTML
<a href="#" class="btn btn-alt see-full-list-btn">See Full List</a>
you can use either of the following locator strategies:
Using link_text:
driver.find_element(By.LINK_TEXT, "See Full List").click()Using css_selector:
driver.find_element(By.CSS_SELECTOR, "a.btn.btn-alt.see-full-list-btn").click()Using xpath:
driver.find_element(By.XPATH, "//a[@class='btn btn-alt see-full-list-btn' and text()='See Full List']").click()
Ideally to click on the clickable element you need to induce WebDriverWait for the element_to_be_clickable() and you can use either of the following locator strategies:
Using LINK_TEXT:
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.LINK_TEXT, "See Full List"))).click()Using CSS_SELECTOR:
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "a.btn.btn-alt.see-full-list-btn"))).click()Using XPATH:
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//a[@class='btn btn-alt see-full-list-btn' and text()='See Full List']"))).click()Note: You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait from selenium.webdriver.common.by import By from selenium.webdriver.support import expected_conditions as EC
» pip install cssselect
Because Python has very different syntax and semantics from CSS selectors, I think these problems will only get worse. You'll end up with something that doesn't look like CSS does and something that doesn't work like Python usually does. Therefore I would like to propose a different way of approaching the syntax.
CSS selectors are mostly a linear combination of simple selectors and combinators. I would suggest using that, and representing something like ns|p a:link as something like Tag('p', namespace='ns') + Descendant() + Tag('a') + PseudoClass('Link').
That is, you only use a single magic method to represent concatenation. Everything else is just regular Python objects, using regular Python constructors.
Your example could be
# lorem|foo.bar[baz^="qux"]:has(:invalid)::first-line
selector = Tag('foo', namespace='lorem) + \
Class('bar') + \
Attribute('baz', '^=', 'qux') + \
PseudoClass('has', Selector.SELF + Descendant() + PseudoClass('invalid')) + \
PseudoElement('first-line')
It may not be exactly what you were looking for, but it has the advantage that it is much easier to learn for Python users because it has much fewer rules and exceptions, and you don't need to worry about new selectors or incompatible syntax.
You can also use & instead of +, in which case you can represent a selector list with |, for example: p.warning, #bigwarning can become Tag('p') & Class('warning') | ID('bigwarning').
An alternative idea is to use no magic at all, and represent compound and/or complex selectors using lists or wrapper objects.
foo.bar > a might be something like Child([Tag('foo'), Class('Bar')], [Tag('a')]) (compound selectors are lists, complex selectors are wrapped by combinators) or [Tag('foo'), Class('bar'), Child(), Tag('a')] (complex selectors are lists containing the selectors).
The best option depends on ergonomics, and the ergonomics depend on how users will build and manipulate selectors and for what purpose.
You want to represent CSS concepts using valid python syntax which "looks like" the CSS source text.
Simplest approach would be stick with straight CSS source text,
which we can roundtrip through deserializers and serializers.
Representing punctuation-heavy CSS as python source
will be inherently lossy, so you're going to have to
store the details somewhere, perhaps in a global dict
or in various """docstrings""".
It would be worthwhile to explicitly write down your various goals and tradeoffs. For example, getting IDE navigation / autocompletion "for free" might be one of the things you find attractive about your proposed scheme.
Python notation has been exploited for representing SI units, algebra, vector math, and pathnames. The notation is already a good fit for these domains, in some cases because the language strove to be a good fit. So lossless representation can often be achieved.
There are two mature problem domains that you might wish to take inspiration from.
sqlalchemy
The SQLAlchemy community uses at least one python DSL, arguably multiple ones, to represent SQL operations.
The impedance match is not perfect. Operator
precedence
is a bit of a rough edge, with a OR b turning
into (a) | (b) when the two terms are complex.
For some operators, such as IN, we resort to
.in_() method call notation despite the
in
keyword seeming to be available.
Table or column names in principle can incorporate SPACE
and many other characters, especially when "`"
or other quoting mechanisms are used.
But in practice DBAs will often choose to adhere
to a conservative regex such as r("^\w+$").
Your approach might offer enough advantages that
web designers would choose to adhere to conservative
naming conventions, so e.g. "a-b" --> a_b --> "a-b"
could be safely round-tripped.
SQL JOINs are commonly more than a hundred lines long, and a great many production queries have been recast to fit within this DSL.
type annotation
Type hinting continues to be something of a moving target in the python community. An application's source code might be read by an "old" or "new" interpreter, or type checker.
Expressing types in a back-compatible way for old interpreters or checkers has been a source of tension, often relieved via a string annotation escape valve. Forward references sometimes raise challenges that are resolved in the same way. In recent years we've had less need for this escape valve.
We see annotations appearing in the AST, and also in comment text.
The experiences of the type annotation community seem most relevant to your CSS goals.
Your goals are still a bit nebulous at this point. Several developer communities have traveled down this road, showing what works well, or poorly, or would work better after adopting some PEPs. You may be able to draw inspiration, learn from mistakes, and better predict a path to success by looking back at this history and incorporating some elements in your project goals.
I believe you cannot get the attribute value through CSS selectors. You should get the elements...
>>> elements = doc.cssselect('div.results dl dt a')
...and then get the attributes from them:
>>> for element in elements:
... print element.get('href')
...
/link 1
/link 2
Of course, list comprehensions are your friends:
>>> [element.get('href') for element in elements]
['/link 1', '/link 2']
Since you cannot update properties of attributes in CSS, I believe there is no sense on getting them through CSS selectors. You can "mention" attributes in CSS selectors to retrieve only to match their elements. However, is is just cogitation and I may be wrong; if I am, please someone correct me :) Well, @Tim Diggs confirms my hypothesis below :)
You need to get the attribute on the result of cssselect (it always returns the element, never an attribute):
firstly, I'm not sure about doc.cssselect (but maybe this is your own function?)
lxml.cssselect is normally used:
from lxml.cssselect import CSSSelector
sel = CSSSelector('html body div.results dl dt a[href]')
then, assuming you've already got a doc
links = []
for a_href in sel(doc):
links.append(a_href.get('href'))
or the more succinct:
links = [a_href.get('href') for a_href in doc.cssselect('html body div.results dl dt a[href]')]