You should use class and/or id to make shorter xpath.
When you find cards then you can use every card with xpath which starts with ./ - so it will be xpath relative to this element and it will search only inside this element.
You can also use // in any part of xpath to skip some tags which are not important.
You can use other find_element_by_ and find_elements_by_ with card and it will also search only inside this element - so it will be relative.
import selenium.webdriver
driver = selenium.webdriver.Chrome() # Firefox()
driver.get('https://travel.padi.com/s/liveaboards/caribbean/')
all_cards = driver.find_elements_by_xpath('//div[@class="boat search-page-item-card "]')
for card in all_cards:
title = card.find_element_by_xpath('.//a[@class="shop-title"]/span')
desc = card.find_element_by_xpath('.//p[@class="shop-desc-text"]')
price = card.find_element_by_xpath('.//p[@class="cur-price"]/strong/span')
print('title:', title.text)
print('desc:', desc.text)
print('price:', price.text)
all_dates = card.find_elements_by_css_selector('.cell.date')
for date in all_dates:
day, month = date.find_elements_by_tag_name('span')
print('date:', day.text, month.text)
print('---')
Example result (you can have price in different currency)
title: CARIBBEAN EXPLORER II
desc: With incredible, off-the-beaten path itineraries that take guests to St Kitts, Saba and St Maarten, this leading liveaboard spoils divers with five dives each day, scenic geography and a unique slice of Caribbean culture.
Dates do not match your search criteria
price: PLN 824
date: 7 DEC
date: 14 DEC
date: 21 DEC
date: 28 DEC
---
title: BAHAMAS AGGRESSOR
desc: Featuring five dives a day, the well-regarded Bahamas Aggressor liveaboard is the ideal choice for divers who want to spend as much time under the water as possible then relax in an onboard Jacuzzi.
Dates do not match your search criteria
price: PLN 998
date: 7 DEC
date: 14 DEC
date: 21 DEC
date: 28 DEC
---
Answer from furas on Stack Overflowselenium in python finding an element via relative xpath - Stack Overflow
python - Unable to find element by xpath using selenium - Stack Overflow
Can't get element by XPATH
Using xpath to find element does not always work
What is XPath?
How to find the XPath by Text?
How to find the XPath Value?
Videos
You should use class and/or id to make shorter xpath.
When you find cards then you can use every card with xpath which starts with ./ - so it will be xpath relative to this element and it will search only inside this element.
You can also use // in any part of xpath to skip some tags which are not important.
You can use other find_element_by_ and find_elements_by_ with card and it will also search only inside this element - so it will be relative.
import selenium.webdriver
driver = selenium.webdriver.Chrome() # Firefox()
driver.get('https://travel.padi.com/s/liveaboards/caribbean/')
all_cards = driver.find_elements_by_xpath('//div[@class="boat search-page-item-card "]')
for card in all_cards:
title = card.find_element_by_xpath('.//a[@class="shop-title"]/span')
desc = card.find_element_by_xpath('.//p[@class="shop-desc-text"]')
price = card.find_element_by_xpath('.//p[@class="cur-price"]/strong/span')
print('title:', title.text)
print('desc:', desc.text)
print('price:', price.text)
all_dates = card.find_elements_by_css_selector('.cell.date')
for date in all_dates:
day, month = date.find_elements_by_tag_name('span')
print('date:', day.text, month.text)
print('---')
Example result (you can have price in different currency)
title: CARIBBEAN EXPLORER II
desc: With incredible, off-the-beaten path itineraries that take guests to St Kitts, Saba and St Maarten, this leading liveaboard spoils divers with five dives each day, scenic geography and a unique slice of Caribbean culture.
Dates do not match your search criteria
price: PLN 824
date: 7 DEC
date: 14 DEC
date: 21 DEC
date: 28 DEC
---
title: BAHAMAS AGGRESSOR
desc: Featuring five dives a day, the well-regarded Bahamas Aggressor liveaboard is the ideal choice for divers who want to spend as much time under the water as possible then relax in an onboard Jacuzzi.
Dates do not match your search criteria
price: PLN 998
date: 7 DEC
date: 14 DEC
date: 21 DEC
date: 28 DEC
---
You need to use classes in items ./
I Simply Code For You You Can Try!
from selenium import webdriver
import pdb
browser = webdriver.Chrome()
browser.get('https://travel.padi.com/s/liveaboards/caribbean/')
items = browser.find_elements_by_xpath('//div[@class="boat-info"]')
for item in items :
title = item.find_element_by_xpath('.//a[@class="shop-title"]/span')
description = item.find_element_by_xpath('.//p[@class="shop-desc-text"]')
price = item.find_element_by_xpath('.//p[@class="cur-price"]/strong/span')
print('TITLE: ', title.text)
print('DESCRIPTION: ', description.text)
print('PRICE: ', price.text)
print('------------------NEW-RECORD------------------------')
If you have downloaded the Selenium recently (I mean, not a year ago or so), the you are using the latest version of Selenium (Selenium 4), and there is a change in Selenium 4 where you use a By class to get to the locator strategy.
from selenium.webdriver.common.by import By
driver.find_element(By.ID, "element path")
driver.find_element(By.XPATH, "element path")
driver.find_element(By.CSS_SELECTOR, "element path")
driver.find_element(By.CLASS_NAME, "element path")
etc.
Same goes for find_elements
driver.find_elements(By.ID, "element path")
driver.find_elements(By.XPATH, "element path")
driver.find_elements(By.CSS_SELECTOR, "element path")
driver.find_elements(By.CLASS_NAME, "element path")
The key here is to import the By class and use it.
find_element_by_xpath and
find_elements_by_xpath etc are deprecated in Selenium 4.
In the latest Selenium versions driver.find_element_by_xpath method was changed by driver.find_elements(By.XPATH,.
So, instead of the old syntax like
driver.find_element_by_xpath("//*[@id='Username']")
Now you should use the new syntax like
driver.find_element(By.XPATH, "//*[@id='Username']")
The same about all the other old find_element_by_* and find_elements_by_* methods.
Hello guys,
I'm trying to interact with the interative menu of this page in order to automate several data downloads instead of making it by hand.
The thing is that when I copy an XPATH of a selector (for example when I try to get the XPATH of the "Commodities" menu), selenium says:
Message: no such element: Unable to locate element: {"method":"xpath","selector":"/html/body/div[9]/div[1]/div[3]/ul/li[6]"}
Does anyone know why I can't get the element?
Thank you all in advance!
EDIT WITH SOLUTION:
The problem was that items I want to find are inside an iframe. So I have to switch the context of the webdriver first. Code with the solution:
from selenium import webdriver
from selenium.webdriver.common.by import By
driver = webdriver.Chrome()
driver.get('https://www.dukascopy.com/swiss/english/marketwatch/historical/')
table = driver.find_element(By.XPATH, '/html/body/div/main/div[2]/div/div/div/p[3]/iframe')
driver.switch_to.frame(table)
driver.find_element(By.XPATH, '/html/body/div[9]/div[1]/div[3]/ul/li[13]').click()I am using python 3.7 in jupyter notebook. It works just as it should until it doesn't. It says that it cannot find the element and then it never can find it again. I have restarted kernel and that did not solve it. I am just using xpaths from the homepage of pyhton.org.
edit:
To get the xpath I am coping it from the inspect element in chrome. When I compare the error xpath and the one from the webpage they are an exact match.