Member-only story

Stuck on Scraping Realestate.com.au

Renee LIN
3 min readDec 24, 2023

--

https://medium.com/geekculture/web-scraping-cheat-sheet-2021-python-for-web-scraping-cad1540ce21c

I tried to get the property sold quantity from www.realestate.com.au

I used the code below, but I suspect there are anti-scraping measures in place, as the web driver was blocked from opening the same page.

from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
import time

# Initialize the Chrome driver
driver = webdriver.Chrome()

# Go to the webpage
driver.get("https://www.realestate.com.au/australia/")

# Wait for the search bar to be visible and interactable
search_bar = WebDriverWait(driver, 10).until(
EC.visibility_of_element_located((By.ID, "search-bar"))
)

# Enter the search text
search_bar.send_keys("sydney")
search_bar.send_keys(Keys.RETURN)

# Wait for 100 seconds
time.sleep(100)

# Close the browser
driver.quit()

--

--

Renee LIN
Renee LIN

Written by Renee LIN

Passionate about web dev and data analysis. Huge FFXIV fan. Interested in health data now.

No responses yet