Skip to main content
Participant
November 10, 2022
Question

Scraping javascript data within a grid of a webpage using selenium and python

  • November 10, 2022
  • 1 reply
  • 258 views

My issue is that I need all the data within the grid containing subdomains from the website paloaltonetworks - (data containing NAME , CATEGORY, SUBCATEGORY, RISK, TECHNOLOGY). What I require is [Example: In line number 5: 2ch has 2 subdomains |_2ch-base and 2ch-posting. Like this I only want to get the list of all apps having subdomains]

Right not whenever I have tried adding anything in the line:

table =wait.until(EC.presence_of_all_elements_located((By.CSS_SELECTOR,    'tbody#bodyScrollingTable tr')))

I am getting a timeout error.

Below is the script I have as of now which fetches all the data from the grid but I need only the apps and it's containing subdomains.[Example 2ch, 2ch-base, 2ch-posting]. I have found out a pattern through inspect element which is all apps that doesn't have subdomains have ( ) or we can go by the () field which is common for all apps having subdomains. Any help on solving this problem will be much appreciated.

from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC 

driver   = webdriver.Chrome(executable_path = r'/Users/am/Downloads/chromedriver')
driver.maximize_window()

driver.get("https://applipedia.paloaltonetworks.com/") 

wait = WebDriverWait(driver,30)

table =wait.until(EC.presence_of_all_elements_located((By.CSS_SELECTOR, 'tbody#bodyScrollingTable tr')))

for tab in table:
  print(tab.text)

 

This topic has been closed for replies.

1 reply

Meenakshi_Negi
Legend
November 30, 2022

Hi veera27057138ups0,

 

Thank you for reaching out, and sorry about the delay in response.

 

Could you please let us know the Adobe service you are using?

We have checked your account using the email address used to sign in here on the community, and it shows no active Adobe service on your Adobe account.

If you use an alternate email address, please share it with us via private message. You may initiate a private message using the envelope icon at the top right corner of this community window.

 

Thanks,

Meenakshi