I'm trying to make web scraping with Python 3.10, and the library requests-HTML 0.10.0.
I attach the code:
from requests_html import HTMLSession
url = '
https://bodysolid-europe.com/collections/all'
/>
s = HTMLSession()
r = s.get(url)
r.html.render(sleep=1)
products = r.html.xpath('/html/body/div[2]/div[2]/div', first=True)
for item in products.absolute_links:
r = s.get(item)
print(r.html.find('header.product-header', first=True).text)
When I try to extract information from the URL by Xpath, in the console shows the next output:
[D:urllib3.connectionpool] Starting new HTTPS connection (%d): %s:%s
[D:urllib3.connectionpool] %s://%s:%s "%s %s %s" %s %s
[D:asyncio] Using proactor: %s
[D:websockets.client] = connection is CONNECTING
[D:websockets.client] > GET %s HTTP/1.1
[D:websockets.client] > %s: %s
[D:websockets.client] > %s: %s
[D:websockets.client] > %s: %s
[D:websockets.client] > %s: %s
[D:websockets.client] > %s: %s
[D:websockets.client] > %s: %s
[D:websockets.client] > %s: %s
[D:websockets.client] < HTTP/1.1 %d %s
It doesn't show all the information from the items, only a little bit, like these:
[D:urllib3.connectionpool] %s://%s:%s "%s %s %s" %s %s
[D:urllib3.connectionpool] %s://%s:%s "%s %s %s" %s %s
Body-Solid Europe
Best Fitness Dumbbell Rack BFDR10
[D:urllib3.connectionpool] %s://%s:%s "%s %s %s" %s %s
[D:urllib3.connectionpool] %s://%s:%s "%s %s %s" %s %s
Best Fitness
Best Fitness Bench BFFID10
[D:urllib3.connectionpool] %s://%s:%s "%s %s %s" %s %s
[D:urllib3.connectionpool] %s://%s:%s "%s %s %s" %s %s
Best Fitness
Best Fitness Mountain Climber BFMC10
[D:urllib3.connectionpool] %s://%s:%s "%s %s %s" %s %s
[D:urllib3.connectionpool] %s://%s:%s "%s %s %s" %s %s
Body-Solid Europe
Best Fitness Multi-Station Gym BFMG30
[D:urllib3.connectionpool] %s://%s:%s "%s %s %s" %s %s
[D:urllib3.connectionpool] %s://%s:%s "%s %s %s" %s %s
Best Fitness
Best Fitness Center Drive Elliptical BFE1
[D:urllib3.connectionpool] %s://%s:%s "%s %s %s" %s %s
[D:urllib3.connectionpool] %s://%s:%s "%s %s %s" %s %s
Best Fitness
Best Fitness Olympic Bench BFOB10
[D:urllib3.connectionpool] %s://%s:%s "%s %s %s" %s %s
[D:urllib3.connectionpool] %s://%s:%s "%s %s %s" %s %s
Best Fitness
Best Fitness Functional Trainer BFFT10
[D:urllib3.connectionpool] %s://%s:%s "%s %s %s" %s %s
[D:urllib3.connectionpool] %s://%s:%s "%s %s %s" %s %s
Best Fitness
Best Fitness Leg Developer and Preacher Curl Attachment BFPL10
[D:urllib3.connectionpool] %s://%s:%s "%s %s %s" %s %s
[D:urllib3.connectionpool] %s://%s:%s "%s %s %s" %s %s
Best Fitness
Best Fitness Inversion Table BFINVER10
[D:urllib3.connectionpool] %s://%s:%s "%s %s %s" %s %s
[D:urllib3.connectionpool] %s://%s:%s "%s %s %s" %s %s
Body-Solid Europe
The most of the output are only:
D:websockets.client] < %s
[D:websockets.client] < %s
[D:websockets.client] < %s
[D:websockets.client] < %s
[D:websockets.client] < %s
[D:websockets.client] < %s
[D:websockets.client] < %s
[D:websockets.client] < %s
[D:websockets.client] < %s
[D:websockets.client] < %s
[D:websockets.client] < %s
[D:websockets.client] < %s
[D:websockets.client] < %s
[D:websockets.client] < %s
[D:websockets.client] < %s
[D:websockets.client] < %s
[D:websockets.client] < %s
[D:websockets.client] < %s
[D:websockets.client] < %s
[D:websockets.client] < %s
[D:websockets.client] < %s
I don't know what the problem is. I already installed the pyppeteer==1.0.0, because previously, I had this:
NoSuchKey. The specified key does not exist. No such object: chromium-browser-snapshots/Win_x64/1181205/chrome-win.zip
but now it's showing the "[D:websockets.client] < %s
[D:websockets.client] < %s"
I need to fix that error with the output to get the information from the URL by web scraping.
Python Web scraping [D:websockets.client] > GET %s HTTP/1.1 [D:websockets.client] > %s: %s doesn't show all the results
Programing Coderfunda
September 06, 2024
No comments
Related Posts:
Generate and Extract Random Data with RandoPHPRandoPHP is a package by Roberto Butti that implements random generators (Integer, Char, Byte, etc.) and takes random samples from arrays. This pack… Read More
Manage Job Workflows with Venture Venture is a package to help you build and manage complex, asynchronous workflows in your Laravel applications. Here’s an example from the doc… Read More
Run Artisan Commands With a GUI Laravel Artisan GUI is a package by Bakhtiyar Issakhmetov for running Artisan command from a GUI instead of the command line. By default, you … Read More
Laravel Envoyer API Package Laravel Envoyer SDK is a community package by Steve McDougall to work with the Laravel Envoyer API. Laravel released an API for Envoyer on Novem… Read More
PHP Progress Library to Track User Progress Progress is a PHP package to determine steps and progress with an expressive, object-oriented API: This progress package is a simple way of tak… Read More
0 comments:
Post a Comment
Thanks