Scrapy Syntax and Parameters
Scrapy follows a particular syntax in which it accepts quite a number of parameters mentioned and explained below,
Syntax: yield scrapy.Request(url, callback, method=’GET’, headers=None, body=None)
Parameter:
As you can see the above code line is a typical Scrapy Request with its parameters which are,
- `url`: This is the URL of the website you want to scrape.
- `callback`: The method or handler that will process the response received
- `method`: This is an optional parameter that represents the HTTP method to be used.
- `headers`: Again an optional parameter this includes the header in the request.
- `body`: This refers to the request body mostly used while performing a POST request.
Scrapy – Requests and Responses
In this article, we will explore the Request and Response-ability of Scrapy through a demonstration in which we will scrape some data from a website using Scrapy request and process that scraped data from Scrapy response.
Contact Us