Nested Loaders
Nested loaders are useful when we are parsing values, that are related, from the subsection of a document. Without them, we need to mention the entire XPath or CSS path, of the data we want to extract. Consider, the following HTML footer example –
Python3
# Create loader objectloader = ItemLoader(item=Item())# Item loader method for phoneno,# mention the field name and xpath expressionloader.add_xpath('phoneno', '//footer/a[@class = "phoneno"]/@href')# Item loader method for map,# mention the field name and xpath expressionloader.add_xpath('map', '//footer/a[@class = "map"]/@href')# populate the item loader.load_item() |
Using nested loaders, we can avoid, using the nested footer selector, as follows:
Python3
# Define Item Loader object by passing itemloader = ItemLoader(item=Item())# Create nested loader with footer selectorfooter_loader = loader.nested_xpath('//footer')# Add phoneno xpath values relative to the footerfooter_loader.add_xpath('phoneno', 'a[@class = "phoneno"]/@href')# Add map xpath values relative to the footerfooter_loader.add_xpath('map', 'a[@class = "map"]/@href')# Call loader.load_item() to populate valuesloader.load_item() |
Please note the following points about nested loaders:
- They work with CSS and XPath selectors.
- They can be nested randomly.
- They can make the code look simpler.
- Do not use them needlessly, else the parser can get difficult to read.
Scrapy – Item Loaders
In this article, we are going to discuss Item Loaders in Scrapy.
Scrapy is used for extracting data, using spiders, that crawl through the website. The obtained data can also be processed, in the form, of Scrapy Items. The Item Loaders play a significant role, in parsing the data, before populating the Item fields. In this article, we will learn about Item Loaders.
Contact Us