Get startedGet started for free

Starting with Start Requests

In the last lesson we learned about setting up the start_requests method within a scrapy spider. Here we have another toy-model spider which doesn't actually scrape anything, but gives you a chance to play with the start_requests method. What we want is for you to start becomming familiar with the arguments you pass into the scrapy.Request call within start_requests.

As before, we have created the function inspect_class to examine what you are yielding in start_requests.

This exercise is part of the course

Web Scraping in Python

View Course

Exercise instructions

  • Fill in the required scrapy object into the class YourSpider needed to create the scrapy spider.
  • Fill in the blank in the yielded scrapy.Request call within the start_requests method so that the URL this spider would start scraping is "https://www.datacamp.com" and would use the parse method (within the YourSpider class) as the method to parse the website.

Hands-on interactive exercise

Have a go at this exercise by completing this sample code.

# Import scrapy library
import scrapy

# Create the spider class
class YourSpider( ____ ):
  name = "your_spider"
  # start_requests method
  def start_requests( self ):
    yield scrapy.Request( ____ )
  # parse method
  def parse( self, response ):
    pass
  
# Inspect Your Class
inspect_class( YourSpider )
Edit and Run Code