Scraping Restaurant Menus from Postmates.com: Extract Valuable Data for Your Business

1
(No Transcript)
2
Scraping Restaurant Menus from Postmates.com
Extract Valuable Data for Your Business
In today's competitive market, businesses seek
new ways to gain an edge over their rivals. One
of the most effective ways is by analyzing data
and extracting valuable insights. Restaurant
owners, for instance, can use data on customer
preferences and popular menu items to improve
their offerings and attract more customers.
Postmates.com is a popular platform allowing
users to order food from various restaurants. The
website must provide an easy way to access and
analyze the menus of all the restaurants listed
on the platform. This article will explore how to
scrape restaurant menus from Postmates.com using
Python and a few libraries. After reading this
guide, you will have the knowledge and tools to
maximize the value of Postmates for restaurant
data.
3
What is web scraping?
Web scraping is the process of extracting data
from online sources using code. In its basic
form, web scraping consists of HTML requests and
responses. Web scrapers can collect many data
types from a given source, including simple lists
of items or more complex data tables. The scraped
data can then be analyzed to determine
significant trends and insights to help business
owners make more informed decisions regarding
their operations.
About Postmates.com
Postmates is an on-demand delivery platform that
allows users to order food and other goods from
restaurants in their area via an app or website,
with delivery handled by local couriers on bikes,
on foot, or in cars. Postmates is popular with
users and has an impressive number of restaurants
on its platform.
4
(No Transcript)
5
Extract Valuable Data for Your Business
We first need to identify the information we want
to create an algorithm that can extract the
necessary data. The following table lists
valuable data points business owners can use to
improve their operations.
1. Menu Prices
This data helps determine if a restaurant is
offering an opportunity to gain profit by
lowering prices. If the cost of a dish indicates
the menu is overpriced, customers may not return
to that restaurant and may no longer use the
platform. A significant customer decline leads to
less revenue, and less income leads to a
decreased ability to expand your business.
2. Food Offers Quality
The quality of dishes on Postmates varies, with
delicious dishes like steak being much more
common than low-quality ones like fries (see
screenshot). It's easier for businesses of all
sizes and types to compete effectively by
improving the quality of their offerings.
6
3. Menu Popularity
Some food products sell better than others, and
some dishes only appear on the menu once or
twice, as noted by the "Limited Time Offer"
label. Some words may be an excellent opportunity
to gain more customers, while others should be
avoided.
4. Food Delivery Options
Postmates is known for offering convenient
delivery options for customers, including
delivery by bicycle and on foot, in addition to
standard car delivery. Businesses can use this
data to identify opportunities to expand their
reach and attract new customers.
7
Postmates.com Web Scraping
8
After you know the data you want to extract, you
can use Python and a few web scraping libraries
to collect data from a web page. In the following
tutorial, we'll use scrapy and beautiful soup.
Both are available in the Anaconda package
manager, an easy-to-use application simplifying
Python code installation. If you don't have the
Anaconda application installed, follow these
instructions to install it.
Step 1 Install scrapy and beautiful soup
To start, open a terminal window and create a new
directory to store your Python code. Then use a
pip to install scrapy and beautiful soup. mkdir
postmates_scraper cd postmates_scraper pip
install scrapy beautifulsoup4 Next, create an
empty Python module called scrape_postmates.py
where your code will be stored. We'll add our
code in a later step.
Step 2 Create your scrape_postmates.py module
Our objective is to scrape the restaurant menus
from Postmates, so let's create a Scrapy project
of Postmates on our machine and import the scrapy
library. From the same directory where you
installed the Anaconda package manager (pip) gt
scrapy start project postmates_scraper -d web gt
cd postmates_scraper gt Python manage.py load data
restaurants.json
The output confirms that the project was
successfully created and has scraped all the data
from the Postmates website.
9
Step 3 Set up your Postmates username, password,
and API key
Before we start scraping, we need to set up our
credentials, so the scrapy library can connect to
Postmates to scrape their data. The certificates
are stored on jsonplaceholder.typicamp.com, a
free service allowing users to create CRUD web
APIs (create/read/update/delete). Create an
account on this service and fill in your
information so you can access your credentials
via JSON or YAML format.
A. Set up your environment,
Create a text file called postmates_creds.py in
the same directory where scrape_postmates.py is
stored and add the following code username
'postmates_username' password
'postmates_password' APIkey 'postmates_apiKey'
B. Set up your API key with Postmates,
Sign in to your Postmates account, click "Account
Settings," and select "API Keys." You'll then see
a screen with your API credentials You'll need
to go to the "Keys" section to see the key for
each environment (Production, Staging, QA). The
postmates_apiKey variable in your code should be
the ltAPI Keygt field value.
10
C. Set up your environment variables,
Once you have all your credentials, create a new
file called env.txt in the same directory as
scrape_postmates.py and add the following code
export POSTMATES_USERNAMEltusernamegt export
POSTMATES_PASSWORDltpasswordgt export
POSTMATES_TOKENltAPI keygt
Alternatively, using a Mac or Linux machine, you
can add these variables to the .bashrc file. As
soon as these environment variables are set, our
Python code will use them to connect to Postmates
as if it pulls information from it.
Step 4 Add postmates_creds.py to your project
We need to import the configuration file to our
project, so create a new file called
urls_scrape.py (in the same directory as
scrape_postmates.py) and add the following code
from scrape_postmates import.
We need to import the configuration file to our
project, so create a new file called
urls_scrape.py (in the same directory as
scrape_postmates.py) and add the following code
from scrape_postmates import.
11
Next, run the code in Terminal with Python scrapy
urls_scrape.py. You should get output similar to
the following, showing all restaurants and the
available time Scraped 0 restaurants, 0
available.
Step 5 Check the variables for the data you want
In your project, you can use the variable to
check how many restaurants you have scraped and
how much time they are available
gt scrapy crawl postmates_scraper -o
restaurant_times.json
You should see this output Scraped 0
restaurants, 0 available.
Step 6 Extract data from Postmates
To extract data from Postmates, we can use the
BeautifulSoup module in combination with a custom
HTML parser called XML. With these two tools, we
can easily scrape a web page to get its HTML
structure plus some of its attributes.
12
Create another empty file called
parse_postmates.py and add the following code
from the same directory as scrape_postmates.py
import lxml.html as HTML from scrapy.contrib.postm
ates_scraper.
Step 7 Add parse_postmates.py
It is where we use BeautifulSoup to get the
page's structure and extract data from it. Let's
start with saving money on delivery charges
gt from parse_postmates import.
We'll identify some words to gain more customers,
while others are risky, giving us an idea of our
competitors' success.
The code extracted in this tutorial is for all
restaurants that have "delivery" in their name.
This function returns a dictionary containing all
the restaurant data gtgtgt restaurant_times
restaurant_times.get('delivery') gtgtgt
print(restaurant_times) 'serves_alcohol' False,
'delivery_charges' 0.0, 'phone' '',
'checkout_available' True, 'wheelchair_accessible
' False, 'cash_only' False, ...
13
We can expand this function and get the related
restaurant information if we have more than ten
restaurants with "delivery" in their name. Now,
let's use BeautifulSoup to get the restaurant
information
gtgtgt restaurant_times restaurant_times.get(res
taurant_times.has_key('delivery')).next() gtgtgt
print(restaurant_times) 'serves_alcohol' False,
'phone' '', 'checkout_available' True,
'wheelchair_accessible' False
We can scrape other attributes, such as the
address, phone number, and all the necessary
information.
14
Conclusion
You have successfully created and run your web
scraper program. You learned how to set up your
Postmates username, password, and API key. You
also learned how to scrape Postmates' restaurant
data. The next part of this series will explore
how we can create a restaurant crawler program
that scrapes other websites for restaurant
information.
The next part of this series will explore how we
can create a restaurant crawler program that
scrapes other websites for restaurant
information.
15
(No Transcript)