This lesson is in the early stages of development (Alpha version)

Introduction to the Web and Online APIs

The Internet and the World-Wide Web have been an integral part of our lives for many years now. Increasingly, however, it is also being used to expose systems that you can interact with programmatically, via web-based Application Programming Interfaces (APIs). Many such APIs offer data that can form the basis of useful data science.

In this lesson, we will introduce enough about how the Web works to be able to take advantage of these APIs, and then we will learn how to interact with web-based APIs.


In this lesson we will make a little use of the Unix shell, and will work more heavily with the Python programming language. If you are unfamiliar with either of these, we would recommend you review the Software Carpentry lessons on the Unix Shell (episodes 1–3 will provide sufficient background) and programming in Python.


Setup Download files required for the lesson
00:00 1. HTTP What are protocols and ports?
What are HTTP and HTTPS?
What are requests and responses? How can we look at them?
00:15 2. What do APIs look like? How can requests be made of web APIs?
How can responses from web APIs arrive?
How can requests to web APIs be authenticated?
00:40 3. dicts What is a Python dict?
How do I use a dict?
00:55 4. Requests How can I send HTTP requests to a web server from Python?
How to interact with web services that require authentication?
What are the data formats that are used in HTTP messages?
01:25 5. Elements of Web Scraping with BeautifulSoup How can I obtain data in a programmatic way from the web without an API?
01:55 Finish

The actual schedule may vary slightly depending on the topics and exercises chosen by the instructor.