APIs and Getting Web Data

Strip away the jargon and an API is a menu of requests a service agrees to answer. When you open a weather website, a human reads a page designed for eyes. An API is the same data offered in a form designed for code: you send a precise request to a web address (a URL), and the server sends back clean, structured data - no page layout, no adverts, just the values. Think of a restaurant: the menu (the API documentation) lists exactly what you can order and how to ask; you place an order (a request); the kitchen returns your dish (the response). You do not need to know how the kitchen works, only how to order.

Most design-relevant APIs are web APIs spoken over HTTP - the same protocol your browser uses. The most common action is a GET request, which reads data: give me the weather at this latitude and longitude, geocode this address, look up this material. You often refine the request with parameters - the lat, lon, date bolted onto the URL after a ?. The server answers with two things: a status code telling you how it went (200 means OK; 404 not found; 429 you are asking too often) and a body, usually JSON - the same key-value structure as a Python dictionary. That mapping is the whole reason APIs are so usable from Python: the response is, in effect, a dictionary waiting for you.

You will often hear these called REST APIs and their addresses called endpoints - an endpoint is simply a specific URL for a specific kind of request, one for weather, another for geocoding, another for a material lookup. The vocabulary sounds formal, but for the everyday job of reading data it always reduces to the same three questions: which URL, which parameters, and what does the returned JSON look like. Answer those from the documentation and the code almost writes itself.

requests (pip install requests) is the friendly standard for talking to web APIs. The core is requests.get(url), which returns a response object carrying the status code and the body. A minimal call against a real, key-free API - Open-Meteo for weather:

import requests

url = "https://api.open-meteo.com/v1/forecast"
params = {"latitude": 12.97, "longitude": 77.59,
          "current": "temperature_2m"}
r = requests.get(url, params=params, timeout=10)
r.raise_for_status()
data = r.json()
print(data["current"]["temperature_2m"], "deg C")

Each line earns its place. Passing params as a dictionary lets requests build the ?latitude=12.97&longitude=77.59&... query string for you, correctly encoded - never hand-glue URLs. timeout=10 caps how long to wait so a slow server cannot hang your script forever. r.raise_for_status() turns a failure code (404, 500) into a clear Python error instead of letting a broken response slip silently downstream. r.json() parses the JSON body into Python objects - dictionaries and lists - so data["current"]["temperature_2m"] reaches into the structure exactly as you index any dictionary. That is the entire mechanic of pulling web data: build a request, check it, parse it, index it. Everything else is reading the particular API's documentation to learn its URL, its parameters, and the shape of the JSON it returns.

It is worth pausing on the response object r, because you will lean on it constantly. Beyond r.json(), r.status_code is the numeric result to compare against 200, r.ok is a quick boolean for success, and r.url shows the exact address requests built from your parameters - printing it is the fastest way to confirm your query went out as you intended. When a call misbehaves, these three attributes usually reveal why long before you need anything more elaborate, so reach for them first whenever a request surprises you.

The response body is JSON - JavaScript Object Notation - a text format of nested objects ({}, which become Python dicts) and arrays ([], which become Python lists). Once r.json() parses it, navigating is exactly the dict-and-list indexing from Module 2, just deeper. A geocoding response - turning a place name into coordinates - shows the nesting:

import requests

r = requests.get("https://geocoding-api.open-meteo.com/v1/search",
                 params={"name": "Jaipur", "count": 1}, timeout=10)
result = r.json()["results"][0]
lat = result["latitude"]
lon = result["longitude"]
print(f"Jaipur is at {lat}, {lon}")

Walk the path: r.json() is a dict; ["results"] is a list of matches; [0] takes the first; ["latitude"] reads a field from it. Reading the API's documentation - or simply print(r.json()) once to see the shape - tells you that path. Real data is messy, so code defensively: a place might have no matches ("results" missing or empty), a field might be absent. Reach in with .get() and check before indexing:

results = r.json().get("results", [])
if results:
    first = results[0]
    lat = first.get("latitude")
else:
    print("No match found")

This is the same defensive instinct as guarding against blank spreadsheet cells - assume the data can surprise you, and handle the empty case rather than crashing on it. A practical tip for exploring an unfamiliar response: import json; print(json.dumps(r.json(), indent=2)) pretty-prints the whole structure with clear indentation, so you can see exactly where every value sits before writing a single line of parsing. Now chain the two: geocode an address to coordinates, then feed those into the weather call - a two-request pipeline that turns 'what is the climate at this site?' into a few lines, live.

Two practical realities separate a toy call from a dependable script. First, authentication: many APIs require a key - a personal token that identifies you - which you register for and then pass with each request, often as a parameter or a header. Treat a key like a password: never paste it into code you share or commit to git. Read it from an environment variable instead, so the secret stays out of the source:

import os, requests

key = os.environ["WEATHER_API_KEY"]
r = requests.get(url, params={"q": "Delhi", "appid": key}, timeout=10)

Second, etiquette. An API is someone else's server, and abusing it gets you blocked. The rules are simple courtesy. Respect rate limits - the cap on requests per minute or day, stated in the docs; a 429 status means slow down. If you are fetching for many sites in a loop, add a small time.sleep(1) between calls so you do not hammer the server. Cache results you will reuse: geocoding the same address a hundred times is a hundred needless requests - fetch once, save the JSON to a file, and read it back. Always set a timeout, and handle failure so one bad response does not sink a long run:

import time, requests

for city in cities:
    try:
        r = requests.get(url, params={"name": city}, timeout=10)
        r.raise_for_status()
        handle(r.json())
    except requests.RequestException as e:
        print(f"skipped {city}: {e}")
    time.sleep(1)

Finally, know the line between an API and web scraping. An API is data offered for programs, with terms you should honour; scraping - parsing a human web page - is a last resort when no API exists, and it comes with legal and ethical strings and brittle code that breaks when the page changes. Prefer the front door. Used politely, APIs turn the live web into just another input to your design scripts.

Reading data with GET covers the overwhelming majority of what a designer needs, but it helps to know the shape of the rest. Alongside GET, APIs use verbs like POST (send data to create something on the server) among others; you will rarely need them for pulling design data, but you will recognise them in the documentation. Some services want their key or options in headers rather than URL parameters - requests takes a headers={...} dictionary for exactly that. And the response object carries far more than JSON: r.status_code, r.headers and r.text (the raw body) are all there when you need to debug why a call behaved oddly - printing them is the first thing to do when a request surprises you.

The real power comes from joining web data to the rest of your toolkit. A parsed response is an ordinary Python dictionary, so you can write it to disk with the json module, load it into pandas for analysis, drop the numbers into an openpyxl spreadsheet, or feed coordinates straight into a geometry script. Saving the fetched data locally is not only politeness - it makes your analysis reproducible: the snapshot is stored with the project, so re-running the study months later gives the same result even if the live service has since changed its numbers.

import json, requests

r = requests.get(url, params=params, timeout=10)
r.raise_for_status()
data = r.json()
with open("site_weather.json", "w") as f:
    json.dump(data, f, indent=2)

That one habit - fetch once, save the JSON, then work from the file - keeps you polite to the server, fast on every re-run, and honest about exactly which data a design decision rested on. It is the natural bridge back to the file, spreadsheet and document tools of this module: the web becomes just another source feeding the same dependable pipelines.