What to Look For

Five things, in this order.

1. The base URL, and whether a version is part of it.

2. Whether it needs authentication, and of what kind.

3. The rate limit, and what happens when you exceed it.

4. How pagination works, because every API does it differently.

5. What an error looks like, so you can tell one from real data.

Find those five and you can write the client. Skip them and you will discover each one the hard way, usually at page four hundred of a crawl.

Then make one request by hand. Before any code, use curl and look at the actual response.

Half of API debugging turns out to be the discovery that the response is not the shape the documentation implied. Finding that out in ten seconds at a terminal is much better than finding it out inside a program you have already written.

SANKOFA SANDBOX
A sandbox shell. The saved response below is a real shape: a wrapper object, a count that does not match, and records with missing fields.
learner:/work$

Notice numFound says 3 and one record has no year. Both of those are things you want to know before writing a loop over ten thousand records.

Try it → Read response.json with cat. Count how many records have a year and how many do not. Then add what you found to notes.txt.

✦ Undocumented is not the same as forbidden, or as stable

Plenty of useful endpoints are not in the documentation. You may still use them, and you should expect them to change without warning and without anyone owing you notice. If something you rely on is undocumented, write that down next to it.

Mark this lesson complete

Signed in, your progress follows you to every device.

Next: The Shape Around the Data →
← Back