Some time ago, I decided that I wanted to learn Python. I was already quite experienced with Perl and PHP, and I wanted to add the third P to my repertoire.
So I decided to start writing all my personal code at home in Python, and overall I have quite enjoyed this decision. The majority of my personal software is now written in Python and I could ramble about why I enjoy working with it. I decided to push myself some more though, so I started writing data parsers in Python and now I want to start working with web APIs using it as well. Which brings me to “httplib” vs “requests”…
I can see why one of the first lines on the “httplib” documentation page is: “The Requests package is recommended for a higher-level http client interface” in Python (https://docs.python.org/2/library/httplib.html).
I read the examples in the docs, changed the code to match my needs, and ran the code. However, the responses to my requests were always “500 Internal Server Error”. Now… I was pretty sure the remote server wasn’t having a problem. I tried a different tool I’d written in Node.js and it worked fine. I tried a few other tests, and they worked fine. There was something wrong with the “POST” that I was trying…
I was getting a bit frustrated, and wanted a quick win, so I used “pip” to install “requests” and “requests[security”. I typed out a few lines and sure enough… I received a response of “200”. Beautiful!
But… I’m an insatiable problem solver. I wanted to know WHY “httplib” wasn’t working for me. I also didn’t really want to add a non-core Python dependency.
So I looked at the example code again… compared the code that differed between “httplib” and “requests”, and realized… it had to be the data I was sending in the body of the request. Or rather… it had to be a missing header describing the content that I was sending!
Sure enough, when I added “Content-type”: “application/x-www-form-urlencoded” to the headers I was sending, I met with success!
I don’t think I’ve ever had to do that with any library I’ve ever used before but apparently you need to be that explicit when using “httplib”. “requests” must take care of that under the hood.
Anyway, I just thought that I’d share that struggle. If I had just read the documentation for “httplib” completely clearly, it would have been obvious. I feel rather foolish for making that mistake, but… we learn by making mistakes! And hopefully this post helps out someone else out there.
It can be difficult learning a new language, especially when you’re doing it for fun. I am completely confident coding in Perl, so it was tempting to do this project in Perl. I probably would have been finished by now, because I could’ve just typed like the wind. But it’s about the process.
I know Perl, because I have years of paid and unpaid experience with Perl. I’ve used it a million different ways; I’ve made mistakes and found solutions. The same could be said for PHP, Javascript, and Bash/shell, as I’ve had to use those for countless hours at my day job.
But I don’t use Python at my day job. That means the hours I can invest in it are significantly fewer. It means it’s more of a challenge, but I think it’s a worthwhile challenge. It stimulates my brain, it’s fun to learn new things, and who knows when I might end up on a new project or a new job where I need to know Python?
Anyway, we’re getting into ramble territory. While I’ve figured out my problem with “httplib”, I might stick with “requests” just because it’s a bit more convenient. Additional dependencies be damned!
Libraries && Technology 