I am releasing HTTP Request Translator, a tool for converting HTTP requests between formats, such as from a curl command-line invocation to a Python urllib call. This should be useful for automation, penetration testing, and scraping.

curl 'https://example.com/?q=foo+bar' \
  -XPOST \
  --header 'Content-Type: application/x-www-form-urlencoded'
  --header 'X-Magic-Header: Xyzzy' \
  --data 'key=abscissa'

Becomes:

import urllib.parse
import urllib.request

req = urllib.request.Request(
  'https://example.com/?' + urllib.parse.urlencode({
    'q': 'foo bar',
  }),
  headers={
    'Content-Type': 'application/x-www-form-urlencoded',
    'X-Magic-Header': 'Xyzzy',
  },
  data=urllib.parse.urlencode({
    'key': 'abscissa',
  }).encode(),
  method='POST'
)

with urllib.request.urlopen(req) as response:
    body = response.read().decode()

It works really well with the Safari Web Inspector to quickly generate code for replaying an XMLHTTPRequest:

If you are translating from something other than curl, you can just capture the raw HTTP request (say, with Wireshark) and the tool can parse that as well.

I wouldn’t say that the tool is complete; it doesn’t know about some basic things right now (such as cookies) and I don’t expect that I’ll ever have a need to support less common curl options such as --proxy-header. It would be nice to be able to generate JavaScript that makes XHRs, also. But the code should make it pretty easy to contribute support for new features as future use cases call for.

This was my first project using a modern JavaScript toolchain, in particular React, LibSass, and webpack. As such, I’m sure there are things to be improved.