Tracing or Preventing HTTP Redirects in Golang

In this post we're going to have a look at how to have a close look at HTTP requests processed by Go and how to prevent automatic redirection through 301, 302 or similar redirects. This is useful both for investigating tracking links from twitter, buffer, bit.ly or email marketing, or to make sure your tools to generate them actually work ;) For a demo, check out the Redirect Checker Tool

Note: You're expected to have go 1.7, if you don't some features of the net/http package might be unavailable to you. You can check this by typing go version in your terminal. Sample output: go version go1.7.3 linux/amd64.

Making an HTTP request in Go

// file: http-request.go
package main

import(
  "fmt"
  "net/http"
)

func main(){
  resp, err := http.Get("http://www.jonathanmh.com/")

  if err != nil {
    fmt.Println(err)
  }

  fmt.Println("StatusCode:", resp.StatusCode)
  fmt.Println(resp.Request.URL)
}

Output:

StatusCode: 200
http://jonathanmh.com/

Note that the resp.Request.URL is not the one we defined for our GET request, but that it has changed. That's because I'm redirecting users accessing www.jonathanmh.com to jonathanmh.com. Go does the sane thing and just follows along. In most cases, that's what we want, not so if we want to build a tool like the mighty Redirect Checker Tool! For a case like this, we want to know every step of the way and the status codes for every request.

Making an HTTP request in Go not Follow Redirects

// file: http-nofollow-request.go
package main

import(
  "fmt"
  "net/http"
)

func main(){
  client := &http.Client{
    CheckRedirect: func(req *http.Request, via []*http.Request) error {
      return http.ErrUseLastResponse
  } }

  resp, err := client.Get("http://www.jonathanmh.com")

  if err != nil {
    fmt.Println(err)
  }

  fmt.Println("StatusCode:", resp.StatusCode)
  fmt.Println(resp.Request.URL)
}

Output:

StatusCode: 301
http://www.jonathanmh.com

Great! This time we didn't get the forwarded URL, but only the one we were requesting, including the status code, which is perfect! Well, it's almost perfect. Unfortunately now our program finishes happily after executing the first request and doesn't at all tell us about where it was supposed to be redirected to, which is a bummer.

Looping through HTTP redirects in Go

Let's say we assume that if we get an HTTP status code of 200, we're happy and we'll stop trying to do anything else.

// file: http-redir-loop.go
package main

import(
  "fmt"
  "net/http"
)

func main(){
  myURL := "http://www.jonathanmh.com"
  nextURL := myURL
  var i int
  for i < 100 {
    client := &http.Client{
      CheckRedirect: func(req *http.Request, via []*http.Request) error {
        return http.ErrUseLastResponse
    } }

    resp, err := client.Get(nextURL)

    if err != nil {
      fmt.Println(err)
    }

    fmt.Println("StatusCode:", resp.StatusCode)
    fmt.Println(resp.Request.URL)

    if resp.StatusCode == 200 {
      fmt.Println("Done!")
      break
    } else {
      nextURL = resp.Header.Get("Location")
      i += 1
    }
  }
}

Output:

StatusCode: 301
http://www.jonathanmh.com
StatusCode: 200
http://jonathanmh.com/
Done!

This example is pretty much like the last one, except that we have some variables for the URLs, like myURL and nextURL. At the end of the loop, if the status code is not 200 - OK, we set the nextURL to resp.Header.Get("Location") which is the HTTP header field for redirect targets. You can by the way also look at those headers if you turn to the Network tab of your Chrome or Firefox Developer Tools and tick the box Preserve Log.

That's it! You now know how to do some pretty cool things with net/http in Go. All you need to do now to make it available to the public (if you want to) is to wrap it in an http API, run it on a server and build a small user interface for it, like I've done on the Redirect Checker Tool ;)

Further Reading and Sources

• net/http golang package
• stackoverflow: How Can I Make the Go HTTP Client NOT Follow Redirects Automatically?
• echo golang web framework

Let me know what you think and if this post was useful to you!