Tracing or Preventing HTTP Redirects in Golang
In this post we're going to have a look at how to have a close look at HTTP requests processed by Go and how to prevent automatic redirection through 301
, 302
or similar redirects. This is useful both for investigating tracking links from twitter, buffer, bit.ly or email marketing, or to make sure your tools to generate them actually work ;) For a demo, check out the Redirect Checker Tool
Note: You're expected to have go 1.7
, if you don't some features of the net/http
package might be unavailable to you. You can check this by typing go version
in your terminal. Sample output: go version go1.7.3 linux/amd64
.
Making an HTTP request in Go
Let's have a look at how to create a simple HTTP request in Go and read the status code.// file: http-request.go
package main
import(
"fmt"
"net/http"
)
func main(){
resp, err := http.Get("http://www.jonathanmh.com/")
if err != nil {
fmt.Println(err)
}
fmt.Println("StatusCode:", resp.StatusCode)
fmt.Println(resp.Request.URL)
}
Output:
StatusCode: 200
http://jonathanmh.com/
Note that the resp.Request.URL
is not the one we defined for our GET
request, but that it has changed. That's because I'm redirecting users accessing www.jonathanmh.com
to jonathanmh.com
. Go does the sane thing and just follows along. In most cases, that's what we want, not so if we want to build a tool like the mighty Redirect Checker Tool! For a case like this, we want to know every step of the way and the status codes for every request.
Making an HTTP request in Go not Follow Redirects
In order to make Go NOT follow redirects, we need to create our ownhttp.Client
instance with a custom function for CheckRedirect
. This will enable us to return from the request before it follows a redirect and again give us the HTTP status code and the address it's trying to access.
// file: http-nofollow-request.go
package main
import(
"fmt"
"net/http"
)
func main(){
client := &http.Client{
CheckRedirect: func(req *http.Request, via []*http.Request) error {
return http.ErrUseLastResponse
} }
resp, err := client.Get("http://www.jonathanmh.com")
if err != nil {
fmt.Println(err)
}
fmt.Println("StatusCode:", resp.StatusCode)
fmt.Println(resp.Request.URL)
}
Output:
StatusCode: 301
http://www.jonathanmh.com
Great! This time we didn't get the forwarded URL, but only the one we were requesting, including the status code, which is perfect! Well, it's almost perfect. Unfortunately now our program finishes happily after executing the first request and doesn't at all tell us about where it was supposed to be redirected to, which is a bummer.
Looping through HTTP redirects in Go
Okay, last step, we're doing to have a look at how to loop through redirects with Go. First of all we should set a max limit for request (so nobody can send us in an infinite loop), also we need to create a condition for when we're going to be done.Let's say we assume that if we get an HTTP status code of 200
, we're happy and we'll stop trying to do anything else.
// file: http-redir-loop.go
package main
import(
"fmt"
"net/http"
)
func main(){
myURL := "http://www.jonathanmh.com"
nextURL := myURL
var i int
for i < 100 {
client := &http.Client{
CheckRedirect: func(req *http.Request, via []*http.Request) error {
return http.ErrUseLastResponse
} }
resp, err := client.Get(nextURL)
if err != nil {
fmt.Println(err)
}
fmt.Println("StatusCode:", resp.StatusCode)
fmt.Println(resp.Request.URL)
if resp.StatusCode == 200 {
fmt.Println("Done!")
break
} else {
nextURL = resp.Header.Get("Location")
i += 1
}
}
}
Output:
StatusCode: 301
http://www.jonathanmh.com
StatusCode: 200
http://jonathanmh.com/
Done!
This example is pretty much like the last one, except that we have some variables for the URLs, like myURL
and nextURL
. At the end of the loop, if the status code is not 200 - OK
, we set the nextURL
to resp.Header.Get("Location")
which is the HTTP header field for redirect targets. You can by the way also look at those headers if you turn to the Network tab of your Chrome or Firefox Developer Tools and tick the box Preserve Log
.
That's it! You now know how to do some pretty cool things with net/http
in Go. All you need to do now to make it available to the public (if you want to) is to wrap it in an http API, run it on a server and build a small user interface for it, like I've done on the Redirect Checker Tool ;)
Further Reading and Sources
One blog post can not rule them all, maybe what I explained isn't what you wanted at all so go have a look at the following links:- net/http golang package
- stackoverflow: How Can I Make the Go HTTP Client NOT Follow Redirects Automatically?
- echo golang web framework
Let me know what you think and if this post was useful to you!