HTTP Middleware Written in Go

Middleware, It’s Everywhere

The PHP community has recently formalized the representation of HTTP messages in PSR-7. The hope is that agreeing on common interfaces for representing HTTP messages will lead to more interoperability between PHP frameworks. As a result, web development will be more like composing pieces of re-usable middleware into an application. This will be good for the PHP community. Some frameworks like the popular Slim micro framework are already providing PSR-7 support. If you use the Slim, you’ve probably noticed some changes already.

I was first exposed to middleware when writing a prototype using Express a few years ago. The language and framework were different but the principle around middleware stays the same. You have a function that has access to a web request and response, you do something with these and optionally do more in the next piece of middleware.

What’s nice about middleware is that it can be shared across applications. What kinds of tasks can you do in middleware ? Logging requests, opening a sharable database connection, authentication and authorization checks can all be done in middleware. A lot of languages/frameworks implement HTTP middleware so the approach is good to learn and stays familiar as you jump from framework to framework.

In this post, I’m going to explain how I’m currently writing middleware in Go, as well as provide a complete, start to finish example. Some tutorials that you find on-line only highlight certain portions of the code. They leave you with the task of filling in the blanks. While this is useful. A working, start to finish example, that you can download and start fiddling with, is what I hope to provide with this post. Please take what is written here with a grain of salt. There may be better approaches to doing this. I’m still learning Go. If there is a better approach, please leave it in the comments section below.

Middleware in Go

Let’s first look at an excerpt of our example project’s main function :

func main() {
	//Setup the logger
	logger.InitLogger(ioutil.Discard, os.Stdout, os.Stdout, os.Stderr)

	//Read configuration information from server_config.json
	config.ReadServerConf()

	etc...

	//Setup our routes
	router.Handle("/places",
		handlers.Execute(&handlers.PlacesHandler{},
			handlers.AuthHandlerAdapter(),
			handlers.DBConnAdapter())).Methods("GET")

	etc ...
}

We’re using the Gorilla web toolkit in our example. The Router’s Handle method takes a path (/places in this case) and something that implements the http.Handler interface. handlers.Execute() in the code above must return something that implements the http.Handler interface.

The router.Handle() call is making use of middleware. Middleware is something you can chain together. Pieces of middleware can be reused to build other chains. If something breaks a chain, an error is typically returned and the rest of the chain’s elements are not executed. It is nice to express this chain with with a method signature that is easy to understand.

In our example, the handlers.Execute() method expresses a chain of middleware to call for the /places route. Specifically, this route will execute the AuthHanlderAdapter then DBConnAdatper middleware code and finally the PlacesHandler handler.

The AuthHandlerAdapter middleware is used to perform an authentication check. The DBConnAdapter middleware is used to open a shareable database connection. We’ll look at hanlders.Execute() and the middleware a little closer in in the next sections.

Getting to handlers.Execute()

If you browse the sample project code accompanying this post for a file called ./handlers/auth.go, you will see the AuthHandler type definition. The AuthHandler type has a field called next, which refers to the next middleware in the chain to call ServeHTTP(w, r) on. The AuthHandler’s type definition looks like the following :

type AuthHandler struct {
	next http.Handler
}

We know that http.Handler interface implementations, will have a ServeHTTP(ResponseWriter, *Request) method. For example, here is a portion of the AuthHandler’s ServeHTTP method :

func (h *AuthHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
	logger.Info.Println("In AuthHandler ... ")

	etc ...

	//Call the next element in the chain
	if h.next != nil {
		h.next.ServeHTTP(w, r)
	}
}

Notice the call to h.next.ServeHTTP(w, r) in the code above. That’s how we link to the next middleware in a chain.

We can create a new type called Adapter, which encapsulates the idea that you pass an http.Handler type and get a new adapted one. This new type’s definition looks like the following :

type Adapter func(http.Handler) http.Handler

Functions can now return this new Adapter type. For example, the AuthHandlerAdapter function below returns a an Adapter type and looks like the following :

func AuthHandlerAdapter() Adapter {
	//The adapter type is a function that
	//takes an http.Handler interface and returns
	//an http.Handler inteface
	adapter := func(h http.Handler) http.Handler {
		//The AuthHandler type implements the
		//http.Handler interface
		return &AuthHandler{next: h}
	}
	//Return the adapter
	return adapter
}

In the function above, the h http.Handler parameter (that is passed in the adapter) is stored in a new AuthHandler middleware and that middleware is returned. The h http.Handler parameter becomes the next middleware that the AuthHandler will call.

The Execute function takes a http.Handler and a list of Adapter types. The Execute function iterates over all adapters, calling them one by one in a chained manner and then returns the result of the first adapter (which is a http.Handler). The router.Handle then makes use of that.

func Execute(h http.Handler, adapters ...Adapter) http.Handler {
	//Note that we're ranging in reverse order.
	//The last piece of middleware will be the first
	//to execute.
	for i := range adapters {
		adapter := adapters[len(adapters)-1-i]
		h = adapter(h)
	}
	return h
}

Before we look at the authentication middleware, we’ll have a quick look at how the server’s configuration is read and used for authentication.

Server Configuration

When the server starts, it calls the config.ReadServerConf() function. If you look at the code inside ./configuration/serverconfig.go, this function looks for a file called ./server_config.json. That file is used to populate a ServerConf variable of type Config. The ServerConf variable has a setting for the server’s port (Port) and another for whether an authentication check (AuthPass) should pass or fail.

In main.go, getting the server’s port looks like the following :

//Read configuration information from server_config.json
config.ReadServerConf()

//The port our server listens on
listenPort := config.ServerConf.Port

The server_config.json file looks like :

{
    "Port" : 8080, 
    "AuthPass" : true
}

The AuthPass setting drives our authentication middleware. If you want the authentication check to fail in the AuthHandler middleware, set the AuthPass value to false, restart the server, and point your browser to http://localhost:8080/places or http://localhost:8080/people. The server should return a 403 error.

Authentication Middleware

The first middleware, that all routes in this example use, is the AuthHandler middleware. If you have a look at the code inside ./handlers/auth.go , you’ll see the AuthHandler struct implementation. The first thing to note is that it has a field called next which is of type http.Hanlder. That’s the next middleware in the chain to call if the authentication check is successful.

The AuthHandler struct implements the http.Handler interface. The ServeHTTP method is our stub implementation for this middleware. The method determines whether the authentication check passes based on the AuthPass value that is set in the server_config.json configuration file. In a real world example, authSuccess could be set by validating a token sent in the request header.

If the authentication check passes (i.e. authSuccess is true), we call the next middleware’s ServeHTTP method. If the authentication check fails, we log the error, send an error response back to the client and return from the function. This stops the middleware execution chain.

The AuthHandler’s ServeHTTP method looks like the following :

func (h *AuthHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
	logger.Info.Println("In AuthHandler ... ")

	logger.Info.Println("Authentication check....")

	//An authentication handler stub. authSuccess could be set by validating
	//a token sent in request header. User sends a valid token, authSuccess is set to true

	//Right now it's set in a config file.
	//Try setting AuthPass to false in the server_config.json file and restarting the server
	//to see the authentication faliure
	authSuccess := config.ServerConf.AuthPass

	if !authSuccess {
		errJson := `{"Error":"Access Denied"}`
		logger.Error.Println("Access Denied")
		http.Error(w, errJson, http.StatusForbidden)
		return
	}

	logger.Info.Println("Authentication check passed.")

	if h.next != nil {
		h.next.ServeHTTP(w, r)
	}
}

Database Middleware

The next piece of middleware, that all routes in this example use, set’s up a SQLite database connection and uses Gorilla’s context package to share that connection with other middleware. If you have a look at the code inside ./handlers/dbconn.go, you’ll see the DBConn struct implementation.

The magic happens in the ServeHTTP method with the call to the context.Set(r, consts.DB_KEY, db). Also note that we close the database once all middleware has executed with the call to defer db.Close(). If there are no errors connecting to the database, we call the next middleware’s ServeHTTP method.

Other handlers can retrieve this database connection with a call like : db, ok := context.GetOk(r, consts.DB_KEY). We’ll see this connection being used in the next section.

The DBConn’s ServeHTTP method looks like the following :

func (h *DBConn) ServeHTTP(w http.ResponseWriter, r *http.Request) {
	logger.Info.Println("Setting up the database connection")
	_, ok := context.GetOk(r, consts.DB_KEY)

	if ok {
		errJson := `{"Error":"Internal Server Error"}`
		logger.Error.Println("DBConnector middleware error. DB_KEY already set.")
		http.Error(w, errJson, http.StatusInternalServerError)
		return
	}

	//https://github.com/mattn/go-sqlite3/blob/master/_example/simple/simple.go
	db, err := sql.Open("sqlite3", "./middleware-test.db")

	if err != nil {
		errJson := `{"Error":"Internal Server Error"}`
		logger.Error.Println("DBConnector database connection problem. Check Logs.")
		http.Error(w, errJson, http.StatusInternalServerError)
		return
	}

	context.Set(r, consts.DB_KEY, db)

	//Close database connection once middleware chain is complete
	defer db.Close()

	if h.next != nil {
		h.next.ServeHTTP(w, r)
	}
}

The Places Handler

The PlacesHandler type makes use of the shared connection (during a request’s lifetime) that we setup in the DBConn middleware above. In fact, there is another handler in our sample application that also makes use of this shared connection. That’s the PeopleHandler. For now, let’s look at the ServeHTTP method of the PlacesHanler.

The ServeHTTP method in this handler looks like :

func (h *PlacesHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
	logger.Info.Println("In PlacesHandler ... ")

	//Get the connection that was set in the dbconn middleware
	db, ok := context.GetOk(r, consts.DB_KEY)

	if !ok {
		errJson := `{"Error":"Internal Server Error"}`
		logger.Error.Println("DBConnector middleware error. DB_KEY not set")
		http.Error(w, errJson, http.StatusInternalServerError)
		return
	}

	//Query the database to get the places...
	rows, err := db.(*sql.DB).Query("select *  from place")

	if err != nil {
		errJson := `{"Error":"Internal Server Error"}`
		logger.Error.Println("DB query issue. ", err)
		http.Error(w, errJson, http.StatusInternalServerError)
		return
	}

	defer rows.Close()

	type Place struct {
		Name string
	}

	type Places []Place

	type WResponse struct {
		Places
	}

	resp := WResponse{}

	//Stuff places inside a response object
	for rows.Next() {
		var name string
		err = rows.Scan(&name)

		if err != nil {
			errJson := `{"Error":"Internal Server Error"}`
			logger.Error.Println("DB query issue. ", err)
			http.Error(w, errJson, http.StatusInternalServerError)
			return
		}

		res := Place{
			Name: name,
		}

		resp.Places = append(resp.Places, res)
	}

	//Create JSON out of our response object
	mResp, err := json.Marshal(resp)
	if err != nil {
		msg := "Response Marshal() error."
		json := fmt.Sprintf(`{"Error":"%s"}`, msg)
		logger.Error.Println(msg, err)
		http.Error(w, json, http.StatusInternalServerError)
		return
	}

	//Create json with all results...
	w.Header().Set("Content-Type", "application/json; charset=UTF-8")
	w.WriteHeader(http.StatusOK)
	fmt.Fprintf(w, string(mResp))
}

The line that reads db, ok := context.GetOk(r, consts.DB_KEY) in the code above, gets the shared connection using the Gorilla web toolkit’s context package. The rest of the method retrieves some data from the database and returns it as a JSON response.

Full Example Source

Source files for sample project outlined in this post : GitHub Repository

Running the Example

  • Check out the sample project code from from GitHub repository.
  • Type : go run main.go from inside the ./server directory
  • Point your browser to http://localhost:8080/places
  • You should see a response from the server that looks something like the following :
  • The server’s logs should look something like :
comments powered by Disqus