All Articles

Type-safe APIs with ReScript

In the previous post, we learned how to build an API for AWS Lambda in ReScript. Our API was a simple “hello-world” example that accepted a request in JSON format containing a name, and echoed it back in the response.

However, our processing of the input JSON did no request validation before converting it to a ReScript type, effectively defeating the purpose of using a type-safe language like ReScript (garbage-in, garbage-out).

In this example, we’ll learn how to build a parser for our input to ensure that the conversion to our ReScript types is correct. We’ll use funicular for this example, a simple JSON encoder/decoder that allows flexible, composable parsers to be constructed, all while maintaining meaningful error-output.

You can find the complete source code that accompanies this article here

API

Instead of “hello-world”, this time we’ll build a dummy RegisterUser API that accepts a name, email, and password and returns a user record with an auto-generated ID.

Request

The request type is a simple object that contains the input values needed to generate the user:

module RegisterUserRequest = {
  type t = {
    name: string,
    email: string,
    credentials: Credentials.t,
    age: option<int>
  } 
}

NOTE: we’ve used the convention of putting the target type within a module and naming the type t. We’ll add the decode and encode functions for each type to their respective modules in the next section.

It refers to a Credentials which contains the password string used to initialise the account:

module Credentials = {
  type t = {
    password: string,
    passwordAgain: string
  }
}

User type

The API will be responsible for generating the user, assigning it an ID, and returning it in the response. For this, we’ll need a domain model type of User.t to store the user.

module User = {
  type t = {
    id: string,
    name: string,
    email: string,
    age: option<int>
  }
}

Note that we don’t echo back the password (for obvious reasons).

Response Type

Our response type simply carries the User.t we created in the body, and hence is rather simple:

module RegisterUserResponse = {
  type t = {user: User.t}
}

Error Type

Lastly, because this will be a checked API that validates its input and provides meaningful errors in the case of a bad request or some other issue, we will define a simple error response type.

module Error = {
  type t = { code: string, message: string }
}

Writing the types and encoders/decoders

In the previous article, we simple wrote a wrapper around JSON.parse to decode request data, such that the code assumed that the request was provided in the right format. We also demonstrated how it could be broken by sending unexpected input.

This time round, we want to validate our inputs, so we will use funicular to write JSON decoders for each request type to validate and produce the values we expect in a typesafe way.

Creating a naive decode function

Firstly we’ll look at writing a decoder function without any helpers to get an idea of what we need to achieve.

We want to write a decoder that takes a string with a JSON of the same layout and parses it safely into the User.t record type. In order to do this, we would need to first deserialise with Js.Json.parseExn (which produces a Js.Json.t) and then:

  • Parse it with Js.Json.decodeObject() to produce a Js.Dict.t<Js.Json.t>, which is a dictionary with Js.Json.t values.
  • For each key in the User.t record type (id, name, etc.) try find a value in the dictionary, and then use the equivalent Js.Json.decode* function to parse it
  • The Js.Json.decode functions all return an option type (Some(value) if defined, None if it is not the matching type), so then we’d have to pattern match on the result and map to the User.t

The result would be a bit of a mess, but look something like this:

  let decode = value => {
    switch Js.Json.decodeObject(value) {
      | Some(o) => {
        let id = Js.Dict.get(o, "id")->Belt.Option.map(Js.Json.decodeString)
        let name = Js.Dict.get(o, "name")->Belt.Option.map(Js.Json.decodeString)
        let email = Js.Dict.get(o, "email")->Belt.Option.map(Js.Json.decodeString)
        let age = Js.Dict.get(o, "age")->Belt.Option.map(x => Belt.Option.map(Js.Json.decodeNumber(x), y => Belt.Int.fromFloat(y)))
        switch (id, name, email, age) {
          | (Some(Some(id)), Some(Some(name)), Some(Some(email)), Some(Some(profile))) => Ok({ id, name, email, age })
          | _ => Error(`one or more properties missing`)
        }
      }
      | None => Error(`Not an object`)
    }
  }

As you could tell, this quickly becomes unwieldly and unreadable, and we haven’t even handled each possible error (missing field, incorrect type) for each field separately, so a consumer may not know what went wrong with their input.

Using funicular for a composable decoder

We can do better with funicular:

  let decode = val => {
    open Funicular.Decode;
    // Decode the val as an object
    let o = val->object_

    // Decode each field
    let id = o->field("id", string)
    let name = o->field("name", string)
    let email = o->field("email", string)
    let age = o->field("id", optional(integer))

    // Combine the field values into the User.t record
    rmap((id, name, email, age) => { id, name, email, age })
    ->v(id)
    ->v(name)
    ->v(email)
    ->v(age)
  }

In the above example, the value is first decoded as an object using object_ from the Funicular.Decode package. This function actually operates on, and returns, a result type, which contains the intermediate value (as does the other helpers like field() or string()).

This means we can write functions that only perform an operation if the input result actually contains a value i.e. they are Ok(value); otherwise they pass on the error as Error(error). By structuring our functions like this, we can write readable code using composition that has error-handling built-in, but does not require pattern matching everywhere.

Next, each of the fields is extracted using field(), which takes the field name and a decoder function (string, integer, etc) based on type. The optional helper is used to wrap decoders, and is used for the age field, which is of type option<int> (hence optional(integer)).

The last bit is the trickier bit because we have to reassemble each of the field results into the record. Thankfully, rmap() and v() take care of this for us by wrapping the map function into a result and then feeding each result in, ultimately returning a result containing the User.t record.

Writing funicular decoders

funicular uses a composable API that consists of providing a decode function for each type 't. Decoders always have the type signature:

let decode: (jsonTreeResult) => jsonParseResult<'t>

such that we can call deserialise any JSON with a given code function by calling Funicular.Decode.parse(string, decode).

In the previous example, we provided a decode function for the User.t, so to decode a serialised user string you would write:

let userString = `{
  "id": "A", 
  "name": "chris armstrong", 
  "email": "chris@example.com" 
}`;
let userResult = Funicular.Decode.parse(userString, User.decode);

jsonTreeResult and jsonParseResult are instantiated forms of the result monad that look like this:

type jsonTreeResult = 
  result<
    { tree: Js.Json.t, path: string[] }, 
    jsonParseError,
  >;
type jsonParseResult<'t> = result<'t, jsonParseError>;

result is used to encode each stage of the result and carry errors, instead of throwing exceptions. The decode functions must be aware of this and ‘open up’ the result (using pattern matching or Belt.Result.map) each time to transform it. jsonTreeResult also carries JSON path information at each node in the tree to produce errors that point at the part of the structure that caused the error.

Although this sounds more complicated than just throwing exceptions on any error, it lets us write type-safe error handling code with full exhaustiveness checking, because each possible error is specified in jsonParseError (and therefore must be matched on anything processing it).

Thankfully, each of the components of funicular is designed to manage most of this detail from you, so you only have to define map functions for your object types.

Composing Decoders

In the User example, we only decoded a flat object that contain no nested values (only scalar properties like string or int). The request type, on the other hand, contains a nested field credentials of type Credentials.t:

module RegisterUserRequest = {
  type t = {
    name: string,
    email: string,
    age: option<int>,
    credentials: Credentials.t,
  }
  ...
}

When we write the decode function for RegisterUserRequest.t, we can simple use the Credentials.decode function to decode the credentials field:

  let decode = val => {
    open Funicular.Decode
    let o = val->object_
    rmap((name, email, age, credentials) => {
      name: name,
      email: email,
      age,
      credentials
    })
    ->v(o->field("name", string))
    ->v(o->field("email", string))
    ->v(o->field("age", optional(integer)))
    // Decode credentials by delegating to Credentials.decode
    ->v(o->field("credentials", (Credentials.decode)))
  }

Writing an Encoder

Writing an encoder is thankfully much easier, as we don’t have the type validation problem. An encoder for the User type looks like this:

  let encode = ({id, name, email, age}) => {
    open Funicular.Encode
    object_([
      ("id", string(id)),
      ("name", string(name)),
      ("email", string(email)),
      ("age", optional(age, integer))
    ])
  }

Implementing a type-safe API handler

Now that we’re using funicular for decoding, we can parse the request body (which should be decodable to type RequestHandler.t) in a type-safe manner and offer meaningful errors if the JSON parsing fails.

open Belt
open ApiGateway
open Schema

let handler = (event: awsAPIGatewayEvent) => {
  Js.log2("body", Option.isNone(event.body))
  // Convert body to a UTF-8 string if base64-encoded
  let body =
    event.body
    ->Option.map(body =>
      event.isBase64Encoded
        ? body
            ->Buffer.from(~encoding=#base64)
            ->Buffer.toString(~encoding=#utf8)
        : body
    )
    ->Option.flatMap(body => body != "" ? Some(body) : None)

  let response = switch body {
  | Some(body) => {
      let responseBody =
        body
        // Parse the event body with the correct decoder
        ->Funicular.Decode.parse(RegisterUserRequest.decode)
        // Construct the response body
        ->Result.map(message =>
          RegisterUserResponse.encode({
            user: {
              id: "id",
              name: message.name,
              email: message.email,
              age: message.age,
            },
          })
        )

      switch responseBody {

      // Request was decoded ok - send success response
      | Ok(body) => apiGatewayJsonResponse(200, body)

      // Error decoding request 
      | Error(#SyntaxError(description)) =>
        apiGatewayJsonResponse(
          400,
          Error.encode({
            code: "SyntaxError", 
            message: description
          }),
        )
      | Error(#NoValueError(path)) =>
        apiGatewayJsonResponse(
          400,
          Error.encode({
            code: "NoValueError", 
            message: `Expected a value at path ${path}`
          }),
        )
      | Error(#WrongTypeError(path, type_)) =>
        apiGatewayJsonResponse(
          400,
          Error.encode({
            code: "WrongTypeError",
            message: `Expected a value type ${type_} at path ${path}`,
          }),
        )
      }
    }
  // The body was empty - send error response
  | None =>
    apiGatewayJsonResponse(
      400,
      Error.encode({
        code: "MissingBody", 
        message: "No request body found"
      }),
    )
  }
  Js.Promise.resolve(response)
}

In the above example, you can see how the body is parsed using the RegisterUserRequest.decode function, and that if successful, a response is encoded and packed into a apiGatewayResponse. Otherwise, each known JSON error type is switched on, and a corresponding error response issued.

The JSON errors are of type Funicular.Decode.jsonParseError, and we can use a helper function to pretty-print to simplify our error handling:

  switch responseBody {
  | Ok(body) => apiGatewayJsonResponse(200, body)
  | Error(error) => {
    let message = Funicular.Decode.jsonParseErrorToString(error)
    let code = "ParseError"
    apiGatewayJsonResponse(400, Schema.Error.encode({ code, message }));
  }
  }

Although we no longer have a unique error code per JSON error type, our error messages will still be meaningful for a developer testing their code against the API.

Deployment and Validation

Start the API locally with SAM CLI:

> npm run build
> sam local start-api

This will build our code and start a local API server.

First lets see what happens if we send an empty response body:

> curl http://localhost:3000/users -XPOST --data ''
{"code":"MissingBody","message":"No request body found"}% 

So far so good - we’ve verified the first part of our handler which checks the string is working.

Next, let’s try send a complete user request:

> curl http://localhost:3000/users --data '{"name": "Chris", 
  "email": "chris@example.com", "credentials": { "password": 
  "passw0rd", "passwordAgain": "passw0rd" } }'
{"user":{"id":"id","name":"Chris","email":"chris@example.com"}}% 

This confirms our request handler is working.

What happens if we leave out a necessary value?

> curl http://localhost:3000/users --data '{"name": "Chris", 
  "email": "chris@example.com", "credentials": { "password": 
  "passw0rd" } }'
{"code":"ParseError","message":"No Value Error: expected a value at $.credentials.passwordAgain"}%

Or if we specify something with the wrong type?

> curl http://localhost:3000/users --data '{"name": "Chris", 
  "email": "chris@example.com", "age": true, "credentials": { "password": 
  "passw0rd", "passwordAgain": "passw0rd" } }'
{"code":"ParseError","message":"Wrong Type Error: integer was expected at path $.age"}% 

Or if the JSON is ill-formed?

> curl http://localhost:3000/users --data '{"name": "Chris", 
  "email": "chris@example.com", , "credentials": { "password": 
  "passw0rd", "passwordAgain": "passw0rd" } }'
{"code":"ParseError","message":"Syntax Error: Unexpected token , in JSON at position 51"}% 

Unlike in our previous article, when we used external to bind against JSON.parse() directly, this time our input request must be well-formed in order to continue with the API request.

Summary

In this article, we explored how we could improve on the previous attempt to build an API, by using a self-validating JSON parser built with funicular. We then tested that our API handled syntax, type and missing value errors.

Next time, we’ll evaluate how we can fill out our API with a real implementation, exploring Promise usage, AWS API calls and implementing multiple API endpoints with API Gateway.