Articles

How to use gRPC and protobuf with JavaScript and ReactJS?

Raphael Harmel | June 29, 2020

This article was written in June 2020.

As the JavaScript implementation of protobuf by Google is still recent, and will probably be updated in the future, check the official documentation for any changes since this article was written.

Livongo’s FrontEnd stack was historically using AngularJS and REST APIs, but we are migrating all our webapps to ReactJS and gRPC.

As gRPC is not really straight-forward coming from the JavaScript world, here is what I learned while working on improving our enrollment web app.

What are gRPC and protocol buffers (aka protobuf)?

According to wikipedia:

“gRPC (gRPC Remote Procedure Calls) is an open source remote procedure call (RPC) system initially developed at Google in 2015. It uses HTTP/2 for transport, Protocol Buffers as the interface description language, and provides features such as authentication, bidirectional streaming and flow control, blocking or nonblocking bindings, and cancellation and timeouts. It generates cross-platform client and server bindings for many languages. Most common usage scenarios include connecting services in microservices style architecture and connecting mobile devices, browser clients to backend services.

gRPC uses protocol buffers to encode the data. Contrary to HTTP APIs with JSON, they have a more strict specification. Due to having a single specification, gRPC eliminates debate and saves developer time because gRPC is consistent across platforms and implementations.”

Why did Livongo decide to use gRPC?

At Livongo, we started using gRPC recently for several reasons:

  • Contrary to REST APIs, gRPC provides a strongly-typed solution with a clear contract between back-end and front-end developers to avoid human confusion and system errors.
  • We are moving to a more distributed architecture to be able to support our growth and scale seemingly. gRPC is just…designed for that.
  • The gRPC compiler automatically generates client and server stubs in the various coding languages we use: Javascript, Swift, Java, Scala.

Downsides of using gRPC with JavaScript

Unfortunately, the documentation is almost nonexistent for JavaScript and ReactJS, and the   Google official protocol-buffers implementation JavaScript generated files are not completely straightforward for a JavaScript developer to wrap their heads around it, with the extensive use of getter and setter to create and read objects.

As Dave Engberg, our CTO puts it:

“For a JavaScript developer, gRPC generated JavaScript is like JavaScript, but as a foreign language”.

So I hope this post, going a bit more in depth that what I was able to find online, will help other JavaScript and ReactJS developers like me!

Protobuf and JavaScript

There are currently 2 main implementations of protobuf for JavaScript: the official Google implementation and Protobuf.js

Official Google implementation

There seem to be 2 GitHub repositories for that:

The first one is: https://github.com/grpc/grpc-web

The second one is: https://github.com/protocolbuffers/protobuf/releases

Protobuf.js

The rest of this article will focus on the Google official protocol-buffers implementation.

How do you use gRPC and protobuf with JavaScript and ReactJS?

1. Creating the protobuf file(s)

In the case of our enrollment webapp, we need to create various proto files to handle the user’s answers to be sent to our server. These answers are captured using various <input> html fields.

Here is the file structure:

enrollment
|_ enrollment_api.proto
|_ common
|   |_ typed_value.proto
|   |_ question_ids.proto
|_ questions
    |_ answer.proto
    |_ submit_answers.proto

TypeValue message

/enrollemnt/common/typed_value.proto

This file contains the different possible types of answers:

  • A text string for text inputs
  • A boolean expression for checkboxes
  • An integer for integer inputs
  • A float for flat inputs
  • A date. For this one we had to use a Livongo-specific date definition
  • A single_select_enum_code string for single enums for radio buttons with only one possible choice
  • A RepeatedEnum multi_select_enum string for radio buttons with multiple possible choice
syntax = "proto3";

syntax = "proto3";

package com.livongo.protobuf.grpc.external.enrollment.common;

import "livongo/protobuf/grpc/external/enrollment/common/question_ids.proto";
import "livongo/protobuf/common/temp/date.proto";

message TypedValue {
    message RepeatedEnum {
        repeated string enum_code = 1;
    }

    oneof value {
        string text = 2;
        bool boolean = 3;
        int32 integer = 4;
        float float = 5;
        .livongo.protobuf.common.temp.Date date = 6;
        string single_select_enum_code = 7;
        RepeatedEnum multi_select_enum = 8;
    }
}

QuestionId message

/enrollemnt/common/question_ids.proto

This file contains the message defining the questions, containing:

  • id (type=values) to identify the question. Check the comments in the code to see which question corresponds to which data type defined in typed_value.proto
  • version (type=int32) defining the version of the question definition, in case it’ll change in the future
syntax = "proto3";

package com.livongo.protobuf.grpc.external.enrollment.common;

// Defines the set of known questions
message QuestionId {
    // Defines set of questions we can ask
    enum Values {
        UNKNOWN = 0;

        FIRST_NAME = 1;          // type: text
        LAST_NAME = 2;           // type: text
        EMAIL = 3;               // type: text
        TERMS_AND_CONDITIONS = 4 // type: boolean
        ZIP = 5;                 // type: int32
        A1C_VALUE = 6;           // type: float
        DIABETES_TYPE = 7        // type: single_select_enum_code: Type 1, Type 2, I don't know
        LANGUAGES = 8            // type: multi_select_enum: English, Spanish, French
    }

    Values id = 1;

    int32 version = 2;
}

Answer message

This file contains the message defining the answer, containing:

  • Question_id (type=QuestionId) to identify the question
  • value (type=TypedValue) defining the type of the answer
syntax = "proto3";

package com.livongo.protobuf.grpc.external.enrollment.questions;
import "common/question_ids.proto";
import "common/typed_value.proto";
message Answer {
    common.QuestionId question_id = 1;
    common.TypedValue value = 2;
}

Submit Answers message

/enrollment/questions/submit_answers.proto

This file contains the message defining the submit answer request, containing:

  • answers (type=Answer) defining the type of the answer, as defined in answer.proto
syntax = "proto3";

package com.livongo.protobuf.grpc.external.enrollment.questions;

import "livongo/protobuf/grpc/external/enrollment/questions/answer.proto";

message SubmitAnswersRequest {
  repeated Answer answers = 3;
}

message SubmitAnswersResponse {     // we define here the server response}

Defining the service and its rpcs

/enrollment/enrollment_api.proto

This file is the entry point, defining our service and all the rpc to be called:

  • SubmitAnswers which we are going to use in this article
  • OtherRPC is just here to show how you can include more than one rpc
syntax = "proto3";

package com.livongo.protobuf.grpc.external.enrollment;

import "livongo/protobuf/grpc/external/enrollment/questions/submit_answers.proto";

// APIs for enrolling new members.
service Enrollment {

    rpc SubmitAnswers (questions.SubmitAnswersRequest) returns (questions.SubmitAnswersResponse) {
    }
    rpc OtherRPC (...) returns (...) {
    }
}

2. Generating the corresponding JavaScript files

The protocol buffer compiler can generate JavaScript output when invoked with the –js_out= command-line flag.

The code can be generated with either Closure-style or CommonJS-style imports.

We chose the CommonJS-style imports as it made more sense for us to do so.

Here is the corresponding command for that:

$ protoc --js_out=import_style=commonjs,binary:. messages.proto base.proto

This command generates various files in our case:

enrollment
  |_ enrollment_api_pb.js
  |_ common
        |_ typed_value_pb.js
        |_ question_ids_pb.js
  |_ questions
        |_ answer_pb.js
        |_ submit_answers_pb.js

3. Using the generated JavaScript libraries

Now it’s time to use those libraries.

1. Initializing the server-side gRPC service

The first step is to initialize the server side gRPC service and then start it. That’s beyond the scope of this article, but you can check how to run a NodeJS service here.

2. Accessing the gRPC service from the front-end

For that, we define a proto.js file containing the following code:

Then we configure gRPC Web Developer Tools chrome extension, to be able to see the gRPC calls in the browser’s developer tool, as gRPC calls are not directly visible by browsers.

import {EnrollmentPromiseClient} from '@livongo/enrollment/lib/livongo/protobuf/grpc/external/enrollment/enrollment_api_grpc_web_pb';

const enrollmentPromiseClient = new EnrollmentPromiseClient(
    SERVICE_URL
);

if (process.env.NODE_ENV === 'development') {

    const enableDevTools = window.__GRPCWEB_DEVTOOLS__ || (() =&gt; {});

    // enable debugging grpc calls
    enableDevTools([enrollmentPromiseClient]);
}


3. Wrapping the generated file(s) and building your request

By wrapping, I mean encapsulating some of the complexity of the Google official protocol-buffers implementation to make the gRPC calls more natural to JavaScript developers, so that simple javascript objects can be used.

To build your request, you have to use setters like setParam(value), “Param” corresponding to the specific message parameter you are trying to set.

The best way to double check which setter you have to call is to check the code in the generated file(s).

import {SubmitAnswersRequest} from '@livongo/enrollment/lib/livongo/protobuf/grpc/external/enrollment/questions/submit_answers_pb';
import {QuestionId} from '@livongo/enrollment/lib/livongo/protobuf/grpc/external/enrollment/common/question_ids_pb';
import {TypeName} from '@livongo/enrollment/lib/livongo/protobuf/grpc/external/enrollment/common/typed_field_names_pb';
import {Answer} from '@livongo/enrollment/lib/livongo/protobuf/grpc/external/enrollment/questions/answer_pb';

// Converts an answer from a JavaScript object to a gRPC one
const answerToProtobuf = ans =&gt; {
    if (!ans || !ans.questionId || !ans.questionId.version) {
        throw new Error(errorMessage('Error', ans));
    }

    const questionId = new QuestionId();
    const typedValue = new TypedValue();
    const answer = new Answer();

    questionId.setId(ans.questionId.id);
    questionId.setVersion(ans.questionId.version);

    // switch to make the right call depending on the answer's data type
    // additional data parsing is done to make sure we send the data in the right format
    switch (ans.type) {
        case TEXT:
            typedValue.setText(ans.value.toString());
            break;
        case BOOLEAN:
            typedValue.setBoolean(ans.value === true);
            break;
        case INTEGER:
            typedValue.setInteger(parseInt(ans.value, 10));
            break;
        case FLOAT:
            typedValue.setFloat(parseFloat(ans.value));
            break;
        case DATE:
            typedValue.setDate(ans.value);
            break;
        case ENUM_SINGLE:
            typedValue.setSingleSelectEnumCode(ans.value);
            break;
        case ENUM_MULTI: {
            const repeatedEnum = new TypedValue.RepeatedEnum();

            repeatedEnum.setEnumCodeList(ans.value);
            typedValue.setMultiSelectEnum(repeatedEnum);
            break;
        }
        default:
            throw new Error(
                errorMessage('Unsupported data type for answer', ans)
            );
    }

    answer.setQuestionId(questionId);
    answer.setValue(typedValue);

    return answer;
};

// This were we call the rpc
export const submitAnswers = answers =&gt; {
    const request = new SubmitAnswersRequest();

    answers.forEach(answer =&gt; {
        request.addAnswers(answerToProtobuf(answer));
    });

    return enrollmentPromiseClient.submitAnswers(request);
};

You can see that we are actually building nested gRPC object, respecting what was defined in the proto files.

4. Using the wrapper

Now, let’s use this wrapper by importing it to your .js or .jsx file, whether you are working on a JavaScript (non-ReactJS) or ReactJS project.

1. Importing the wrapper

import {submitAnswers} from '/src/common/proto';

2. Submitting your request to the server

submitAnswers(
    questionId: {
      id: 1, // FIRST_NAME
    },
    type: TypeName.TEXT,
    value: 'Michael',
)

5. Accessing the response data

You can then use:

  • .toObject() to check and access the content of the response gRPC object and see the methods and fields which can be accessed
  • getParam() to access these values, “Param” corresponding to the specific message parameter you are trying to set. The best way to double check which setter you have to call is to check the code in the generated file(s).

Voilà, we were finally able to do what we were looking for: to submit a gRPC request to the server and read its answer!

Back to list