Corteza Gateway

Corteza, the leading open-source low-code development platform just had the first release of its routing and processing gateway for HTTP requests and I’m here to tell you all about it.

As an ongoing effort, the plan for this routing subsystem has always been

• to minimize the effort in adding dynamic routes with any validation
• to be able to connect the routing possibilities with most of the Corteza subsystem
• to take the developer-first approach in modular architecture and data flow transparency

During our development processes and via a wide range of our client’s proposals and business requirements, we quickly noticed we outgrew the possibilities Corteza offered via it’s sink feature.

Sink allowed a Corteza developer to detect incoming HTTP requests on a fixed route via a generated signature wrapped in query parameters and provide the request to the automation scripts.

But having only one route with limited capabilities to validate query parameters, headers and have an overview of the amount of incoming data fell short.

Central routing and processing gateway for arbitrary HTTP requests

Our main goal was to be able to provide the developer or administrator to easily add new routes (and possibly group them), validate, process and provide a meaningful response all the while keeping an eye on security and performance.

One of the biggest advantages of Planet Crust being a dev-centric company is we know the pains we sometimes need to go to in order to get a meaningful feedback from the process or a rich enough debug stream to fix or set up our business process in time.

With that in mind, we came up with a gateway subsystem that allows us to apply filters to different phases in the life cycle of a request coming into Corteza.

Too much written content for today? Jump down to a showcase

Moving parts

Routes

The main culprit for the handling of the request, simply added to the gateway with a specific endpoint. Defined with several predefined (but customizable) filters that can be applied to one of the three phases in the request life cycle.

Prefilters

Once the request comes into the system, it can be validated, authenticated or capped via rate or another origin limit. The authentication layer will eventually be available as a separate part from the Corteza authentication mechanism, but there is currently ways to limit the API access via query parameters or headers.

Processers

The processers include most of the business logic for the request flow, be it arbitrary payload data processing from the request body or perusing the proxy processer with abilities to use outbound authentication of choice and proxying the requests to the destination.

A lot of our routes use the Corteza low-code workflows tool to process the requests and make additional HTTP requests to external systems while having the ability to process the responses to our internal structures.

The idea of the processing phase is to be able to process arbitrary data, use it in the low-code Corteza tools, log via server output, taking note of the performance.

Not all requests can be done in real-time, so there is also support for asynchronous routes. The way the asynchronous routes work is to still be able to use the validation and authentication phases, the processing part is then
offloaded to the backend, so in effect, only the prefiltering and postfiltering is done synchronously. Since we have no way of knowing the output of the processing part, the last part — post filtering is done as a simple HTTP 202 request received response.

Postfilters

The idea for the postfilters is to finalize the request life cycle depending on the processed data with the output format of the response that is defined in the post-filters.

Debugging

Most of the time, when connecting different API’s or receiving web-hooks, we are mostly aware of incoming data structures. Even when we have good API documentation and consistent data structures, we still need a useful debugging tool or at least logging facilities.

Currently, debugging level can be enabled via standard Corteza environment configuration, so any input requests and body data can be logged to the underlying data aggregator or at least docker output.

There is basic performance analysis based on route and filter levels so any bottlenecks can be quickly spotted.

The idea for the next iterations is to be even more verbose in request — response cycle output with the ability to analyze the incoming requests and provide helpful feedback to the developer. One of the really helpful features with the current gateway is to point the external API webhooks to the designated Corteza gateway route and be able to analyze incoming traffic from the request. So say goodbye to print statements in the business logic 😉

Configuring the gateway

Adding a new endpoint is as simple as configuring a new route with a designated endpoint URL and the HTTP method it handles. Once the route is added and enabled, the endpoint is up and running, though open to the world as it has no authentication validation in check.

A quick demo of a header validation, request payload processing in the workflow and default response postfilter.Example of the debugging mode server logs

Conclusion

I hope this article gave you at least a glimmer of insight into what can be achieved with the ever-growing set of features the open-source Corteza CRM has already built.

The API gateway is an evolving subsystem that has already given us business value and will continue to do so even more with the planned features.

Want to dive deep?

Corteza github repository: https://github.com/cortezaproject/corteza-server/

Corteza documentation: https://docs.cortezaproject.org/corteza-docs/2021.3/index.html

Public forum: https://forum.cortezaproject.org/