Microservices are lightweight, self contained network applications designed to serve a single purpose or small set of related functions. These services are used to rapidly deploy new features implemented in Pulse and other applications.

Each service must conform to the criteria outlined in this guide in order to be suitable for deployment in a production environment.

Important: This guide covers the essential requirements needed for all microservices @ Pulse. For best practice development refer to the individual framework's recommendations.

Common Service Attributes

All services must be:

  • Completely configurable through the use of environment variables
  • Fully documented (swagger)
  • Designed with a consistent API, implementing error handling and validation on all requests
  • Secure (using API keys or CloudAuth)
  • Deployable as an ephemeral docker container in Linux
  • Tested through unit and integration tests

Preferred Frameworks / Tech Stacks / Languages

When it comes to choosing a framework for a microservice the answer is simple: the framework that best solves the problem is the preferred framework. Use the list below as a guide for choosing the most suitable framework for a given service, bearing in mind that this list may be expanded based on requirements:

  • Node.js (version 12 LTS) - for quickly developing simple services. If the service is purely a backend API with no UI, use Fastify. If the service maintains a simple UI or uses packages that require Express, use Express. If the service requires the features of a typical, full-stack web application (UI templating, i18n support, SQL ORM, DB migration management, multiple authentication methods, independent user/service accounts), use Adonis.
  • .NET Core - if the service requires .NET core packages not found in Node.js, use .NET Core.
  • Go - for simple services that require high performance use Go with Echo as the primary web framework.
  • Python - for services that interact with machine learning frameworks such as Tensorflow and PyTorch.

To reiterate, this is not a definitive list. If there is a solid argument for using any other framework / tech stack, speak to your peers and together we can decide if it's a right fit Pulse.

Service Naming

Since services are designed to be used by multiple applications / other services, we don't need to prefix an owning name. Additionally, we know it's a service so we don't need to include the word 'service' or 'tool' etc. The name of the service should be short and simple, something which adequately describes the service. Examples:

  • CloudAuth (cloudauth.pulsesoftware.com): Service for authenticating requests across services
  • FilePreview: (filepreview.pulsesoftware.com): Service for previewing documents in the browser

API Design and Documentation

  • Regardless of the framework used, all RESTful API design and development must follow these guidelines.
  • The root/home/index controller must contain, auto redirect or link to complete swagger API documentation. Similar to this example.
  • The documentation should contain all information required to use the service, including authentication methods and sample code.

API Security / Authentication

Every API endpoint must be secure except for those serving static resources.

  • If an API endpoint is designed to be called from a browser or any third party untrusted application, then CloudAuth is to be used as the primary authentication method. CloudAuth enables services to pass secret information between one another via proxies (in most cases the proxy is a browser or a third party application). In some cases, a proxy may need permission to access a service, however the proxy should not be able to view the details of the permissions and data involved. Example: ServiceA has a UI and needs to interact with ServiceB. ServiceB is called from the browser (served from ServiceA). If we allow the browser to interact with ServiceB using a simple API security header, then it'd be trivial for a malicious actor to manipulate the request details sent to ServiceB. Instead, ServiceA uses CloudAuth to hide secret data (parameters for ServiceB) then issues a token to the browser. The browser then uses this token to interact with ServiceB. ServiceB takes the CloudAuth token and retrieves the secrets set by ServiceA, knowing the data has not been interfered with.
  • If the API endpoint is only ever to be consumed by trusted services (ie. Pulse services) then API keys based on request headers may be used. A request header named 'x-api-key' must be handled by the service for the means of authenticating a request. The 'x-api-key' setting will be set as an environment variable in the service (not in a config file). For server-side Node.js apps, it's recommended to use .env in development, however, ensure that the .env file is NOT included as a part of the source code repository.
  • All unauthorised requests must return 401 responses with details of the failed permission check.
  • Important: Document the type of security used. If using CloudAuth, specify the parameters that need to be set by the calling service.

Memory / Data Caching Methods

  • The local memory cache should not be used to store frequently accessed data between requests due to a load balanced production environment.
  • If a service requires caching data for the sake of keeping latency to a minimum, then it's best to use Redis. A Redis server can be provisioned if it does not already exist.

File I/O

Microservices run inside containers. Containers are ephemeral and should not be treated as a permanent location to store files. If a service requires a temp folder to write files to, then the entire life cycle of the file (creation, modification and deletion) must be handled in a single request. Any longer term storage should persist the files to a Amazon S3 bucket. The service also needs to maintain the file (including deletion after a period) in S3 or push the file to a folder with a limited life span for auto-deletion.

Since all microservices run on Linux, file paths must be compatible with both Linux and Windows (ie. use forward slashes in file paths or path.join() in Node.js). Always use relative paths where possible and always assume the service does not have write access outside of the root folder.

Error Handling, Logging & Validation

The appropriate HTTP status codes must be returned from every request.

  • All events including errors should be logged to the console. Request level logging should occur on every request.
  • Exceptions at an API level must be propagated to the caller. Unhandled exceptions should return a 500 response with details of the error. This does not automatically happen in most Node.js web frameworks. Each request must be surrounded with a try / catch and ensure the error is propagated.
  • Parameter validation should occur for every request. Invalid parameters should return a 400 response.

Database Support

Every service has different database requirements (if any). Services may use any of the following database providers:

  • PostgreSQL (prod + dev) - version 12
  • Azure SQL Server (prod + dev) / SQL Server 2016 (backup)
  • MongoDB (prod + dev) - version 4.2
  • Redis (prod + dev) - version 6

Readme / Service Descriptions

Each service should have a descriptive readme.md file in the root folder of the repo. This should describe:

  • The purpose of the service
  • Any service dependencies
  • How to run the service in development mode including which environment variables need to be set for debugging

Docker Config

Each service needs to be deployed with the following files in the root folder of the repo:

  • dockerfile - this is a text file which contains the docker commands used to build the docker image.

Every service must be deployed and tested as a container in Linux prior to deployment to production. Each docker image must package pre-built files. This includes files generated via transpilation such those created by webpack or other transpilers. Some simple rules for creating docker files:

  • Don't use the 'latest' tag when referring to the base image (ie. the FROM section). Chose a version of a base image that you know works with the solution.
  • Ensure you include all dependencies.