Posts for Tag: developer tools

A Simple Node.js PM2 Setup Guide

Introduction

PM2 is a great tool that can help you manage not just your processes when they are running but also env types, vars, and configs. I still find many questions around the basic operation of the tool around the internet such as stackoverflow and other boards. So this article will cover how I setup most of my projects using PM2; short, sweet, and to the point. 

Setting up a quick app

First let's create a quick API that returns hello world as a response. Move into an empty directory and create a file called API.js and put the following into it. 

Then run npm init -y  This will cause npm to create a bare bones application package.json for you so other packages can be installed. You can also leave the -y out if you want to manually put in all the information for your project. 
After you have your project setup with NPM install the express library package npm i express --save 

With express installed you should now be able to run your api app with the following command node API.js 
You can confirm that it is running by going to http://localhost:3000/ in your browser. You should see "Hello World!" printed within the page.
You now have a basic app running! Within the terminal that you started the app in you can input ctrl + z to stop the API from running as we will be setting up PM2 to be our process runner.

Starting up PM2

Within the project directory install PM2 as a package globally and into your project dependencies npm i pm2 -g; npm i pm2 --save 

You have now installed all you need to setup production level process running and environment control ( with enough configs and a pipeline of course ). With the latest versions of PM2 the tool comes with the ecosystem command that will help us generate what we need to create and application definition for our small API. However to see some immediate action you can just run pm2 start API.js 

Confirm that your API start by visiting localhost:3000/ again and checking if the text "Hello World!" is rendered. You can see that your process starts up and you get a nice looking table print out with some facts about your process. You can shut down your process by running pm2 delete API .

Note: Delete will remove the application entirely from PM2's registry; however you can also use  pm2 stop API  and it will stop the application but not remove it so you can use  PM2 start API  to start it up again. In our case however we want PM2 to forget about our process since we are going to create an ecosystem config for it and start it that way.

Setting up ecosystem.js

Now that we have shown that pm2 will start our API, and that our API still returns what we expect we can setup a more maintainable and useful way to start our app. Within the project app run pm2 ecosystem 

This will generate a application definition with a few other things inside of a ecosystem.js file that it created. Lets go through the sections that get generated and a quick over view of what we can do with them.

Apps Section

This section is arguably the most important for getting your application up and running. This is where you will define how your application runs, environment specific vars, logging behaivors, and more. In addition you can define multiple apps in the same ecosystem file; this can be used to start up co-processors, log streamers, queue managers, and more.

I will go over a few of the fields that can be used with an app definition that I think are some of the most common or useful.

Instances - Int

The number of app instance to be launched. This can be set to -1 to start as many processes as the system has CPU cores subtract 1. I use this in docker setups a lot because I can allow the application to consume the entire container since that is what it is dedicated to. 

node_args - String Array

This is an array of arguments that will be passed to the actual node execution which allows you to pass things like the --harmony flags for older node version or things such as debug flags.

error_file, out_file, pid_file - String ( directory/file path )

These values point to the directory and file name that you want pm2 to export the generated logs from your application to. This is valuable again in the containerization scenario when you want your logs to go specific places to be picked up by log aggregation systems. 

max_restarts - Int

This is the number of consecutive unstable restarts (less than 1sec interval or custom time via min_uptime) before your app is considered errored and is stopped from being restarted any more. This is a great option to allow your application to show that it can't connect to mission critical services or API's at start up time. I currently use this as a flag that something is very wrong when a new deploy goes out.

max_memory_restart - String

This option allows a max memory limit to be set that, when hit, causes PM2 to restart you application automatically. Useful to ensure a rogue process doesn't bring down boxes if they suddenly get a ton of load or the process simply goes off the rails. The value of this field is a String that uses the normal volume types M = megabyte, b = byte, so if you wanted the max memory to be 400 megabytes then the value here would be "400M".

min_uptime - String

This option lets pm2 know that your application should be up and stable for x amount of seconds before being considered started instead of in the starting state. This can be important if your project has to connect to a service and timeouts are an issue or there is a lengthy read operation on disk, etc.

env - Object Key/Val

The env object is what allows you to define environment level vars based on the environment flag you give PM2 at startup time. the plain env field is an object that contains values that will be available all of the time regardless of the environment that is passed to pm2. 

Any field that follows the pattern env_${myEnvironmentName} is considered a valid env setting and the values defined within will only be available when the --env flag us used and a valid key that matches the following rules is found. For example a env_production env object then allows you to use the following command pm2 start ecosystem.js --env production 

Note I think it is worth mentioning how you get to these environment variables. All of these keys will be available via the global process variable. So to get the env NODE_ENV to see what mode the application has been started in you would use process.env.NODE_ENV.

Here is an example of all the discussed settings for an application definition. There are plenty more options that you can find in the PM2 docs page.

Deployments Section

As a disclaimer I do not use the PM2 deployment tool in any projects currently aside from deploying to some rasberry pi's I have hooked up on my local network. This is because these days all of my CI/CD pipelines use Docker or run through Heroku. That being said I will give you what I know on how to get deployments working for you via PM2. 

The use case for using the PM2 deployment tool set fits if you have static servers that are not containerized that you don't want to bring down your systems just to do a small update to your application. This is the case at with my home setup where I have node instances running on Rasberry Pi's. 

To setup deployments via PM2 first define your application section with the options you want from the previous section. Then in your ecosystem.js go to the deploy section of the config. Here you will find the two generated deployment options production and dev, these are enviornment configs just like the application definition has; they will be used to define what env your are deploying to. Here there are a few key definitions that need to be flushed out.

user - String

This is the user that the target machine will use to run any commands that are pushed to it via PM2. It must have the approprite permissions to execute the commands ( git pull, npm install, etc ) on the target machine. In addition this is the user that PM2 will attempt to authenticate via ssh using a key on your machine or a key you provide within the deployment configuration ( more on this a little later )

host - String/String Array

The host field can hold a single host or an array of hosts. These hosts can be IPs or hostnames that will get resolved via DNS. The machine doing the deploy must have an SSH key for these machines so that authentication can occur OR a .pem file must be given as part of the deployment config.

key - String

The file location of a .pem file that contains the approprite key to authenticate against the hosts using the User as the username.

ref - String

Ref is the git origin branch that you want to get deployed. Most of the time this will be a "production" branch or something similiar that you merge into when a version has been tagged or something similiar. 

repo - String

The git repository URI that the ref branch is in and the hosts have access to pull from. 

path - String

The path that the git branch will be downloaded into on the host.

pre-setup - String

This field is (a) command(s) to be run BEFORE the git checkout for the branch is done on the host.

post-setup - String

This field is (a) command(s) to be run AFTER the git checkout for the branch is done on the host. This is generally where you would put your npm install, gulp builds, etc.

post-deploy - String

This field is (a) command(s) to be run AFTER the pre and post setups events. This is where you will put your application restart/start command(s)

pre-deploy-local - String

This field is (a) command(s) that will run on the deployment machine, not the hosts that are being deployed to BEFORE it actually fires off the deployments. This is useful for putting deployment configs, alerts, etc into things like slack, emails, etc.

An example of the all these fields for a production env.

Once you have your application running for the first time on the hosts listed in the production deploy config. From a build server ( or your local box ) you can run pm2 deploy ecosystem.js production .

If everything runs successfully you should see a message similiar to : "Deploy Succesful" , you are all done!

Daemonizing your application with PM2 (2.2 >)

PM2 allows you to configure a startup script that will ensure your application comes back up if for some reason a restart or shut down has occured. This is useful again if you have bare metal you are running on that is not containerized or if for some reason you have a very unstable server setup; or have to do rolling restarts of servers for deployments. 

PM2 comes with the pm2 startup command that will out put a command for root to run that will add the approprite config so your application will start on machine startup. You can also pass pm2 an explicit type of startup script to generate if you know your environment OS. Here is the supported options: ubuntu, ubuntu14, ubuntu12, centos, centos6, arch, oracle, amazon, macos, darwin , freebsd, systemd, systemv, upstart, launchd, rcd, openrc. 

Here is an example output from the command :

On a build server you can write some simple awk, grep, or regex to extract the command and run it on the machine that is being provisioned. 

You can read more about process management via things like init.d and more here.

Wrap up

PM2 is a powerful tool for getting things up and running fast, but it also has the staying power for production level applications. I have many applications both at my day job and my personal applications that run using PM2 and KeyMetrics. Some of the deployment management I feel is antiquated by the rising use of containers and the associated services such as AWS, Google Cloud, Heroku, etc but it still has it's place in situations where you don't have the flexiability of a container driven development environment. 

Links :

http://pm2.keymetrics.io/docs/usage/application-declaration/

http://pm2.keymetrics.io/docs/usage/deployment/

http://pm2.keymetrics.io/docs/usage/startup/

Using Chrome Dev Tools To Debug Your Node.js Projects

Introduction 

To this day I get asked a lot on how I find issues inside my code base, some times even where I put my console.logs(). My answer is that I use a debugger; however almost every time this surprises people in the Node.js/JS community. I thought we had gotten past the strange period of JavaScript as a language where console.log()ing random points in your code base was the way to debug things. 

Apparently I was wrong; at least based on how often I get asked this type of thing.

So in the spirit of hoping to propagate something I strongly feel should be a standard and something every JS developer ( or any type of developer really ) should now how to setup and use, this is a small article on how to use Chromes dev tools to debug your Node.js projects.

Installation and Setup For Node 6.3 & Above

In Node 6.3 we got a native debugger module that Node.js now ships with that is actually developed by the Node team. To use this there is now command flag options that we can pass in when starting our node projects. It will do some simple quality of life things as well, if the same file ans instance are brought down and back up the debugger will reattached itself, which is pretty helpful.

Debugging Just Using Node

When you start your application now you just append a --inspect to the node command and it should do everything needed on the process level. 

node myProject.js --inspect

Next open up chrome and go to about:inspect in the URL bar. This will bring you to a panel that looks like the image here 

You can then click the "Inspect" link under the name and path of your running application and it will open up a standard chrome debugger that is attached to your process.

Debugging using PM2

PM2 is a great process runner that I personally use for all my node related projects. However due to the how PM2 works and handles configurations for projects it requires a little extra work to get running with --inspect. 


Managing developer debug configs and app definitions for pm2

A lot of the time you don't want to have create two different files just for debug mode. So what the teams I have been on normally do is just create two application definitions in the same ecosystem.json file and then create different startup commands in our package.json for the devs and startup scripts. You can see the following gist for an example.

Installation and Setup For Node 6.2 & Below

Like most things these days, there is already a package that you can grab that does most of the heavy lifting for you. This package is node-inspector. Install it via the npm command globally via command line :

$ npm i node-inspector -g

Now to ensure that all went well during installation run the inspector command :

$ node-inspector

It should print out a version and a local URL that you can visit.

Hooking up the debugger to your process

Now that the inspector is up and running on your machine you need to hook up your process to it so that it can evaluate the code base as it runs. To accomplish this you will need the process ID of your project after you have started its. 

Start your project using something like : node myProject.js

Or if you use PM2 : pm2 start myProject.js

Getting your PID

I normally run my projects through PM2 which gives you the PID in the process table that it prints out; however if you are not doing that you can find your PID by using the ps command as follows :

$ ps -ax | grep node

That should give you a list of all the node instances that are running on your box at the time in which you can pick out the source file that was started ( myProject.js ). Once you have the PID you can then send the PID a signal that tells the process to enable debugging.

Sending the Debug signal to your process

The process of sending the debug signal is very straight forward. I will use $[pid} where your process id that you found earlier should go. Now lets send that signal :

$ kill -s USR1 ${PID}

Now this won't actually kill your process, we are simply sending a system level signal to it, that i what the -s is for in the command. You are now ready to start debugging your running Node.js application. 

Getting to your debugger

Getting the node-inspector is as easy as visiting the URL that was print out for you near the beginning of the article with one change. By default V8 starts the debugger on port 5858, if for some reason yours is different, or you have multiple debugging sessions going you can tell node-inspector what port you want to hoot the debugger up to by providing a port as a GET param. For Example

http://127.0.0.1:8080/?port=5858

You can change that port param to whatever your process printed when you sent the system signal.

Wrap up

That's it! Pretty simple, yeah? I hope this is something that people will find useful and we can get away from the console.log() times. Debugging will help save you countless hours, especially when trying to determine what variable is now what value when. You just set a break point and watch it flow to the break point, then you can evaluate the entire state of the application at that moment.

Links

PM2 - https://github.com/Unitech/pm2

Node-Inspector - https://www.npmjs.com/package/node-inspector

Debugger docs - https://nodejs.org/en/docs/inspector/


 

Using PM2 and Vorpal to manage a Node.js service stack on developer machines.

Introduction rant

Throughout the various places I've worked, how to setup a development environment has always been a subject of contention and argument. Each developer always has their preference as to what to use to develop which is fine in most cases; however when it comes to how the application you are all working on runs and how consistently it runs that application is of great importance in my mind. 

This comes down to a few simple things that need to be met for development when I am building out a project.

  • consistency 
  • test-ability
  • flexibility
  • developer/ops friendliness. 

It is hard to get all of these, and truth be told you won't get all 4 perfect in any project. The important thing is that you get at least good with all of them. This article came from my setup of a micro service infrastructure that I had developed for a company to support high volume traffic without a service discovery solution. This made both developer and ops management of a distributed computation application with micro services a serious challenge. 

The problem

The issue here is that without a solid service discovery solution in use such as Consul ( An article will come on that later ) but still using micro services there was no consistent way for the developers setup all the services and manage them in a reasonable way. Before I came on board to where this project happened devs had to manually keep track of everything running on their box and they didn't have a choice but to start the entire set of services. It was a situation that wasted a lot of time and effort if anything went wrong in a single service. 

The solution

The solution to this issue is long winded, but included converting Java services into Node.js ones, and introducing PM2 as a process manager/runner. But then the issue of how to create consistency and a service management cropped up for developers.

To this end I embarked to find a reasonable and easy way for developers to manage these services on their boxes. I did a fair amount of tools soul searching for a while before dedicating myself to a type of solution but finally just settled on setting up a command line tool that developers could install via NPM from our internal Nexus. 

The solution ideals

The idea behind the solution is that each service we build has a PM2 config that contains all the details for running that service. On top of that the PM2 Json configuration format allows me to define an instance to be run by name. This is a great setup for developers to be able to run things such as pm2 restart exampleService to manage their environment easily.

The ideal flow is that a developer can checkout any service from our Git servers put them in a single directory and use a tool to generate a PM2 configuration that handles the running of all the services in that directory for them.

The solution code bits

The tool assumes you have a single directory on your developer box with all the services under it. For example : 

  • AllMyServices
    • service1
    • service2

This is so our tool can walk through each service project and extract the configuration for PM2 and use the application definition to build a service cluster configuration. To build the tool I used the following packages and technologies. 

  • Vorpal.js - A Node library for creating command line tools.
  • Babel JS - For modern JS hotness.
  • Node
  • Npm

lets take a look at the actual code behind this tool, be gentle I wrote in about an hour :

Alright, so this isn't perfect, and honestly this is the unpolished but tried and trusted code that is currently being used. But that being said lets take a look at what is happening here. It's a little long winded but pretty straight forward. In essence all this does is :

  • Walks through the directory that it is given look for files named pm2.config.json
  • Loads that configuration file into a JSON object
  • Modifies the pathing so that you can run the configuration in any directory
  • Dumps that config out in the out directory given. 

This will provide you with a PM2 config that can startup and manage each node process that had a pm2.config.json file in the search directory which is pretty sweet. Granted this could use a fair amount of improvement, I won't say otherwise, but I think some of the most useful code is the raw concepts that get a developer moving in the right direction. Lets take a look at a configuration that was generated by this tool : 

If you are familiar with the PM2 configuration structure this will look pretty familiar if not a little boring, but ultimately boring is kind of our goal here; a simple way to manage your node instances. With this configuration you are able to issue commands to specific instance being run, or the entire stack of instances/services.

For example, using this tool to build a config and run things looks like this for one of my own projects that is smaller and doesn't have the support of large infrastructure, it's just a 2 instance application that lives on my own box currently.

As shown above though, it provides a huge amount of usability for a developer as your distributed application grows and becomes more and more separate pieces. With this kind of setup the developer can load the config once and then flip instances off and on as they are needed for development.

Conclusion 

Again PM2 allows us to manage modern Node.js projects with ease and provides us quick ways to build out tools. Though this is not a perfect solution, it has worked great for my team thus far in our state of development. This can also be a great alternative to happening to set up an orchestration system on every developer machine you have which is an absolute nightmare from my experience. There are alternatives to this as always, Docker images being the most common to come up. 

While I love Docker and am currently actually building these services into Docker images for an orchestration system in my current project; I feel that after you hit a threshold of instances that need to exist on the developers box it becomes unmanageable resource requirements wise. 

Being able to create this type of managable ecosystem on your developers machines ultimately leads to more felxability everywhere that your application goes. There are a lot applications here for QA and testing as well, being able to single out specific instances for debugging within the stack or even multiple versions of the same service for debugging becomes an easy and relatively painless task.

Links

  • Vorpal.js - A Node library for creating command line tools.
  • Babel.js - A lovely JS compiler that gives us access to next gen JS
  • PM2 - The lovely Node.js process management tool