If you are thinking of implementing some type of callback URL for a pipeline action, endpoint, or something more complex like an API Gateway, you probably may run in the problem of not knowing which tool to use. Well, then this is the tutorial for you.
In this tutorial, you will learn how to run a small bash script in a server by just hitting a URL. We will use a simple example to demonstrate this. Through this method, the HTTP server itself will execute the script without any help of any proxied backend, such as Node.js or PHP.
For this, we’ll use Alibaba’s Tengine HTTP server with the Lua module.
What Is Lua, Tengine, and OpenResty?
According to their website, Lua “is a powerful, efficient, lightweight, embeddable scripting language. It supports procedural programming, object-oriented programming, functional programming, data-driven programming, and data description”. Sounds like a pretty simple and amazing programming language, yeah? You may know Lua from games like Minecraft, which lets you do some crazy stuff like building computers and robots inside the game.
By combining Lua with Tengine (or Nginx to be technical) you are essentially using OpenResty. OpenResty is, like Tengine, an open-source project started by Alibaba. It was, at first, sponsored by Taobao.com before the year 2011 and it was mainly supported by Cloudflare Inc. from 2012 until 2016. Presently it is mainly supported by OpenResty Software Foundation and OpenResty Inc.
All this is thanks to Alibaba’s efforts contributing to the open-source community. Right now, Alibaba is one of the top companies contributing across GitHub projects. This is a great thing for the end user. It shows how important and mature the open source community is.
What Do We Want to Achieve?
Well, lately, we have been having a lot of fun with legacy applications and CI/CD pipelines. In this example, we will be running a small bash script located in an Alibaba Cloud ECS instance, which, in our case, needs to get triggered every time a build passes an automated stage in DevOps. For this, it will send a request to an HTTP endpoint which we will define in our Tengine config file.
But why a bash script, you may ask? For the sake of simplicity, we will imagine that there is a need to integrate this system with a legacy application. This is not unusual with old systems, in fact, where a cramped virtual machine is full of scripts and services and turning it into microservices takes more effort and resources than what may be worth it.
So, our bash script contains the logic needed to keep track of the build statuses when they happen. For this, we will put Alibaba’s Tengine web server to listen for requests and a config file (In Lua) which will define what script to run.
Let’s do it!
To avoid compiling Tengine with the ngx_http_lua_module on your own and us potentially getting lost installing too many developer tools (and even not really knowing how to properly compile it), I prepared a ready-to-use Docker Image of Tengine with Lua already built-in. Even the team at OpenResty admits that “It is discouraged to build this module with Nginx yourself since it is tricky to set up exactly right”. So, no worries, I already went through all the pain for you!
To install Docker you will need to check how to do it with the Operating System you are using, so please visit the Docker Engine documentation to learn how.
After installing Docker, the best way to test if our Docker installation fully works is by pulling the Docker image I prepared with Tengine+Lua from the Docker registry. For this, just run docker pull roura/tengine:lua and wait until it completes without error. The message you want to see is Status: Downloaded newer image for roura/tengine:lua.
Now, we assume the bash script we need to run with this is located at /opt/tracking/build-stage.sh.
To be able to run the script, we will mount the file itself as a volume (/opt/tracking/build-stage.sh:/script.sh) when running the Tengine container. This is a way to share the script between the host machine and the container, so both can run it with different contexts.
Lua-ing
Is this your first time with Lua? Well no worries there, there is always a first time for everything. The good thing is this is a good chance for you to learn, as the script we need to write is very simple. Another good thing is that, as I’ve mentioned before, you will get exposed to OpenResty, as this combination of using Tengine with Lua can, essentially, be called an “OpenResty distribution”.
Tengine Config
This file, mounted in /etc/nginx/conf.d/default.conf, will contain the following content:
server {
location / {
add_header Content-Type text/html;
content_by_lua_block {
ngx.say("CI/CD Stage Reader")
}
}
location /stage {
set $expected_token "set_your_security_token_here";
content_by_lua_file /opt/bash.lua;
}
}
Lua File
This file, mounted in /opt/bash.lua, has the following content:
if ngx.var.arg_stage == nil or ngx.var.arg_success == nil or ngx.var.arg_token == nil then
ngx.exit(ngx.HTTP_BAD_REQUEST)
elseif ngx.var.arg_token ~= ngx.var.expected_token then
ngx.exit(ngx.HTTP_FORBIDDEN)
end
local handle = io.popen('/script.sh ' .. ngx.var.arg_stage .. ' ' .. ngx.var.arg_success)
local bash_output = handle:read('*a')
handle:close()
ngx.say(bash_output)
bash Script
This will be the script that will process your request at the end and will give an output like this:
#!/usr/bin/env bash
# TODO: Run your logic here.
# output
echo "STAGE: $1. SUCCESS: $2"
Running the example
As you may have noticed, the Lua script is expecting a GET parameter called stage and another called success, which is the build stage reported by your CI/CD software and if it was a success or not. Another parameter is the token, defined as token and defined in the Tengine file as expected_token.
An example request would be http:/example.org/stage?token=my_token&stage=testing&success=true
Conclusion
If you followed the instructions outlined here, you should now have a better understanding of how to quickly prototype or create an API with this not-so-common tool, so that you can build from proof of concept to production-ready solutions. Enjoy!