Terraform is a great tool for provisioning infrastructure. One of the best practices that evolve over time as you play more with Terraform, is a directory structure that works best for your project. Some prefer having each component in its own directory so that modification and destruction of resources is easy, while others treat a software stack (e.g. 1 ELB + 2 instances + 1 Elasticache) as a logical unit which should reside together.
The aim of this post is not to re-enforce that idea because that topic has already been written about in depth. Gruntwork’s articles and Reddit discussions give a fair insight into the trade offs of each approach.
But one thing that is common across all approaches is that you will not have a single, large main.tf file housing all of your resources. You will have to create a maintainable, extensible, intuitive directory structure. I follow the below structure:
The variable info is spread across 3 files when we are creating infra in stage/app/ or stage/base/:
This way we don’t create a very granular directory structure (each component in its own directory) and also limit the coupling (I shouldn’t worry about changing VPC when I want to mess around with the app).
Hence, we have to provide all of them while creating our Terraform plan:
As you can see when we are in stage/service1/app we will have to run our Teraform plans like so:
Side note: I never export AWS_PROFILE because I want to be sure on every run that I am running Terraform commands against the right AWS account.
If someone else wants to run the same plan, they need to understand the directory structure (which they should) and specify the lengthy init, plan, apply commands (which they shouldn’t). The flexibility of spreading vars trades off the complexity on the command line.
What is the Makefile ?
I didn’t know MakeFile four months back, but as I got to know, I liked it so much that everything I do with Python, is automated now. Here are few use cases, for which I always go to the Make command:
If you don’t want to remember any long command with multiple arguments then create a Make command in MakeFile and you are good to go.
If you want to run a set of commands one after another but don’t want to copy-paste, again and again, place that code in a block in the Make command
If there is anything that needs to run by many systems, instead of giving a command list, just give Make command.
For non-developers, who don’t need to know the technicalities of execution of a product, Make command could be used as a handy execution command
Creating a cheat sheet to document instructions and don’t want to put blocks of executable scripts, then create Make commands and write those in the cheatsheet
Want to provide installation steps to few people without any mistake, use the Make command
I can literally go on, Use Cases are endless and if you still have not tried Make commands in MakeFile, then I insist “Go, Try and Create Magic”
What exactly is the Make command:
Make command is a rule that can be set for a script or block of commands. Consider it as a one-word rule that you would set to execute a bunch of commands in series.
The format of setting the rule is
target: prerequisite
recipe
example of a make rule
Target is the one-word rule, or multiple words concatenated to make one word. Target is the name that will be used to call that rule. An example here is ‘bye’
Prerequisites are optional, these are dependencies, if you want any file to run or other make command to run before this target, then add that as a prerequisite. The prerequisite will run first before the target rule runs. An example as in the screenshot is ‘hello
The recipe is the main recipe of that rule, whatever is planned to add in the rule for execution should be added to the recipe. An example for ***Recipe is ‘***echo “bye, see you next time”’
Having the above explanation, now we see that this is where Makefile comes to our rescue.
Terraform and MakeFile
Having a simple Makefile in our target Terraform directories ensures uniformity of approach and eliminates the need to refer to documentation on which are the right commands to run. Now our commands become:
$ AWS_PROFILE=xyz make help
$ AWS_PROFILE=xyz make init
$ AWS_PROFILE=xyz make plan
$ AWS_PROFILE=xyz make apply
This comes with the added benefit of validation i.e. this section ensures that the pre-requisite environment variables are set before running your Terraform commands.
That’s it. Use Makefiles and avail the flexibility without documenting lengthy commands.
Update: Makefiles are useful and there are many people using it. Came across a very comprehensive Terraform Makefile — do check it out.
Conclusion
We looked today at making our workflow more easy making use of Makefiles. For any questions, please leave a comment below.
#YouAreAwesome #StayAwesome
Stay tuned and follow me for more updates. Don’t forget to give us your 👏 if you enjoy reading the article as a support to your author!!
