hey everyone. Git question more than anything else. I recently joined a new org where there is a lot of multi-tasking and context-switching between unrelated tasks. Our Terraform code is in a git repo. So In the morning I could be working on terraform code for, say, the networking components. Then I have to pause it in whatever state it is and switch to deploying, say, ElastiCache. We would create one git branch for the networking task and another for the ElastiCache task. Add in a couple of more concurrent tasks and managing the git branches and keeping the code in all of them organized becomes overwhelming when the feature branches have not been merged. This could mean a test deployment from the ElastiCache branch destroys the unmerged changes from the networking branch and vice versa. All this seems like a mess. What am I doing wrong and how can I make it better?
his could mean a test deployment from the ElastiCache branch destroys the unmerged changes from the networking branch and vice versa.
Are you implying that you’re deploying changes that haven’t been merged into the trunk, and deploys from other branches are overwriting those changes?
yes. To be clear, I’m deploying on a dev environment so destroying things isn’t too much of a concern as long as I can recreate them and not block someone else. But it does add time and effort
this workflow is for testing and bug-fixing our terraform code. We first do it on dev and only when all goes well do we merge it into the trunk
Ah, yeah that’s annoying. I’d probably break the infra down into more templates or repos to have less of this.
Any articles or tips you could share on how to do that? The code is already modularized, so I have one network module, one ElastiCache module, one database module and so on
Use different git repos (for folders) for different stuff… For example, have one for all network related stuff, one for the big (or shared) components (like a shared database).
Then put the application terraform code (the code that creates the resources for a certain application) close to the application itself.
That has been the path for us to scale terraform repos as they tend to not impact the others.
Unless I’m misunderstanding something, it sounds more like the issue is that all those resources are in the same terraform state, so having to switch branches of features will inevitably lead to terraform wanting to destroy things it thinks you’ve removed.
It then poses the question, are all these bits & pieces part of the same logic project, i.e., when this project goes past production, does it make sense that it should be in a single repo. If so, then it sounds like more of a management issue and that the development cycle is far too chaotic and unplanned and that people need to plan first so there’s much less volatility in your infrastructure.
If they should be reasonably separated out into different repos because they aren’t closely connected with each other then it sounds like you’ve probably got infrastructure that’s big enough & complex enough that it warrants splitting up into multiple repos (which comes with its own set of considerations).
Thanks Nuno and Dan. All the components are interrelated and part of the same application
Off the top of my head you could mull over whether some sort of halfway house might be reasonable via merging features which are “good enough, but could still have bugs” could be merged into a dev branch, on the understanding that dev is still very much beta and that merges from dev => staging would need to be properly reviewed and managed (because by that point you’ve lost the specific features as they are all sloshing around in dev… but that feels like making excuses for poor planning + taking the stress on that should be directed to others :shrug:
I actually like the halfway idea as it will be the easiest to implement
Maybe. But I would be careful to make some rules about what merging from a feature branch to dev means… and when it should be allowed and not allowed so people aren’t merging 
to please people/hit deadlines.
n/w good luck with it
thanks, appreciate all the advice!
