Most organizations, especially the older more enterprise ones, are crippled with environment complexity - what started as Dev / QA / Prod quickly had a stage environment added, maybe a UAT, then perhaps a few QA environments for various versions.  Sitting on top of those numerous environments may be race conditions - who can test when - as well as challenges with getting data, content, dependencies, etc in the right environment.

The problem here is that this view on 'move my application through environments' is fundamentally flawed.  The environments were a response to an unknown.  We need to be able to test this outside of a dev workstation - create a qa environment.  We need to load test this - create a pre-prod environment...and so on.

 

 

You can think about this problem differently.

Imagine a world where your application did not move through environments - it moved through tests.  What would that look like?  What problems would that solve?

If nothing else - it starts the right discussion sooner.  What do we want to verify when?  What do we need to test that?

This is how I lead my continuous delivery and devops training and consulting.  I never focus on 'automate everything' - automating complexity is silly and frustrating.

And through this, I find that most places simply don't know how to test.  That is not a knock on the people, just the discussions aren't happening.

 

  • For my work, what are my assumptions and what do I need to verify? (Unit tests)
  • For this user experience / functionality - what do I need to verify?  (Story Tests)
  • For this to functionality to work in the system, what do I depend on / what depends on me, and how will I know if it is working? (Integration tests)
  • What assumptions do I need to validate about the environment and what may change? (Infrastructure Tests)
  • What data do I need to verify? Hint - more than likely it isn't 'All of Production Data'
By no means is the above list complete.  I want to provide a window of what it can look like when you stop worrying about environments and start working back to the root - how do you test and what do you need to verify?
Would this help you?