The Testing Iceberg

Not that long ago I had a conversation with @mattwynne, which led to the hastily sketched piece of paper, below. The diagram on the left (since redrawn and blogged about by @tooky) shows the relationship between End-to-end tests and Business-readable tests. Not all Business-readable tests need to be End-to-end and not all End-to-end tests need to be business readable.


The middle part of the sketch is my attempt to show the relative size of these sets of tests. There should be far more Business-readable tests than End-to-end tests. It also shows that most End-to-end tests are Business-readable because there are very few situations where purely technical concerns require anything broader than integration tests. The point is, where possible, test the domain model directly and only use End-to-end tests to verify correct ‘wiring up’ of the entire system.

The far right of the sketch attempts to relate the Venn diagram to the well known Testing Pyramid. Business-readable tests that hit the domain model directly map to the middle section of the pyramid – integration/component tests. Business-readable tests that hit the full stack map to the top of the pyramid. Not shown is where non-business-readable End-to-end tests should map.

At this point I’m going to re-imagine the Testing Pyramid as a Testing Iceberg (another product of conversations with @mattwynne). Those portions of the iceberg above the waterline are business readable, while those below are not. As you can see, in this diagram there are examples of all test types both above and below the readability waterline.


Now I can map non-business-readable End-to-end tests to the submerged system test portion of the iceberg, which is very small because most End-to-end tests should be business-readable. Some projects may have specific technical concerns that can only be validated using a fully deployed system, and that are of no interest to business people, but these will be few and far between.

You may be wondering why I consider the Testing Iceberg important? In my next blog I’m going to talk about lack of trust between developers, testers and business people and I think the Testing Iceberg will help us talk about how to address this. I’ll also explain why some of the tests that verify components in isolation (‘unit’ tests) are above the waterline.





6 responses to “The Testing Iceberg”

  1. Carlos Ble Avatar

    Thanks for this article Seb. I heard of it in the last CukeUp! edition (it was you or Matt) and am going to link to this article in a TDD training manual that I am working on.
    See you soon in the Agile Testing Days 🙂

  2. Nick Jenkins Avatar

    Lack of trust is an interesting issue to look at. In one large enterprise I worked in, we had no less than five separate testing teams that used to check each others work. The “business” didn’t trust IT, infrastructure didn’t trust apps, operations didn’t trust apps or infrastructure etc. Needless to say this was untenable and inefficient but once in place was almost impossible to deconstruct. Testing isn’t an end in itself, quality is an end and testing a possible means to get there. Everything you deploy in the pursuit of quality needs to be judged on a cost-benefit spectrum, it’s no longer good enough to use the “risk avoidance” argument to justify the non-value add costs of testing…

  3. Vikram VI Avatar
    Vikram VI

    Hi Seb,

    I didn’t understand terms “domain model” , “full stack”, “workflow format”

    can you please give more info about these.


    1. Administrator Avatar

      Hi Vikram,

      I don’t maintain this website – sorry for the delay.

      Domain model –
      Full stack –
      Workflow format – tests that document a whole user journey/business process, rather than tests that focus on a single behaviour.


  4. […]   ( Testing Iceberg ,  Not all Business-readable tests need to be End-to-end and not all End-to-end tests need to be business readable. ) […]

  5. […] test. Visualizing this distinction on the testing pyramid makes our pyramid looking more like a testing iceberg, where the business-interesting tests are the ones that are above the […]

Leave a Reply

Your email address will not be published. Required fields are marked *