Models

Test Type Definitions

We use this to ... so that we can ...

Unit Test (UT)

  • Low-level, focusing on a small part of the software such as a Java class or Angular component together with associated collaborators

  • Usually written by developers or perhaps software engineers in test, using their regular IDEs and testing frameworks such as IntelliJ, JUnit, VS Code and JEST

  • Run faster than any other kind of test, allowing a suite of thousands of unit tests to be run in a few seconds

  • Very scalable

  • Automatable

  • The primary facilitating mechanism for refactoring

Integration Test (INT)

  • Determine if independently developed units of software work correctly when they are connected to each other.

  • Focusing on larger groups of collaborating Java classes or Angular components together with I/O dependencies such as databases or network connections.

  • I/O dependencies should not made at the expense of execution speed - for example a fast running in memory or local docker database can be leveraged over a hosted development database or a Contract Test to mock out an API endpoint

  • Usually written by developers or perhaps software engineers in test, using their regular IDEs and testing frameworks such as IntelliJ and junit or VS Code and jasmine.

  • Run fast allowing a suite of hundreds or thousands (depending on I/O dependencies) of integration tests to be run in a few seconds

  • Very scalable

  • Primary facilitating mechanism for refactoring

Contract Test (CT)

  • Similar to an integration test used specifically for the purpose of ensuring that a pair of applications will work correctly together by checking each application in isolation to ensure the messages it sends or receives conform to a shared understanding that is documented in a "contract”.

  • The network dependency is mocked by the contract broker so contract tests run fast

  • Usually written by developers or perhaps software engineers in test, using their regular IDEs and testing frameworks such as IntelliJ and junit or VS Code and jasmine.

  • Run fast allowing a suite of hundreds or thousands of contract tests to be run in a few seconds

  • Very scalable

  • Primary facilitating mechanism for refactoring interdependent services independently of one another

Exploratory Test

  • An exploratory test is one which emphasizes a rapid cycle of learning, test design, and test execution. Rather than trying to verify that the software conforms to a pre-written test script, an exploratory test explores the characteristics of the software, raising discoveries that will then be classified as reasonable behaviour or failures.

Acceptance Test (AT)

  • Testing based on story card acceptance criteria to enable the tester (manual) or developer (automated) to determine whether or not the software is ready for consumption by the customer.

  • Scoped to a particular story only

  • Performed from the customer perspective - usually a UI (eg browser or DISC terminal) or an API client (eg CTM integration)

  • Maximum context and value with respect to customer requirement - significant determining factor in confidence of story definition of done

  • Slower than integration test as a local service together with supporting out of process collaborating services are necessary to exercise the code

  • As such, these tests are harder to maintain and scale

System Test (ST)

  • Similar to an Acceptance Test but instead requires live versions of one or many deployed services in hosted environments, requiring test environment and network access

  • Teams currently need to wait overnight before test subject is deployed to NXI or NXQ

  • Exercises code paths through one or many services and dependent services

  • Usually written by developers or perhaps software engineers in test, using their regular IDEs and testing frameworks such as IntelliJ and junit or VS Code and jasmine.

  • Run duration is an order of magnitude slower and largely determined by the scope of the execution path through the dependent services. A single system test could take between one and several seconds to run.

End to End Test (E2E)

  • Confirming that full user journey processes can be completed throughout the entire value stream. E.g Establish customer, Create/amend/service or renew a policy and lodge and process a claim

  • Scoped to a customer value stream

  • Performed from the customer perspective - usually a UI (eg browser or DISC terminal) or an API client (eg CTM integration)

  • Maximum context and value with respect to customer requirement - significant determining factor in level of confidence of working software

  • Very slow to run - depending on scope of test, execution duration can minutes to tens of minutes

  • Hard to maintain

  • Does not scale

  • Depending on implementation, can be repurposed for load testing

  • Automated or Manual

Penetration Test (PEN)

  • Authorised simulated cyber attack on a computer system, performed to evaluate the security of the system.

  • Automated or Manual

Synthetic Test (SYN)

  • Method of understanding how our call centre agents and customers experience our applications by proactively testing public production environment endpoints and 3rd party APIs such as payment gateways.

  • Automated

References

Last updated