A Quick Guide to Sanity Testing for My Fellow Testers

As a software testing guru with over 10 years of expertise testing websites and apps on thousands of browser and device combinations, one question I often get asked by budding testers is: "What exactly is sanity testing and when should you do it"?

Let me start by giving you a quick overview of sanity testing.

Sanity testing, also called smoke testing, is a preliminary testing activity performed by testers whenever a new software build is received from the development team.

The goal is to verify that after any code changes, the critical functions of the application still work fine without major defects. I like to think of it as a quick health checkup of the critical functional parts before doing an MRI scan during comprehensive testing.

Now that you know what it is at a high level, let’s dig deeper into the specifics of sanity testing…

Why Do We Need Sanity Testing?

As an application evolves, new features get added, existing flows get updated, defects get fixed – leading to frequent build changes.

In a recent survey conducted by Gartner, enterprises deploy new software builds on average every 2 weeks. Some even have multiple deployments in a single week reflecting the pace of agile development!

While getting rapid feedback is great, this also means plenty of scope for unintended defects and regressions to creep in impacting application stability.

This is why sanity testing is crucial before embarking on heavy-duty comprehensive testing.

Imagine your team spends weeks methodically testing thousands of scenarios in the latest build only to realize basic critical functions like login, payments, checkout are broken.

That would be super frustrating and a tremendous waste of testing effort and time!

Sanity testing allows your team to quickly validate if the vital parts of your app are working fine before actual detailed testing.

It helps answer questions like:

• Is it reasonable to test further?
• Should we abort and request a new build fix?
• What next steps should we take?

Just like visiting your doctor for a quick health exam before getting lab tests or scans, sanity testing delivers immense value.

It provides a rapid assessment of quality and confidence to move forward.

Key Objectives of Sanity Testing

The core goals of sanity testing include:

Verify critical functionality – Check if the key features like login, payments, search etc. work reasonably without bothering too much about details.

Save time and effort – By testing basic functions early, you prevent wasted downstream testing efforts if something is totally broken.

Guarantee stability – Certify business critical flows like checkout have no ridiculous defects before actual end-to-end tests.

Confirm bug fixes – Validate that high severity bugs fixed in the new release actually work fine as expected.

Inform decisions – Provide insights into the stability of the build to decide next steps – whether to proceed, rebuild or investigate further.

The ultimate purpose is to give a quick first-look at business critical aspects before exhaustive testing.

Let me break down exactly how the sanity testing process works…

How Sanity Testing Works Step-By-Step

Over the years, I’ve refined an optimal 6 step framework that I guide my teams through for smoothly executing sanity tests:

Step 1 – Analyze Scope

Firstly, analyze what major areas in the system were updated in the new build based on release notes from devs.

Step 2 – Identify Priority Functions

Next, based on experience select 2-5 important flows to test e.g user login/logout, checkout, payment confirmation.

Step 3 – Design Test Cases

Prepare test cases focused on the critical flows without bothering too much about exhaustive negative pathways for now.

Step 4 – Configure Test Environment

Setup the latest build in the staging environment across different device/browser combinations.

Step 5 – Execute & Record

Run through priority test cases while recording clear bug reports with steps to reproduce.

Step 6 – Retest & Recommend

If bugs were raised, verify fixes once new build is deployed and recommend next steps around more testing.

And that’s really it! The process does not have to be complicated.

Let the comprehensive testing come later. Sanity testing is all about a quick first pass on priority functionality.

Now you may be wondering – how does sanity testing differ from other testing types?

Sanity Testing vs Other Testing Types

Sanity testing is often confused with smoke testing. While they sound similar, the scope is different:

Testing Type	What‘s Tested	When Performed	Test Depth	Fail Criteria
Sanity Testing	Checks critical components and flows	After new build deployment	Thoroughly tests fewer functions	Fails if critical functions broken
Smoke Testing	End-to-end system test	After initial code integration	Broadly tests entire system	Fails if complete system is non-functional

As you can notice:

• Sanity testing examines priority functionality more deeply after code changes

• Smoke testing does broad system level checks before sanity testing

So in a nutshell:

Sanity Testing → Smoke Testing → Actual Detailed Testing

Got it? Let’s look at a quick example to drive this home…

Sanity Testing By Example

Imagine an ecommerce site. The dev team claims to have fixed defects around the checkout payment page and releases a new build.

What would be the sanity testing approach?

Step 1 – Understand Scope

Analyze release notes – payment page fixes were done recently.

Step 2 – Choose Priority Flows

Checkout payment is critical function to validate if working fine.

Step 3 – Design Test Cases

Create test case – validate successful order placement with payment.

Step 4 – Setup Environment

Access latest build on test site across mobile, tablet and desktop browsers.

Step 5 – Run Tests & Capture Results

Run through checkout payment flows on all devices, log any errors, capture screenshots.

Step 6 – Retest & Recommend

Inform devs of defects with steps to reproduce. Retest fixes after new deployment.

So you see with sanity testing we are laser focused on quickly testing the critical flows before diving into comprehensive scenarios.

Now that you understand the basics, let me share some pro tips…

Tips for Efficient Sanity Testing

Over the decade, I’ve picked up some handy strategies for smoothly executing sanity tests:

Start early when builds are stable – Don‘t wait too long between builds to prevent defect pile up.

Define tight scope – Restrict to 2-5 key functions only, avoid scope creep.

Document clearly – Note environment, steps to reproduce to ease diagnosis.

Time box testing – Keep testing time short since this is just a quick check.

Involve developers – Have devs available during testing to fix issues quicker.

These tips will help your team perform focused sanity tests rapidly. The benefits are well worth it…

The Benefits of Sanity Testing

Some key advantages your team can realize by leveraging sanity testing:

Saved cost – Early detection of defects reduces downstream debugging costs significantly. A study by IBM states fixing bugs early is 100x cheaper.

Faster feedback and deployment – Issues identified early leads to quicker turnaround between builds.

Informed business decisions – Gives leadership confidence that critical functionality at least works without ridiculous issues.

Efficient use of resources – Saves tons of hours wasted testing an unstable build with tons of defects.

Reduced technical debt – Catches defects early preventing significant quality erosion hard to fix later.

Optimized release timing – Deploy code changes when business demands rather than waiting to test extensively.

As you can see, sanity testing offers remarkable benefits for barely a few hours of focused testing.

However, it does come with some challenges…

The Challenges with Sanity Testing

As useful as sanity testing seems, it has some limitations to be aware of:

Incomplete coverage – Shallow testing of core functionality only may miss edge case defects.

No detailed root cause – Quick fail report may lack sufficient info to diagnose issues thoroughly.

No test documentation – Adhoc testing lacking formal test cases lowers reusability.

No confirmation testing – Related functionality not deeply confirmed which may cause side-effects.

No complex flows – Intricate user journeys seldom accounted during cursory testing phase.

Thankfully there are some handy tools that help mitigate these challenges…

Handy Tools for Effective Sanity Testing

BrowserStack is my weapon of choice when it comes to testing internally.

It allows my team to:

✅ Test across multiple devices/browsers – Sanity check UI and flows on 3000+ real mobile devices and browsers in BrowserStack’s cloud for reliable results rather than just my local desktop browser.

✅ Debug quickly – Inspect elements, console logs, network requests on any device to fix issues faster even from remote locations.

✅ Track and report – Log bugs, annotate screenshots and group related defects with labels to coordinate efficiently.

✅ Auto screenshots – Every test step auto-captures screenshots sequentially even for dynamic content to simplify reporting.

✅Share access – Provide secure dashboard access via public links to clients and devs for full transparency.

The ability to validate functionality quickly across platforms along with multi-user access, reporting and debugging features makes BrowserStack a fantastic fit for rapidly executing sanity tests.

Give BrowserStack a try or schedule a demo to see how it can enable your team to perform sanity testing with maximum efficiency!

So in summary, we covered a lot of ground around sanity testing including:

🔹 What is it and why it‘s needed

🔹 How it differs from smoke testing

🔹 Exact step-by-step process to carry it out

🔹 Tips to do it effectively and tools that help

I hope this guide gave you clarity around what sanity testing entails!

Feel free to reach out in the comments section if any other questions come to mind. I‘m glad to help as you embed sanity testing in your projects.

Here‘s to building quality into your software releases! 🥂