From Zero to Hero with Puppeteer – The Complete Guide

Hi there! I‘m Asif, a seasoned test automation consultant with over 12 years of experience in the field. Today I‘m excited to give you a full lowdown on mastering browser testing end-to-end using Puppeteer and JavaScript.

My Background in Test Automation

I‘ve been fortunate enough to have worked on 200+ test automation initiatives for clients across banking, retail, healthcare and technology verticals. The app landscape today with SPAs, progressive web apps and high degree of asynchronous flows requires sophisticated tools compared to traditional Selenium suited for simple document-based systems.

Out of the thousands of devices and browsers in my integrated lab infrastructure, I‘ve found Puppeteer to be invaluable for scaling test coverage and speeding up feedback cycles. Lightweight yet powerful, Puppeteer has emerged as my tool of choice for testing modern web apps.

Through this hands-on tutorial, I‘ll be sharing all my tips and tricks to help you get productive with Puppeteer as well!

What to Expect from This Tutorial

We‘ll start from the basics like setting up Puppeteer from scratch, write our first scripts and then gradually level up to advanced concepts like:

• Dealing with auth flows, popups and form entries
• Mocking API calls and test data
• Automating complex user journeys
• CI/CD integrations
• Debugging tools and best practices
• Alternative tools and their capabilities

I‘ll be using simple analogies, crisply annotated code samples and recommendations tailored to your context wherever relevant.

Let‘s begin our exciting Puppeteer journey!

Test Automation Today

The changing technology landscape has put greater demand on test automation than ever before:

• Adoption of latest web stacks like React, Angular with complex asynchronous user flows
• Rising popularity of mobile apps, electron desktop apps
• Rapid iterations and continuous delivery

||Selenium|Puppeteer| Playwright|
|-|————-|—————|————-|
|Architecture|Client-server, needs webdriver|Direct devtools access|Bundled browser binary|
|Language Support| Multiple languages |JavaScript/TypeScript only|Multiple languages|
|Speed|Slow due to remote execution|Very fast without browser driver|On par with Puppeteer|

As seen above, Puppeteer eliminates the browser driver bottleneck to enable reliable high-speed test execution.

With roots in the Chrome DevTools team, Puppeteer delivers blazing performance by leveraging the cross-platform DevTools Protocol instead of traditional Selenium bindings. Let‘s dig deeper and see what makes it tick!

Key Benefits of Using Puppeteer

As a headless Chrome node API, Puppeteer provides full control of Chrome (and Chromium) via automated interactions exposed through DevTools. Some benefits this unlocks:

• Direct control eliminates need for WebDriver server
• Automatic waiting and synchronization built into API methods
• Supports latest web features like flexbox, grid, shadows etc
• Enables testing progressive web apps, extensions, electron apps etc.
• Allows screenshot, PDF generation, crawling SPA and more!

These capabilities and so much more make Puppeteer a versatile tool for testers. It‘s actively maintained by Chrome teams at Google with over 18k GitHub stars so lots of continued momentum.

Ok enough background, let‘s get your hands dirty by installing Puppeteer and writing some code!

Prerequisites

To follow along, you would need:

• Node.js v14+: Required to run JavaScript
• Code editor like VSCode: I‘d highly recommend VSCode 😊
• Basics of async/await: We‘ll leverage it heavily

Optional but recommended:

• JavaScript fundamentals
• Familiarity with terminal usage
• Existing test automation experience

Let‘s execute step-by-step to configure Puppeteer on your machine next.

Installing Puppeteer

The easiest way is via npm package manager:

Step 1: Create a new folder for your tests and cd into it

mkdir puppeteer-demo 
cd puppeteer-demo

Step 2: Initialize npm project manifest

npm init -y

Step 3: Install Puppeteer package

npm i puppeteer 

This will create a package.json file with Puppeteer added under dependencies.

Some other helpful packages I recommend installing:

• jest for assertions and organizing test suites
• jest-puppeteer for integration
• dotenv to inject secrets

npm i jest jest-puppeteer dotenv

And we‘re set! Let‘s start exploring some Puppeteer fundamentals next.

Core Concepts

Before we start scripting tests, you need to get familiar with some key concepts:

The Browser vs Browser Contexts vs Pages

The browser instance encapsulates a Chromium browser session. Browser contexts are like incognito windows isolating state. Pages load web content and can manipulate loaded pages.

Pages enable you to interact with stuff like DOM, service workers etc. Many pages can exist in a single context and many contexts in a browser.

Lifecycle of Puppeteer Resources

It‘s vital to manage the lifecycles of resources properly:

• Browsers should be closed manually
• Contexts should be disposed when not needed
• Pages should be closed post use for optimal performance

// Lifecycle example

const browser = await puppeteer.launch(); 

const context = await browser.createIncognitoBrowserContext();

const page = await context.newPage();

await page.goto(‘https://www.example.com‘);

// Test logic goes here  

await page.close();
await context.close();
await browser.close();

Gracefully releasing resources avoids leaks and downstream issues.

Now that you have the basics down, let‘s start using Puppeteer for what it‘s best at – automated browser testing!

Our First Test Script

We‘ll write a simple script to validate broken link checking on a page.

Step 1: Create a tests folder

Step 2: Add a test file brokenLinks.test.js

Step 3: Launch a new page and navigate:

const page = await browser.newPage();
await page.goto(‘http://www.example.com‘); 

Step 4: Get all anchor tags and verify links:

const anchors = await page.$$(‘a‘);

for(let a of anchors) {
  const href = await page.evaluate(anchor => anchor.href, a);

  const resp = await page.request.head(href);

  expect(resp.status()).toBe(200); 
}

This showcases some common patterns like:

• Using page.$ and page.$$ to query elements
• Extracting properties via page.evaluate
• Asserting responses using Jest

Let‘s build on these fundamentals and tackle some real-world scenarios!

Dealing with Forms

Submitting forms is one of the most common actions for users. But as automation engineers, dealing with form elements can prove challenging.

Here is an example script to handle form input and validation dynamically:

// Fill username  
await page.type(‘#username‘, ‘john.doe‘);

// Enter secure password
await page.type(‘#password‘, ‘123456‘, {sensitive: true});  

// Upload file for avatar 
const [fileChooser] = await Promise.all([
  page.waitForFileChooser(),
  page.click(‘#avatar‘) 
]);

await fileChooser.accept([‘./my-pic.jpg‘]);

// Submit form  
await Promise.all([
  page.waitForNavigation(), 
  page.click(‘.submit-btn‘)  
])

// Verify page loaded 
expect(page.url()).toContain(‘/dashboard‘);

This covers:

• Inputting text into elements
• Secure password entry
• Uploading files via file chooser
• Waiting for navigation to complete
• Asserting redirect to next page

Some key things to note:

• Use Promise.all to parallelize dependent operations
• Wait for file chooser dialog to appear before uploading
• Wait for navigation explicitly after form submit clicks

There are some other complex scenarios like multi-select, rich text editors etc. that require special handling as well.

Now that you‘ve seen examples of how to use Puppeteer for testing typical app flows, let‘s shift gears and talk about executing at scale.

Setting up Puppeteer in CI/CD

While writing Puppeteer scripts on your local is simple enough, running them efficiently in the cloud as part of continuous delivery pipelines requires some additional steps.

Here is a recommended GitHub Actions workflow for executing parallel test suites on CI:

name: Puppeteer Tests  

on: [push]

jobs:

  test:

    timeout-minutes: 60  
    runs-on: ubuntu-latest

    strategy:
      matrix:        
        node: [14, 16]

    steps:
    - uses: actions/checkout@v3
    - uses: actions/setup-node@v3
      with: 
        node-version: ${{ matrix.node }}
    - run: npm ci
    - run: npm run test:headless
      env: 
        # Pass CI flag to disable visuals 
        HEADLESS: true

    - uses: actions/upload-artifact@v3
      if: ${{ always() }}
      with:
        name: puppeteer-screenshots
        path: test-results/screenshots

   # Upload videos to external provider 
    - run: node scripts/upload-videos.js
      env:
        SAS_URL: ${{ secrets.AZURE_SAS_URL }}

Some key things configured:

• Install dependencies via npm ci for consistency
• Run tests headless by default
• Upload screenshots and videos for debugging
• Execute across multiple runtime versions in parallel

There are tons of other optimizations like sharding tests across processes, integrating reporting etc. With GitHub Actions giving free minutes monthly, I‘d highly recommend leveraging it over self-managed infrastructure to start with.

Alright, we‘ve covered a lot of ground working with real test examples. Let‘s wrap up with some best practices!

Concluding Thoughts

We‘ve covered a whole gamut of topics in this action-packed Puppeteer adventure! Here are some key takeaways:

• Structure test scripts well using describe() and it() blocks
• Follow the page objects pattern for maximum reusability
• Use helpers for common utilities like CLI output, file uploads etc
• Enable headless mode selectively for CI runs
• Analyze browser logs to debug issues early
• Run Lighthouse to catch performance regressions

I hope you‘ve found this comprehensive tutorial helpful. Please feel free to reach out if you have any other questions.

Excited to see what you build with your newfound Puppeteer superpowers!