Web development has grown at a tremendous pace with lots of automation testing frameworks coming in for both front-end and backend development. Websites have become smarter and so have the underlying tools and frameworks. With a significant surge in the web development area, browsers have also become smarter. Nowadays, you can find headless browsers, where users can interact with the browser without a GUI. You can even scrape websites in headless browsers using packages like Puppeteer and Node.js.
Efficient web development hugely relies on a testing mechanism for quality assessment before we can push code to production environments. Headless browsers can perform end-to-end testing, smoke testing, etc. at a faster speed, as it is free from overhead memory space required for the UI. Moreover, studies have proved that headless browsers generate more traffic than the non-automated ones. Popular browsers like Chrome can even help in debugging web pages in real-time, analyze performance, notify devs of memory consumption, enable developers to tweak their code and analyze performance in real-time, etc.
Is this evolution of browsers heading towards a smarter web development process? In this post, we will give an overview on headless browsers and understand how they help create a smarter and faster website development process.
What Is a Headless Browser?
The Need for a Headless Browser
Enables Faster Web Testing Using Command Line Interface
With headless cross-browser testing, we are saved from using overhead memory consumption in the GUI, hence it enables faster website testing, using the command line as the primary source of interaction. Headless browsers are designed to execute crucial test cases like end-to-end testing which ensures that the flow of an application is performing as designed from start to finish. Headless browsers cater to this use case as they enable faster website testing.
The headless browser saves the overhead of opening the GUI, thus enabling faster scraping of websites. In headless browsers we can automate the scraping mechanism and extract the data in a much more optimized manner.
Taking Web Screenshots
Though the headless browsers do not avail any GUI, they do allow the users to take snapshots of the website that they are rendering. It’s very useful in cases where the tester is testing the website and needs to visualize the code effects and save the results in the form of screenshots. In a headless browser you can easily take a large number of screenshots without any actual UI.
Mapping User Journey Across the Websites
Headless browsers allow you to programmatically map the customer journey test cases. Here, headless browsers help users to optimize their experience throughout the decision making journey on your website.
Now that we understand headless browsers and their numerous features, along with the their key quality of being lightweight browsers which help in accelerating the speed of testing, let’s look at the most popular headless browser, Headless Chrome, and what it unlocks for developers.
Diving Into Headless Chrome and Chrome DevTools
There are a number of headless browsers out there, such as Firefox 55 and 56, PhantomJs, Html Unit, Splinter, jBrowserDriver, etc. Chrome 59 is Chrome’s answer to headless browsers. Headless Chrome and Chrome DevTools are quite a powerful combination which give users great out-of-the-box features. So let’s have a look at Headless Chrome and Chrome DevTools.
What Is Headless Chrome?
Headless Chrome is basically just a regular Chrome browser running in a headless environment without a GUI. This light weight, memory sparing, and quick running browser brings all the features provided by Chromium and Blink rendering engines to the command line.
Automated browsers have always generated more traffic than the non-automated ones. In a recent survey, it was discovered that headless Chrome generated more traffic than the previous leader in the headless space, Phantum Js, within a year of its release.
Apart from this, there are several reasons why Chrome is the most popular headless browser. One of the reasons is that it’s always updating out of the box features, which constantly introduce new trends in web development. It also consists of a rendering engine called Blink, which constantly updates itself as the website evolves. Headless Chrome gives developers the ability to:
- Test the latest web platform features like ES6 modules, service workers, and streams.
- Programmatically tap into the Chrome DevTools and make use of awesome features like network throttling, device emulating, website performance analysis, etc.
- Test multiple levels of navigation.
- Gather page information.
- Take screenshots.
- Create PDFs.
Now let’s have a look on the most common flags you need to know to start working with headless Chrome.
For starting a headless instance, you need Chrome 59+ and to open the Chrome binary from the command line. If you have Chrome 59+ installed, then start the Chrome with the headless flag,
To print the DOM, create a PDF, or take screenshots, we can use the following flags:
- Printing the DOM: The
- Create a PDF: The
–print-to-pdfflag creates a PDF of the page.
- Taking Screenshots: To capture a screenshot of a page, use the
Debugging a Code Without the Browser UI
If you want to debug your code in a headless browser using Chrome’s DevTools then make note of the following flag:
–remote-debugging-port=9222. This flag helps you to open headless Chrome in a special mode, wherein Chrome DevTools can interact with the headless browser to edit the web page during run-time. We will dig deeper into Chrome Devtools in the later section of the blog.
For debugging a web page with Chrome DevTools, use the
What Are Chrome DevTools?
Chrome DevTools are a set of web developer tools built directly into Google Chrome. It helps to debug web pages on the fly and detect the bugs quickly, which ultimately helps to develop websites faster.
As we can see, Chrome DevTools is a package of diverse functionalities which helps with debugging a web page in the Chrome browser. But what about the headless browser with no UI, how can we debug the web page with no UI?. Can Chrome DevTools help debug a headless browser? Let’s demystify the ways to debug a headless browser with Chrome DevTools, discuss what is puppeteer in the following sections of the blog.
As discussed earlier, one of the ways to debug a web page in a headless browser is to use the flag
–remote-debugging-port=9222 in the command line, which helps you to tap into Chrome DevTools programmatically. But, there is another way to use headless Chrome to perform numerous out-of-the box tasks and make use of headless in a more efficient way. This is where Puppeteer comes into the picture.
Puppeteer is a Node Library which provides a high level API to control Chrome over DevTools protocol. Puppeteer is usually headless but can also be configured to use full non-headless Chrome. It provides full access to all the features of headless Chrome and can also fully run Chrome in a remote server, which is very beneficial for automation teams. It would be quite accurate to call Puppeteer the Google Chrome team’s official Chrome headless browser.
One of the greatest advantages of using Puppeteer as an automation framework for testing is that, unlike other frameworks, it is very simple and easy to install.
The Following Code Snippet Will Help You to Install Node.js
Once you are done with installation of Node.js to your machine, you can run the following flag to install puppeteer:
npm i puppeteer
With this you completed, the installation process for Puppeteer which will also, by default, download the latest version of Chrome.
Why Is Puppeteer So Useful?
Puppeteer provides full access to all the out-of-box features provided the headless Chrome and its constantly updating rendering engine called Blink. Unlike other automation testing frameworks for web applications, like Selenium Web Driver, Puppeteer is so popular because it provides automation for a light weight (UI-less), headless browser which helps to perform testing faster. Likewise, there are multiple functionalities provided by Puppeteer. Let’s have a look at them. Puppeteer can:
- Help generate screenshots and PDFs of pages.
- Crawl a single page application and generate pre-rendered content.
- Automate form submissions, UI testing, end-to-end testing, smoke testing, keyboard input, etc.
- Capture a timeline trace of your site and analyze any performance issues.
- Test Chrome extensions.
- Allow you to perform web scraping.
Now that we’ve gone over what Puppeteer can do, let’s have a look at the code for taking screenshot in Puppeteer.
Once this code gets executed, a screenshot will be saved to your system through the path mentioned in the code snippet.
Faster and better web development has always been and will always be the top priority of QA and development teams. The headless browsers (without GUI) being light weight and easy on memory means it can run at high speed while automation testing. They cater to the need for smarter web development. Moreover, they help in testing all modern web platform features, as well as enabling debugging and performance analysis in real-time. They are responsible for load balancing heavy traffic in web applications and support website scraping with the help of npm packages like Puppeteer. Furthermore, the installation of headless browsers is easier than the installation of other web automation frameworks.