How can I parse HTML using Playwright without navigating a page?

Ask a Question
,
1

I’m working on a test automation project using Playwright in a Windows 10 environment with Node.js version 14. I need to parse some raw HTML content without loading it as a full webpage. I tried using the standard page.goto method, but it seems overkill and not suitable for my case. I also looked into some internal methods like parseHTML, but I’m not sure how to apply them correctly. What would be the best approach to parse raw HTML strings using Playwright?

2

Ran into the same challenge while trying to process HTML directly with Playwright, so you’ve got company!

Playwright typically interacts with live pages, so getting it to parse raw HTML without navigation needs a tweak. Start by loading the content into a new page directly in a browser context and then leverage the standard DOM methods to interact with it.

Here is the snippet that worked for me:

const { chromium } = require('playwright');
(async () => {
  const browser = await chromium.launch();
  const context = await browser.newContext();
  const page = await context.newPage();
  const htmlContent = '<html><body><h1>Hello, World!</h1></body></html>';
  await page.setContent(htmlContent);
  const heading = await page.$eval('h1', element => element.innerText);
  console.log(heading);
  await browser.close();
})();

This code launches a Chromium browser, sets the raw HTML as page content, and extracts the innerText of the <h1> element. Watch out for selector specifics since different HTML structures require different targeting strategies.

By using page.setContent, you sidestep full navigation, maintaining performance. Just ensure your HTML string is correctly formatted to avoid misparsing issues. If dealing with external dependencies, remember that this won’t load linked resources such as scripts or styles unless they are injected explicitly.

3

Initially faced issues trying to parse HTML with Playwright using the common navigation methods. It didn’t fit my requirement as I was dealing with multiple dynamically generated HTML snippets.

Upon digging deeper, I realized setting up a virtual browser environment to evaluate the HTML in isolation can be incredibly effective. You can use Playwright’s ability to set raw HTML directly into a page instance without navigating.

Here is a practical example:

const { firefox } = require('playwright');
(async () => {
  const browser = await firefox.launch();
  const context = await browser.newContext();
  const page = await context.newPage();
  const rawHtml = '<div id="content"><p>Sample Text</p></div>';
  await page.setContent(rawHtml);
  const textContent = await page.$eval('#content p', el => el.textContent);
  console.log(textContent);
  await browser.close();
})();

This snippet uses Firefox, showing how different browser engines handle the same Playwright commands. The script injects HTML, evaluates the page to read the <p> tag’s text content, and logs it. Take care of any JavaScript that the snippet might depend on, adjusting understanding that CSS won’t affect evaluation.

Where this shines is its adaptability. If you’re dealing with JSON strings converted to HTML, this method can parse them too. Explore the page.evaluate capability further to match complex use cases.