Wallet E2E Test API

[ivelin]

The Web3 Frontend Brittleness Problem

Many of today's web3 apps feel brittle, partially because there is no robust e2e test framework API officially supported by wallets. That makes it hard to write end to end test automation covering all steps of end user paths through an app across browsers, wallets and blockchain nets.

There are frequent ongoing UI flow breaks and misplacement or misrepresentation of user's crypto funds due to subpar UX. This is true even for top web3 apps like Uniswap, Axie and Sandbox. As users navigate between different web3 apps and interact with multiple blockchain networks and wallets, these web3 apps often get out of sync with the state of the current wallet and as a result, the user ends up looking at unexpected content.

Given web3's promise to better protect user's identity, privacy and provide more control over user's own assets, it is imperative to offer superior, trustworthy UX. That in turn requires improving on test automation practices and infrastructure.

Current Workarounds

Test frameworks like dappeteer and synpress are trying to bridge the gap, but they feel brittle as they are constantly chasing MetaMask's changing front end code. This test framework brittleness frustrates web3 developers who are used to sophisticated web2 tooling. Writing web3 e2e tests that frequently break due to underlying base layer problems, results in postponing these test automation initiatives, which then results in brittle web3 frontend apps and subpar web3 UX.

The following snippet is an example of web3 e2e test frameworks "screen scraping" metamask code.

const firstTimeFlowFormPage = '.first-time-flow__form';
const secretWordsInput = `${firstTimeFlowFormPage} .first-time-flow__seedphrase input`;
const passwordInput = `${firstTimeFlowFormPage} #password`;
const confirmPasswordInput = `${firstTimeFlowFormPage} #confirm-password`;
const termsCheckbox = `${firstTimeFlowFormPage} .first-time-flow__terms`;
const importButton = `${firstTimeFlowFormPage} .first-time-flow__button`;
const newPasswordInput = `${firstTimeFlowFormPage} #create-password`;
const newSignupCheckbox = `${firstTimeFlowFormPage} .first-time-flow__checkbox`;

and another example:

  const addressInput = await page.waitForSelector('#custom-address');
  addressInput.type(tokenAddress);

  await page.waitForTimeout(4000);

  const nextButton = await page.waitForSelector(`button[data-testid='page-container-footer-next']:not([disabled])`);
  await nextButton.click();

  const footerButtons = await page.$$('footer > button');

While web3 app test developers are looking for reliable API that looks like this:

  const browser = await dappeteer.launch(puppeteer, { metamaskVersion: 'v10.1.1' });
  const metamask = await dappeteer.setupMetamask(browser);

  // you can change the network if you want
  await metamask.switchNetwork('ropsten');

  // you can import a token
  await metamask.addToken('0x4f96fe3b7a6cf9725f59d353f723c1bdb64ca6aa');

  // go to a dapp and do something that prompts MetaMask to confirm a transaction
  const page = await browser.newPage();
  await page.goto('http://my-dapp.com');
  const payButton = await page.$('#pay-with-eth');
  await payButton.click();

  // 🏌
  await metamask.confirmTransaction();

Proposed Solution

Start a small working group with folks from: Ethers, MetaMask, WalletConnect, OlympusDAO and any other interested wallet or web3 app projects to work towards an E2E test API that benefits everyone:

This issue was also raised with the MetaMask, WalletConnect, Brave and RainbowKit communities:
MetaMask/test-dapp#147
https://community.brave.com/t/e2e-testing-for-web3-app-interactions-with-brave-wallet/318181
WalletConnect/walletconnect-monorepo#683
rainbow-me/rainbow#3630