Anki Playwright Agents

This deck is a focused set of Q&A cards on setting up and running Playwright tests effectively, covering everything from the minimal Node configuration and where tests live, to more nuanced topics like selector strategy, handling flaky waits, and visual regression snapshots. Whether you're spinning up Playwright for the first time or looking to tighten up an existing suite, the cards walk through practical decisions you'll make in a real project — auto-starting your app before tests, reusing login state across specs, making tests deterministic with controlled data, and updating or masking snapshot baselines.

It's a good fit for developers and QA engineers who already know JavaScript or TypeScript basics and want quick-reference flashcards to keep the everyday Playwright details sharp — things like reading playwright.config.ts settings at a glance, picking the right selector strategy, or knowing when to reach for a data-testid. If you're newer to Playwright, working through the deck once before writing tests can also help you start with the conventions the cards recommend, rather than reinventing them.

Because the material is hands-on, you'll get the most out of these cards by pairing review sessions with time in your editor: after answering a card about, say, the debugging loop or visual regression assertions, try the technique in a small example. Spread reviews over several short sittings rather than cramming, and revisit cards on flaky waits and selector strategy more often — those are the areas where habits tend to drift toward shortcuts that bite you later.

Playwright is a browser automation framework that drives Chromium, Firefox, and WebKit and ships with its own test runner, @playwright/test. To get a project running, install the test package as a dev dependency, install the browser binaries, and run the test command. Once installed, every project's most important file becomes playwright.config.ts, because it controls where tests live (testDir), what browsers run them (projects), how the app starts (webServer), and how failures are reported.

The webServer block is especially useful when an AI agent or a developer needs to drive a full app during tests: it specifies a command, a URL Playwright should ping until it responds, and reuseExistingServer so that locally you can keep a dev server running while CI always spins a fresh one. baseURL lets tests write short paths like page.goto('/login') instead of full URLs, which makes the same suite portable between local, staging, and CI. Environment variables such as BASE_URL, USERNAME, and PASSWORD are read from process.env inside the config so credentials never have to be hardcoded.

Configuration is also where you decide on browser projects. A project is a named browser/device combination, so you can run the same specs under chromium, firefox, webkit, or an emulated mobile profile by configuring devices from the playwright package. Per-project use settings control viewport, locale, timezone, color scheme, permissions, and reduced motion, all of which help keep tests deterministic across machines. Other use keys worth knowing include headless/headed, video (e.g. 'retain-on-failure'), screenshot (e.g. 'only-on-failure'), trace (e.g. 'on-first-retry'), and ignoreHTTPSErrors for local dev environments.

The single biggest source of flaky Playwright tests is bad selectors, so the framework steers you toward accessibility-driven queries. getByRole, getByLabel, getByPlaceholder, and getByText let you describe elements the way a user or screen reader would, which makes tests resilient to DOM refactors and visual redesigns. When semantics genuinely aren't available, such as unlabeled icon buttons or volatile i18n text, the escape hatch is a stable data-testid attribute added in a way that doesn't tie tests to copy or layout. You can rename the attribute globally through testIdAttribute in the config.

A Locator is more than a CSS string: it's a live handle that re-resolves on every action and participates in Playwright's strict mode. Most locator actions throw if more than one element matches, which forces you to disambiguate up front. Common disambiguation patterns include narrowing with .filter({ hasText } or { has: page.getByRole(...) }), scoping with a parent locator like a card or row, or as a last resort using .nth(i). The same scoping idea works inside iframes through frameLocator, where you call getByRole or getByTestId against the frame rather than the top page.

When the UI changes a lot, snapshotting accessibility trees with page.accessibility.snapshot() and asserting on role/name pairs catches a11y regressions in addition to functional ones. Combined with prefers-reduced-motion, disabled animations, fixed viewports, and consistent fonts, this strategy keeps visual tests stable. Where multiple similar elements exist, e.g. rows that share a button, scope the locator to its container first, then run getByRole inside that scope. This habit prevents the dreaded 'strict mode violation' errors that show up the moment a list grows by one row.

Playwright's auto-wait engine waits for elements to be attached, visible, stable, and enabled before acting, and it waits for navigation when appropriate. That means most tests don't need explicit sleeps at all. When you do need to wait, prefer expect(locator) assertions such as toBeVisible, toHaveText, toHaveURL, toHaveValue, toHaveAttribute, toHaveCount, toBeChecked, toBeDisabled, and toBeFocused. These auto-retry up to a configurable timeout and produce much better error messages than arbitrary sleeps.

Some patterns that frequently cause flaky tests are worth avoiding. page.waitForTimeout() should be a last resort, reserved for genuinely non-deterministic animations or third-party widgets you can't otherwise signal. waitForLoadState('networkidle') is rarely reliable because polling apps never actually go idle; prefer waiting for an explicit signal such as a spinner becoming hidden or a specific API response. When the next click triggers navigation, race the click against waitForURL with Promise.all so the navigation event isn't lost. For conditions that don't map to a built-in matcher, expect.poll runs a function repeatedly until it passes.

Soft assertions let you collect multiple failures in a single run via expect.soft, which is useful for broad health checks where one failure shouldn't stop the rest of the report. Grouping steps with test.step('create report', async () => { ... }) produces a cleaner HTML report and makes failures easier to attribute. For dynamic UI like toasts or status banners, target the role='status' element and assert on a regex with toContainText, since exact text often varies by locale. test.setTimeout and test.describe.configure({ timeout }) let you extend limits for slow flows without making the whole suite slow.

Playwright's CLI exposes a rich set of run modes that help both humans and AI agents iterate quickly. npx playwright test runs everything; you can scope to one file (npx playwright test e2e/auth.spec.ts), to a single project (--project=chromium), or to a substring of a test name (-g 'login'). Tags live in the test title, so test('login @smoke', ...) can be selected with --grep @smoke or excluded with --grep-invert @slow. Other useful flags include --headed to watch the browser, --debug (or PWDEBUG=1) to open the inspector, --ui for the interactive UI mode, --list to enumerate tests without running, --workers=1 to disable parallelism, --max-failures=1 to bail early, --retries=2 to retry, --shard=1/3 to split a suite across machines, and --forbid-only to keep test.only out of the main branch.

The fastest way to bootstrap a new test is npx playwright codegen , which records clicks and typing into a spec. Codegen output is rarely production-ready: it tends to pick brittle CSS selectors and weak assertions, so the standard workflow is to record, then refactor toward getByRole/getByLabel and meaningful expect() checks. When recording against a mobile profile, pass --device 'iPhone 13', and when you need to record post-login flows, load a storage state into a persistent context so the generator starts authenticated.

When tests fail, traces are the single most useful artifact. Setting use.trace = 'on-first-retry' (or 'on' locally) records screenshots, DOM snapshots, network, console, and actions into trace.zip, which you can open with npx playwright show-trace path/to/trace.zip. The HTML report itself is opened with npx playwright show-report, and you can attach extra debug info via testInfo.attach('name', { body, contentType }). Pair this with screenshot: 'only-on-failure' and video: 'retain-on-failure' so artifacts only accumulate when something actually went wrong. For local debugging, --headed --debug or page.pause() drops you into the inspector where you can step through actions and re-evaluate locators in real time.

Logging in once and reusing the session across tests is one of the biggest speed and reliability wins in an E2E suite. The standard mechanism is storageState, a JSON file containing cookies and localStorage that Playwright can preload into a context via test.use({ storageState: 'storageState.json' }). The recommended place to create that file is a globalSetup script that programmatically logs in (ideally through a test-only auth endpoint rather than the full UI) and writes the result into an .auth directory under the test project, e.g. e2e/.auth/storageState.json. Real-user secrets must never be committed; read credentials from env vars and treat the state file as an artifact.

Different roles deserve different sessions. Generate storageState.admin.json and storageState.user.json separately, then attach them with test.use({ storageState: ... }) inside the relevant describe blocks or projects. SSO and 2FA flows are notoriously brittle under UI automation, so prefer a storageState created manually once, an API-based test login, or a non-production auth bypass. After login, assert both the URL change and a unique logged-in UI element (such as a user menu) so a false-positive login is hard to slip through. Storage states should be paired with separate browser contexts per test to prevent auth or cookie bleed.

Deterministic tests need deterministic data. Create test-only seed/reset endpoints and call them from the built-in request fixture (await request.post('/api/test/seed', { data })) in test.beforeEach or beforeAll where appropriate. Be cautious with beforeAll for UI state: shared mutable data across workers is a frequent source of flakiness, so prefer per-test isolation or API seeding. For per-test uniqueness, prefix data with a run id (timestamp or UUID) and store it on testInfo or in an env var so cleanup scripts can find what each test created.

Visual regression in Playwright comes from snapshot tests: expect(page).toHaveScreenshot('home.png') captures the page and compares it to a stored baseline, while expect(locator).toHaveScreenshot('card.png') limits the diff to a single component. These are powerful but unforgiving, so the strategy is to pick a small set of high-value pages or components, fix their viewport, theme, and color scheme in the config, disable animations via CSS injection or prefers-reduced-motion, and snapshot them regularly. The smaller and more focused the snapshot, the easier it is to interpret a diff.

Dynamic regions break baseline comparisons, so Playwright supports a mask option that ignores volatile elements when computing the diff: expect(page).toHaveScreenshot({ mask: [page.getByTestId('clock')] }). Other volatile areas worth masking include timestamps, user avatars, ad slots, and charts that re-render with live data. For data regressions instead of visual ones, fetch JSON via the request fixture or page.request and use toMatchSnapshot() on the parsed object so the diff highlights exact field changes rather than pixel noise.

Baselines are updated with npx playwright test --update-snapshots, which is a sensitive operation. Agents should never update snapshots unless explicitly asked; instead they should attach the diff (or the failing screenshot) and propose the change with rationale, citing the ticket or PR that explains the intentional UI change. A safe workflow for intentional UI changes is to update assertions and test IDs first, refresh the snapshot second, and record the reason in the commit message so future diffs are explainable. For comparing across versions, you can check out an older commit in a git worktree, run the same suite, and diff the HTML reports and screenshot artifacts between runs.

A CI pipeline that runs Playwright reliably needs a few essentials. Install browsers with npx playwright install --with-deps so all required system libraries are present, export CI=1 so the config switches to reuseExistingServer:false, and run npx playwright test as the main step. After the run, upload both playwright-report/ and test-results/ as artifacts so the HTML report, traces, screenshots, and videos are available for debugging without needing to reproduce locally. A common configuration pairs JUnit output (reporter: [['junit', { outputFile: 'results.xml' }]]) with the HTML reporter so CI dashboards and humans both have what they need.

Parallelism in Playwright is on by default and multiplies your throughput, but only if tests are isolated. The recipe is per-test data (unique users, unique IDs), no shared mutable globals, idempotent flows, and per-worker browser contexts. Tag tests so you can run fast smoke suites as a gate (--grep @smoke) and heavy flows behind tags like @slow. For multi-machine execution, --shard=1/3 across three jobs is straightforward, and --workers=1 plus --max-failures=1 are handy when reproducing a single flaky failure locally.

Determinism in CI comes from pinning the Playwright version, installing browsers in the job (not assuming a host install), fixing viewport/locale/timezone/color scheme in the config, and blocking or stubbing external dependencies. For speed, run only the projects you need (often chromium-only), keep the smoke suite tight (login, key page navigation, one create/update flow, one critical report), and prefer API seeding plus storageState over multi-minute UI setup. When a failure happens only in CI, the right reflex is to download the trace.zip and screenshots from artifacts and reproduce locally with the same env vars and --workers=1 rather than blindly retrying.

The local workspace contains two main product areas, each with a Vue 3 app, a Vue 2 app, and a Laravel backend. AMS lives under /root/myapp/admin with admin-2-vue (Vue 3), admin-vue (Vue 2), and admin-laravel, while Connect sits under /root/myapp/portal with portal-2-vue, portal-vue, and portal-laravel. Dev server ports are configured in vite.config.ts under server.port and in start_dev_servers.sh; for example, the Connect script starts portal-vue on 3445 and portal-laravel on 7677. A common pitfall is two Vite apps both defaulting to 5173 or 8080, which is solved by passing npm run dev -- --port 3437 (or another free port) so multiple stacks can run side by side.

Pointing Playwright at the right stack is a matter of baseURL and webServer. For an AMS test, baseURL should be the UI you're driving (e.g. admin-2-vue), while seed/reset calls go to the API separately. To cover both AMS and Connect with one suite, create separate Playwright projects with distinct baseURL and webServer settings, then run them together via --project=ams --project=connect. Tests should live under a dedicated e2e/ folder inside the relevant repo and be referenced via testDir in the config so agents know exactly where to add new specs.

Guiding an AI agent such as Codex CLI through Playwright work is mostly about giving it tight, scoped prompts. A useful template is: 'In /root/myapp/[ams|connect]/..., add Playwright tests under e2e/. Start servers via config webServer. Implement login via storageState. Run npx playwright test and show failures.' Equally important are guardrails: 'Do not change app code or update snapshots unless I explicitly ask. Prefer getByRole/getByLabel. If blocked, propose minimum data-testid additions and wait for approval.' Asking the agent to keep changes under a named folder, avoid committing secrets, and to summarize new files, commands, and assertions at the end yields reproducible, reviewable work. For Anki import, the deck itself follows the standard plain-text/TSV format with UTF-8 encoding, optional headers like #separator:Tab, #html:true, #deck:..., #notetype:..., #tags:..., and #tags column:N, where the first field controls duplicate detection on import and tags are space-separated either globally or in a dedicated column.