BLV AP task gallery

Keyboard-only reproductions of accessibility-property tasks, shown in paired agent modes with task, flow, outcome, and usage provenance.

10 accessibility properties 20 runs 2 modes per property Batch: ap-gallery-full20-flow-grounded-20260808

AP-FM1-01: Focused control invalidated; focus loses meaningful continuation

A11y-tree navigation only

ap-fm1-01-a11y-nav

MP4
Harness completedtrue
Task outcome (agent-reported)failed
steps
14
prompt_tokens
140853
prompt_cached_tokens
41472
completion_tokens
3456
total_tokens
144309

flows.txt provenance

flow_file
${BLV_GENERATED_APPS_ROOT}/expedia/replicate-01/g0-none/app/flows.txt
flow_section
additional
flow_index
3
flow_title
Complete a realistic mock booking.
Exact task
Mode rule: use only keyboard and Orca navigation actions. Use the compact accessibility tree as a semantic map only; do not use mouse, click, coordinate, selector, direct accessibility-node activation, DOM-query, browser navigation buttons, extraction, or hidden implementation details.

Property under demo: AP-FM1-01 - Focused control invalidated; focus loses meaningful continuation.

Complete a realistic mock booking.

Source workflow steps: Search any travel category and choose Select, Reserve, or Choose on a result. Review the item, enter traveler name and confirmation email, and click Confirm booking. A unique `EXP-` confirmation notification appears, and the booking is then listed under Upcoming trips on `/trips`.

Use only keyboard actions. When the source workflow says click, choose, open, select, or activate, perform the equivalent keyboard action with Tab, Shift+Tab, Enter, Space, arrow keys, typing, or waiting. Do not use mouse, click, coordinate, or hidden browser-inspection actions. Finish when the workflow is completed or the specified confirmation/result is visible.
Agent-reported final result
NAVIGATION_RELATED_ERROR: The booking could not be completed. The focused Reserve button was reachable, but neither Enter nor Space produced an observable result after waiting. Pressing Tab afterward moved focus to the Expedia footer link, so focus lost meaningful continuation instead of reaching the booking review form. No EXP- confirmation or Upcoming trips listing was observed.

Keyboard + vision assisted

ap-fm1-01-vision

MP4
Harness completedtrue
Task outcome (agent-reported)completed
steps
49
prompt_tokens
334453
prompt_cached_tokens
0
completion_tokens
9097
total_tokens
343550

flows.txt provenance

flow_file
${BLV_GENERATED_APPS_ROOT}/expedia/replicate-01/g0-none/app/flows.txt
flow_section
additional
flow_index
3
flow_title
Complete a realistic mock booking.
Exact task
Mode rule: use only keyboard actions. Do not use mouse, click, coordinate, DOM-query, browser navigation buttons, extraction, or hidden implementation details.

Property under demo: AP-FM1-01 - Focused control invalidated; focus loses meaningful continuation.

Complete a realistic mock booking.

Source workflow steps: Search any travel category and choose Select, Reserve, or Choose on a result. Review the item, enter traveler name and confirmation email, and click Confirm booking. A unique `EXP-` confirmation notification appears, and the booking is then listed under Upcoming trips on `/trips`.

Use only keyboard actions. When the source workflow says click, choose, open, select, or activate, perform the equivalent keyboard action with Tab, Shift+Tab, Enter, Space, arrow keys, typing, or waiting. Do not use mouse, click, coordinate, or hidden browser-inspection actions. Finish when the workflow is completed or the specified confirmation/result is visible.
Agent-reported final result
Booking workflow completed successfully using keyboard actions only.

- Item: Sunlit Orlando Villa with Private Pool
- Traveler: Jordan Lee
- Confirmation email: jordan.lee@example.com
- Total: $1716
- Confirmation: EXP-004821
- Result: Listed as Confirmed under Upcoming trips on /trips

AP-FM1-01 was reproduced: when the Review and book modal opened, focus remained on a background result control instead of moving into the dialog. Keyboard focus then traversed the remaining background results and footer links before finally reaching the modal’s Close control and traveler fields, so focus lost meaningful continuation.

AP-M1-01: Dialog lacks usable accessible name

A11y-tree navigation only

ap-m1-01-a11y-nav

MP4
Harness completedtrue
Task outcome (agent-reported)completed
steps
14
prompt_tokens
135570
prompt_cached_tokens
48384
completion_tokens
3009
total_tokens
138579

flows.txt provenance

flow_file
${BLV_GENERATED_APPS_ROOT}/imdb/replicate-01/g1-generic-a11y/app/flows.txt
flow_section
required
flow_index
21
flow_title
Give Terminator 2: Judgment Day a rating of 10.
Exact task
Mode rule: use only keyboard and Orca navigation actions. Use the compact accessibility tree as a semantic map only; do not use mouse, click, coordinate, selector, direct accessibility-node activation, DOM-query, browser navigation buttons, extraction, or hidden implementation details.

Property under demo: AP-M1-01 - Dialog lacks usable accessible name.

Give Terminator 2: Judgment Day a rating of 10.

Source workflow steps: Search “Terminator 2,” open the title, select Rate, and choose 10 in the rating dialog. The title page updates Your Rating to 10/10 and shows a confirmation status.

Use only keyboard actions. When the source workflow says click, choose, open, select, or activate, perform the equivalent keyboard action with Tab, Shift+Tab, Enter, Space, arrow keys, typing, or waiting. Do not use mouse, click, coordinate, or hidden browser-inspection actions. Finish when the workflow is completed or the specified confirmation/result is visible.
Agent-reported final result
Completed using keyboard and Orca navigation only. Terminator 2: Judgment Day now shows Your Rating as 10/10. The rating dialog was announced without a usable accessible name, consistent with AP-M1-01.

Keyboard + vision assisted

ap-m1-01-vision

MP4
Harness completedtrue
Task outcome (agent-reported)completed
steps
22
prompt_tokens
124122
prompt_cached_tokens
0
completion_tokens
4011
total_tokens
128133

flows.txt provenance

flow_file
${BLV_GENERATED_APPS_ROOT}/imdb/replicate-01/g1-generic-a11y/app/flows.txt
flow_section
required
flow_index
21
flow_title
Give Terminator 2: Judgment Day a rating of 10.
Exact task
Mode rule: use only keyboard actions. Do not use mouse, click, coordinate, DOM-query, browser navigation buttons, extraction, or hidden implementation details.

Property under demo: AP-M1-01 - Dialog lacks usable accessible name.

Give Terminator 2: Judgment Day a rating of 10.

Source workflow steps: Search “Terminator 2,” open the title, select Rate, and choose 10 in the rating dialog. The title page updates Your Rating to 10/10 and shows a confirmation status.

Use only keyboard actions. When the source workflow says click, choose, open, select, or activate, perform the equivalent keyboard action with Tab, Shift+Tab, Enter, Space, arrow keys, typing, or waiting. Do not use mouse, click, coordinate, or hidden browser-inspection actions. Finish when the workflow is completed or the specified confirmation/result is visible.
Agent-reported final result
Completed using keyboard-only actions. Terminator 2: Judgment Day now shows Your Rating: 10/10, with the confirmation status: “Your rating for Terminator 2: Judgment Day: 10/10.”

AP-M1-03: Escape does not close exact active dialog

A11y-tree navigation only

ap-m1-03-a11y-nav

MP4
Harness completedtrue
Task outcome (agent-reported)completed
steps
8
prompt_tokens
78039
prompt_cached_tokens
27648
completion_tokens
2221
total_tokens
80260

flows.txt provenance

flow_file
${BLV_GENERATED_APPS_ROOT}/nfl/replicate-01/g1-generic-a11y/app/flows.txt
flow_section
required
flow_index
1
flow_title
Watch the halftime show from the most recent Super Bowl
Exact task
Mode rule: use only keyboard and Orca navigation actions. Use the compact accessibility tree as a semantic map only; do not use mouse, click, coordinate, selector, direct accessibility-node activation, DOM-query, browser navigation buttons, extraction, or hidden implementation details.

Property under demo: AP-M1-03 - Escape does not close exact active dialog.

Watch the halftime show from the most recent Super Bowl

Source workflow steps: - Visit `/videos` from the Videos navigation item. - Select the “Halftime Show” category, then choose the featured “Super Bowl LVIII Halftime Show: Usher’s full performance” play control. - A full-screen mock player opens and visibly reports “NOW PLAYING”; its pause/resume and close controls work.

Use only keyboard actions. When the source workflow says click, choose, open, select, or activate, perform the equivalent keyboard action with Tab, Shift+Tab, Enter, Space, arrow keys, typing, or waiting. Do not use mouse, click, coordinate, or hidden browser-inspection actions. Finish when the workflow is completed or the specified confirmation/result is visible.
Agent-reported final result
Completed the keyboard-only workflow. The full-screen mock player for “Super Bowl LVIII Halftime Show: Usher’s full performance” is open and reports “NOW PLAYING.” The dialog also exposes “Close video” and “Pause video” controls.

Keyboard + vision assisted

ap-m1-03-vision

MP4
Harness completedtrue
Task outcome (agent-reported)completed
steps
17
prompt_tokens
91709
prompt_cached_tokens
0
completion_tokens
3388
total_tokens
95097

flows.txt provenance

flow_file
${BLV_GENERATED_APPS_ROOT}/nfl/replicate-01/g1-generic-a11y/app/flows.txt
flow_section
required
flow_index
1
flow_title
Watch the halftime show from the most recent Super Bowl
Exact task
Mode rule: use only keyboard actions. Do not use mouse, click, coordinate, DOM-query, browser navigation buttons, extraction, or hidden implementation details.

Property under demo: AP-M1-03 - Escape does not close exact active dialog.

Watch the halftime show from the most recent Super Bowl

Source workflow steps: - Visit `/videos` from the Videos navigation item. - Select the “Halftime Show” category, then choose the featured “Super Bowl LVIII Halftime Show: Usher’s full performance” play control. - A full-screen mock player opens and visibly reports “NOW PLAYING”; its pause/resume and close controls work.

Use only keyboard actions. When the source workflow says click, choose, open, select, or activate, perform the equivalent keyboard action with Tab, Shift+Tab, Enter, Space, arrow keys, typing, or waiting. Do not use mouse, click, coordinate, or hidden browser-inspection actions. Finish when the workflow is completed or the specified confirmation/result is visible.
Agent-reported final result
Workflow completed using keyboard only. Selected “Halftime Show,” opened “Super Bowl LVIII Halftime Show: Usher’s full performance,” and verified the full-screen mock player displayed “NOW PLAYING.” Confirmed AP-M1-03: Escape did not close the active dialog. The pause control changed the player to “PAUSED” with “Resume video,” resume restored playback, and the “Close video” control successfully closed the player.

AP-M1-04: Dialog closes but focus is not restored to the invoker

A11y-tree navigation only

ap-m1-04-a11y-nav

MP4
Harness completedtrue
Task outcome (agent-reported)completed
steps
8
prompt_tokens
79007
prompt_cached_tokens
41472
completion_tokens
1859
total_tokens
80866

flows.txt provenance

flow_file
${BLV_GENERATED_APPS_ROOT}/nfl/replicate-01/g1-generic-a11y/app/flows.txt
flow_section
additional
flow_index
8
flow_title
Use global search
Exact task
Mode rule: use only keyboard and Orca navigation actions. Use the compact accessibility tree as a semantic map only; do not use mouse, click, coordinate, selector, direct accessibility-node activation, DOM-query, browser navigation buttons, extraction, or hidden implementation details.

Property under demo: AP-M1-04 - Dialog closes but focus is not restored to the invoker.

Use global search

Source workflow steps: - Activate Search in the header (or press `/`), type a team, player, game, or video query, and choose a matching result. The app navigates directly to the appropriate working route.

Use only keyboard actions. When the source workflow says click, choose, open, select, or activate, perform the equivalent keyboard action with Tab, Shift+Tab, Enter, Space, arrow keys, typing, or waiting. Do not use mouse, click, coordinate, or hidden browser-inspection actions. Finish when the workflow is completed or the specified confirmation/result is visible.
Agent-reported final result
Workflow completed and AP-M1-04 was reproduced. I opened global Search from the header, entered “Chiefs,” and activated “Super Bowl LIV 2020 · Chiefs 31, 49ers 20.” The app navigated to `/scores?season=2020`, showing Super Bowl LIV with Chiefs 31 and 49ers 20. After the search dialog closed, focus was on the document body rather than restored to the Search invoker.

Keyboard + vision assisted

ap-m1-04-vision

MP4
Harness completedtrue
Task outcome (agent-reported)completed
steps
15
prompt_tokens
77990
prompt_cached_tokens
0
completion_tokens
2360
total_tokens
80350

flows.txt provenance

flow_file
${BLV_GENERATED_APPS_ROOT}/nfl/replicate-01/g1-generic-a11y/app/flows.txt
flow_section
additional
flow_index
8
flow_title
Use global search
Exact task
Mode rule: use only keyboard actions. Do not use mouse, click, coordinate, DOM-query, browser navigation buttons, extraction, or hidden implementation details.

Property under demo: AP-M1-04 - Dialog closes but focus is not restored to the invoker.

Use global search

Source workflow steps: - Activate Search in the header (or press `/`), type a team, player, game, or video query, and choose a matching result. The app navigates directly to the appropriate working route.

Use only keyboard actions. When the source workflow says click, choose, open, select, or activate, perform the equivalent keyboard action with Tab, Shift+Tab, Enter, Space, arrow keys, typing, or waiting. Do not use mouse, click, coordinate, or hidden browser-inspection actions. Finish when the workflow is completed or the specified confirmation/result is visible.
Agent-reported final result
Completed the global search workflow using only keyboard actions. Selected “Tom Brady — Career passing statistics,” which navigated to `/players?q=Tom%20Brady`; the Tom Brady player profile is visible. AP-M1-04 was confirmed: after the search dialog closed, focus was on the page body rather than restored to the Search invoker.

AP-M1-05: New modal does not receive focus

A11y-tree navigation only

ap-m1-05-a11y-nav

MP4
Harness completedtrue
Task outcome (agent-reported)failed
steps
8
prompt_tokens
77952
prompt_cached_tokens
34560
completion_tokens
2073
total_tokens
80025

flows.txt provenance

flow_file
${BLV_GENERATED_APPS_ROOT}/nfl/replicate-01/g1-generic-a11y/app/flows.txt
flow_section
required
flow_index
1
flow_title
Watch the halftime show from the most recent Super Bowl
Exact task
Mode rule: use only keyboard and Orca navigation actions. Use the compact accessibility tree as a semantic map only; do not use mouse, click, coordinate, selector, direct accessibility-node activation, DOM-query, browser navigation buttons, extraction, or hidden implementation details.

Property under demo: AP-M1-05 - New modal does not receive focus.

Watch the halftime show from the most recent Super Bowl

Source workflow steps: - Visit `/videos` from the Videos navigation item. - Select the “Halftime Show” category, then choose the featured “Super Bowl LVIII Halftime Show: Usher’s full performance” play control. - A full-screen mock player opens and visibly reports “NOW PLAYING”; its pause/resume and close controls work.

Use only keyboard actions. When the source workflow says click, choose, open, select, or activate, perform the equivalent keyboard action with Tab, Shift+Tab, Enter, Space, arrow keys, typing, or waiting. Do not use mouse, click, coordinate, or hidden browser-inspection actions. Finish when the workflow is completed or the specified confirmation/result is visible.
Agent-reported final result
NAVIGATION_RELATED_ERROR: The Super Bowl LVIII Halftime Show mock player opened and visibly reports “NOW PLAYING,” but the new dialog did not receive focus. Focus remained on the underlying “Play Super Bowl LVIII halftime show” button, confirmed by focused-element metadata and Orca’s announcement. The dialog exposes “Close video” and “Pause video” controls in the accessibility tree, but focus was not moved into the modal, so their keyboard operation could not be reliably verified.

Keyboard + vision assisted

ap-m1-05-vision

MP4
Harness completedtrue
Task outcome (agent-reported)failed
steps
16
prompt_tokens
62414
prompt_cached_tokens
0
completion_tokens
3050
total_tokens
65464

flows.txt provenance

flow_file
${BLV_GENERATED_APPS_ROOT}/nfl/replicate-01/g1-generic-a11y/app/flows.txt
flow_section
required
flow_index
1
flow_title
Watch the halftime show from the most recent Super Bowl
Exact task
Mode rule: use only keyboard actions. Do not use mouse, click, coordinate, DOM-query, browser navigation buttons, extraction, or hidden implementation details.

Property under demo: AP-M1-05 - New modal does not receive focus.

Watch the halftime show from the most recent Super Bowl

Source workflow steps: - Visit `/videos` from the Videos navigation item. - Select the “Halftime Show” category, then choose the featured “Super Bowl LVIII Halftime Show: Usher’s full performance” play control. - A full-screen mock player opens and visibly reports “NOW PLAYING”; its pause/resume and close controls work.

Use only keyboard actions. When the source workflow says click, choose, open, select, or activate, perform the equivalent keyboard action with Tab, Shift+Tab, Enter, Space, arrow keys, typing, or waiting. Do not use mouse, click, coordinate, or hidden browser-inspection actions. Finish when the workflow is completed or the specified confirmation/result is visible.
Agent-reported final result
Opened the featured “Super Bowl LVIII Halftime Show: Usher’s full performance” player using only the keyboard and confirmed the visible “NOW PLAYING” state. AP-M1-05 was reproduced: focus remained on the background “Play Super Bowl LVIII halftime show” control instead of moving into the modal. Tab and Shift+Tab did not reach the visible Pause or Close controls, and Escape did not close the player. Therefore, pause/resume and close could not be operated by keyboard.

AP-N1-01: Navigation structure lacks usable anchors

A11y-tree navigation only

ap-n1-01-a11y-nav

MP4
Harness completedtrue
Task outcome (agent-reported)completed
steps
11
prompt_tokens
106091
prompt_cached_tokens
48384
completion_tokens
2367
total_tokens
108458

flows.txt provenance

flow_file
${BLV_GENERATED_APPS_ROOT}/allrecipes/replicate-01/g1-generic-a11y/app/flows.txt
flow_section
additional
flow_index
1
flow_title
Mock account login and logout.
Exact task
Mode rule: use only keyboard and Orca navigation actions. Use the compact accessibility tree as a semantic map only; do not use mouse, click, coordinate, selector, direct accessibility-node activation, DOM-query, browser navigation buttons, extraction, or hidden implementation details.

Property under demo: AP-N1-01 - Navigation structure lacks usable anchors.

Mock account login and logout.

Source workflow steps: Visit `/account`, enter any valid email and a password of at least four characters, then select Log in. The Alex Cook profile is shown. Select Log out to return to the login form.

Use only keyboard actions. When the source workflow says click, choose, open, select, or activate, perform the equivalent keyboard action with Tab, Shift+Tab, Enter, Space, arrow keys, typing, or waiting. Do not use mouse, click, coordinate, or hidden browser-inspection actions. Finish when the workflow is completed or the specified confirmation/result is visible.
Agent-reported final result
Completed the mock account workflow using keyboard and Orca navigation only. Logged in with alex@example.com and an eight-character password, verified the Alex Cook profile, activated Log out, and verified the login form returned with the status “You are now logged out.” Diagnostic observation: Orca heading navigation produced no speech for the login/profile headings, so the controls had to be reached through the Tab order.

Keyboard + vision assisted

ap-n1-01-vision

MP4
Harness completedtrue
Task outcome (agent-reported)completed
steps
12
prompt_tokens
62023
prompt_cached_tokens
0
completion_tokens
1883
total_tokens
63906

flows.txt provenance

flow_file
${BLV_GENERATED_APPS_ROOT}/allrecipes/replicate-01/g1-generic-a11y/app/flows.txt
flow_section
additional
flow_index
1
flow_title
Mock account login and logout.
Exact task
Mode rule: use only keyboard actions. Do not use mouse, click, coordinate, DOM-query, browser navigation buttons, extraction, or hidden implementation details.

Property under demo: AP-N1-01 - Navigation structure lacks usable anchors.

Mock account login and logout.

Source workflow steps: Visit `/account`, enter any valid email and a password of at least four characters, then select Log in. The Alex Cook profile is shown. Select Log out to return to the login form.

Use only keyboard actions. When the source workflow says click, choose, open, select, or activate, perform the equivalent keyboard action with Tab, Shift+Tab, Enter, Space, arrow keys, typing, or waiting. Do not use mouse, click, coordinate, or hidden browser-inspection actions. Finish when the workflow is completed or the specified confirmation/result is visible.
Agent-reported final result
Workflow completed using keyboard actions only. Logged in successfully, verified the Alex Cook profile, then activated Log out. The login form returned and the confirmation “You are now logged out” is visible.

AP-N2-01: Long page lacks a usable bypass

A11y-tree navigation only

ap-n2-01-a11y-nav

MP4
Harness completedtrue
Task outcome (agent-reported)completed
steps
32
prompt_tokens
319024
prompt_cached_tokens
152064
completion_tokens
7399
total_tokens
326423

flows.txt provenance

flow_file
${BLV_GENERATED_APPS_ROOT}/bestbuy/replicate-01/g0-none/app/flows.txt
flow_section
required
flow_index
2
flow_title
View Bluetooth and wireless speakers on sale under $50.
Exact task
Mode rule: use only keyboard and Orca navigation actions. Use the compact accessibility tree as a semantic map only; do not use mouse, click, coordinate, selector, direct accessibility-node activation, DOM-query, browser navigation buttons, extraction, or hidden implementation details.

Property under demo: AP-N2-01 - Long page lacks a usable bypass.

View Bluetooth and wireless speakers on sale under $50.

Source workflow steps: Search “speaker.” In Filters, enable Bluetooth, Wireless, and On Sale, then enter 50 as Maximum price. The results update to the qualifying Insignia and JBL speakers, both below $50.

Use only keyboard actions. When the source workflow says click, choose, open, select, or activate, perform the equivalent keyboard action with Tab, Shift+Tab, Enter, Space, arrow keys, typing, or waiting. Do not use mouse, click, coordinate, or hidden browser-inspection actions. Finish when the workflow is completed or the specified confirmation/result is visible.
Agent-reported final result
Completed the workflow using keyboard and Orca navigation only.

Verified:
- Search: “speaker”
- On Sale: checked
- Bluetooth: checked
- Wireless: checked
- Maximum price: 50
- Results: 2 items
  - Insignia™ - Mini Bluetooth Wireless Speaker - Black
  - JBL - Go Essential Portable Bluetooth Wireless Speaker

The individual displayed prices were not announced by Orca during the available reading attempts, but both products are present in the results with the Maximum price filter set to 50.

Keyboard + vision assisted

ap-n2-01-vision

MP4
Harness completedtrue
Task outcome (agent-reported)completed
steps
29
prompt_tokens
190383
prompt_cached_tokens
0
completion_tokens
5170
total_tokens
195553

flows.txt provenance

flow_file
${BLV_GENERATED_APPS_ROOT}/bestbuy/replicate-01/g0-none/app/flows.txt
flow_section
required
flow_index
2
flow_title
View Bluetooth and wireless speakers on sale under $50.
Exact task
Mode rule: use only keyboard actions. Do not use mouse, click, coordinate, DOM-query, browser navigation buttons, extraction, or hidden implementation details.

Property under demo: AP-N2-01 - Long page lacks a usable bypass.

View Bluetooth and wireless speakers on sale under $50.

Source workflow steps: Search “speaker.” In Filters, enable Bluetooth, Wireless, and On Sale, then enter 50 as Maximum price. The results update to the qualifying Insignia and JBL speakers, both below $50.

Use only keyboard actions. When the source workflow says click, choose, open, select, or activate, perform the equivalent keyboard action with Tab, Shift+Tab, Enter, Space, arrow keys, typing, or waiting. Do not use mouse, click, coordinate, or hidden browser-inspection actions. Finish when the workflow is completed or the specified confirmation/result is visible.
Agent-reported final result
Workflow completed using keyboard actions only. Applied On Sale, Bluetooth, and Wireless filters and set Maximum price to 50. Two qualifying results are visible: Insignia™ Mini Bluetooth Wireless Speaker — $24.99; JBL Go Essential Portable Bluetooth Wireless Speaker — $39.99.

AP-N2-02: Bypass target does not preserve main-content continuation

A11y-tree navigation only

ap-n2-02-a11y-nav

MP4
Harness completedtrue
Task outcome (agent-reported)completed
steps
13
prompt_tokens
88126
prompt_cached_tokens
55296
completion_tokens
1726
total_tokens
89852

flows.txt provenance

flow_file
${BLV_GENERATED_APPS_ROOT}/adoptapet/replicate-01/g1-generic-a11y/app/flows.txt
flow_section
additional
flow_index
7
flow_title
Complete a safe mock donation
Exact task
Mode rule: use only keyboard and Orca navigation actions. Use the compact accessibility tree as a semantic map only; do not use mouse, click, coordinate, selector, direct accessibility-node activation, DOM-query, browser navigation buttons, extraction, or hidden implementation details.

Property under demo: AP-N2-02 - Bypass target does not preserve main-content continuation.

Complete a safe mock donation

Source workflow steps: Open `/donate`, choose an amount, provide an email and name, and submit. A confirmation is shown and explicitly states that no charge was made.

Use only keyboard actions. When the source workflow says click, choose, open, select, or activate, perform the equivalent keyboard action with Tab, Shift+Tab, Enter, Space, arrow keys, typing, or waiting. Do not use mouse, click, coordinate, or hidden browser-inspection actions. Finish when the workflow is completed or the specified confirmation/result is visible.
Agent-reported final result
Safe mock donation completed successfully. The confirmation says: “Your mock $25 gift confirmation is ready. No charge was made.”

Keyboard + vision assisted

ap-n2-02-vision

MP4
Harness completedtrue
Task outcome (agent-reported)completed
steps
11
prompt_tokens
54588
prompt_cached_tokens
0
completion_tokens
1988
total_tokens
56576

flows.txt provenance

flow_file
${BLV_GENERATED_APPS_ROOT}/adoptapet/replicate-01/g1-generic-a11y/app/flows.txt
flow_section
additional
flow_index
7
flow_title
Complete a safe mock donation
Exact task
Mode rule: use only keyboard actions. Do not use mouse, click, coordinate, DOM-query, browser navigation buttons, extraction, or hidden implementation details.

Property under demo: AP-N2-02 - Bypass target does not preserve main-content continuation.

Complete a safe mock donation

Source workflow steps: Open `/donate`, choose an amount, provide an email and name, and submit. A confirmation is shown and explicitly states that no charge was made.

Use only keyboard actions. When the source workflow says click, choose, open, select, or activate, perform the equivalent keyboard action with Tab, Shift+Tab, Enter, Space, arrow keys, typing, or waiting. Do not use mouse, click, coordinate, or hidden browser-inspection actions. Finish when the workflow is completed or the specified confirmation/result is visible.
Agent-reported final result
Completed the safe mock donation using keyboard-only actions. Selected $50, entered test@example.com and Test Donor, and submitted. Confirmation shown: “Your mock $50 gift confirmation is ready. No charge was made.” Also observed AP-N2-02: activating “Skip to main content” left focus on the body instead of continuing at a main-content control.

AP-PS1-01: Focusable element is hidden from the accessibility tree

A11y-tree navigation only

ap-ps1-01-a11y-nav

MP4
Harness completedtrue
Task outcome (agent-reported)completed
steps
21
prompt_tokens
210028
prompt_cached_tokens
86016
completion_tokens
5780
total_tokens
215808

flows.txt provenance

flow_file
${BLV_GENERATED_APPS_ROOT}/stubhub/replicate-01/g1-generic-a11y/app/flows.txt
flow_section
required
flow_index
14
flow_title
Find out how to transfer NFL tickets to a friend.
Exact task
Mode rule: use only keyboard and Orca navigation actions. Use the compact accessibility tree as a semantic map only; do not use mouse, click, coordinate, selector, direct accessibility-node activation, DOM-query, browser navigation buttons, extraction, or hidden implementation details.

Property under demo: AP-PS1-01 - Focusable element is hidden from the accessibility tree.

Find out how to transfer NFL tickets to a friend.

Source workflow steps: Open Help, search “transfer NFL tickets,” and press Search, or choose Transfer NFL tickets under Popular topics. The article explains how to open an order, transfer selected tickets, confirm through the team partner, and have the friend accept them.

Use only keyboard actions. When the source workflow says click, choose, open, select, or activate, perform the equivalent keyboard action with Tab, Shift+Tab, Enter, Space, arrow keys, typing, or waiting. Do not use mouse, click, coordinate, or hidden browser-inspection actions. Finish when the workflow is completed or the specified confirmation/result is visible.
Agent-reported final result
To transfer NFL tickets to a friend:

1. Sign in to StubHub, open **Orders**, and select the NFL game.
2. Choose **Transfer tickets**, select the seats, and enter your friend’s name and email address.
3. Follow the link to the team’s ticket partner and confirm the transfer. StubHub will show **“Transfer sent”** in the order.
4. Your friend opens the email, creates or signs in to the team account, accepts the tickets, and adds them to their mobile wallet.

Transfers are free. The article recommends accepting the tickets before game day and using the team or venue app to scan in.

Keyboard + vision assisted

ap-ps1-01-vision

MP4
Harness completedtrue
Task outcome (agent-reported)completed
steps
1
prompt_tokens
4784
prompt_cached_tokens
0
completion_tokens
333
total_tokens
5117

flows.txt provenance

flow_file
${BLV_GENERATED_APPS_ROOT}/stubhub/replicate-01/g1-generic-a11y/app/flows.txt
flow_section
required
flow_index
14
flow_title
Find out how to transfer NFL tickets to a friend.
Exact task
Mode rule: use only keyboard actions. Do not use mouse, click, coordinate, DOM-query, browser navigation buttons, extraction, or hidden implementation details.

Property under demo: AP-PS1-01 - Focusable element is hidden from the accessibility tree.

Find out how to transfer NFL tickets to a friend.

Source workflow steps: Open Help, search “transfer NFL tickets,” and press Search, or choose Transfer NFL tickets under Popular topics. The article explains how to open an order, transfer selected tickets, confirm through the team partner, and have the friend accept them.

Use only keyboard actions. When the source workflow says click, choose, open, select, or activate, perform the equivalent keyboard action with Tab, Shift+Tab, Enter, Space, arrow keys, typing, or waiting. Do not use mouse, click, coordinate, or hidden browser-inspection actions. Finish when the workflow is completed or the specified confirmation/result is visible.
Agent-reported final result
To transfer NFL tickets to a friend:
1. Sign in to StubHub, open **Orders**, and select the NFL game.
2. Choose **Transfer tickets**, select the seats, and enter your friend’s name and email address.
3. Follow the link to the team’s ticketing partner and confirm the transfer. StubHub will show **“Transfer sent”** in the order.
4. Your friend opens the invitation email, creates or signs in to the team’s account, accepts the tickets, and adds them to their mobile wallet.

The article also notes that transfers are free and tickets should be accepted before game day.

AP-W3-01: Supported activation has no relevant response

A11y-tree navigation only

ap-w3-01-a11y-nav

MP4
Harness completedtrue
Task outcome (agent-reported)completed
steps
48
prompt_tokens
489001
prompt_cached_tokens
248832
completion_tokens
12202
total_tokens
501203

flows.txt provenance

flow_file
${BLV_GENERATED_APPS_ROOT}/airbnb/replicate-01/g2-taxonomy-informed/app/flows.txt
flow_section
required
flow_index
4
flow_title
Favorite a shared room in Belo Horizonte costing up to $100 for a weekend in May.
Exact task
Mode rule: use only keyboard and Orca navigation actions. Use the compact accessibility tree as a semantic map only; do not use mouse, click, coordinate, selector, direct accessibility-node activation, DOM-query, browser navigation buttons, extraction, or hidden implementation details.

Property under demo: AP-W3-01 - Supported activation has no relevant response.

Favorite a shared room in Belo Horizonte costing up to $100 for a weekend in May.

Source workflow steps: Visit `/stays`, select “Shared rooms,” type “Belo Horizonte,” choose a Friday-to-Sunday range in May, open “Filters,” set Maximum price to 100, choose “Shared room,” and select “Show stays.” On “Sunny room in creative townhouse,” select the heart button and choose “Favorites.” A screen-reader-perceivable “Saved to Favorites” confirmation appears.

Use only keyboard actions. When the source workflow says click, choose, open, select, or activate, perform the equivalent keyboard action with Tab, Shift+Tab, Enter, Space, arrow keys, typing, or waiting. Do not use mouse, click, coordinate, or hidden browser-inspection actions. Finish when the workflow is completed or the specified confirmation/result is visible.
Agent-reported final result
Favorited “Sunny room in creative townhouse” in Belo Horizonte for the May 7–9, 2027 weekend. The applied filters were Shared rooms, maximum price $100, and Shared room type; the stay costs $64/night. Success is reflected by the pressed “Remove Sunny room in creative townhouse from wishlist” toggle and “Wishlists 1.” The exact transient “Saved to Favorites” confirmation was not directly captured by Orca after activation and waiting.

Keyboard + vision assisted

ap-w3-01-vision

MP4
Harness completedtrue
Task outcome (agent-reported)failed
steps
59
prompt_tokens
468386
prompt_cached_tokens
0
completion_tokens
14195
total_tokens
482581

flows.txt provenance

flow_file
${BLV_GENERATED_APPS_ROOT}/airbnb/replicate-01/g2-taxonomy-informed/app/flows.txt
flow_section
required
flow_index
4
flow_title
Favorite a shared room in Belo Horizonte costing up to $100 for a weekend in May.
Exact task
Mode rule: use only keyboard actions. Do not use mouse, click, coordinate, DOM-query, browser navigation buttons, extraction, or hidden implementation details.

Property under demo: AP-W3-01 - Supported activation has no relevant response.

Favorite a shared room in Belo Horizonte costing up to $100 for a weekend in May.

Source workflow steps: Visit `/stays`, select “Shared rooms,” type “Belo Horizonte,” choose a Friday-to-Sunday range in May, open “Filters,” set Maximum price to 100, choose “Shared room,” and select “Show stays.” On “Sunny room in creative townhouse,” select the heart button and choose “Favorites.” A screen-reader-perceivable “Saved to Favorites” confirmation appears.

Use only keyboard actions. When the source workflow says click, choose, open, select, or activate, perform the equivalent keyboard action with Tab, Shift+Tab, Enter, Space, arrow keys, typing, or waiting. Do not use mouse, click, coordinate, or hidden browser-inspection actions. Finish when the workflow is completed or the specified confirmation/result is visible.
Agent-reported final result
Partially completed using keyboard only. I set Belo Horizonte, selected the Friday–Sunday range May 7–9, 2027, selected Shared rooms, applied a maximum price of $100 plus the Shared room filter, and showed the matching stay “Sunny room in creative townhouse” at $64/night. The heart remains unselected, and I did not observe a “Saved to Favorites” confirmation before the step limit.