[java] tag os-sensitive browser tests per file and run them on Windows and macOS - #18163

• Tag OS-sensitive Java browser tests per file so CI selects relevant local targets.
• Replace hand-picked Windows and macOS test lists with a shared tag filter.
• Use Selenium Manager browser downloads because CI runners do not cache pinned browsers.

graph TD
  A["Java BUILD files"] --> B["Suite macro"] --> C["Browser macro"] --> D["Stable local tests"] --> E["Filtered CI jobs"] --> F["Windows runner"]
  E --> G["macOS runner"]

Loading

The following are alternative approaches to this PR:

1. Maintain explicit CI target lists

• ➕ Avoids changing the Bazel test-generation macros.
• ➖ Duplicates target selection across CI jobs.
• ➖ New OS-sensitive tests can be missed.
• ➖ Variant names must be maintained manually.

Recommendation: Use the PR's per-file tagging approach. It keeps test selection beside the test definitions and lets both operating systems share a filter, while the browser macro prevents remote and beta variants from matching.

Enhancement (2) +15 / -6

selenium_test.bzlRestrict OS-sensitive tags to stable local browser targets +7/-5

Restrict OS-sensitive tags to stable local browser targets

• Separates the os-sensitive tag from shared tags when generating browser variants. Stable local targets retain it; remote and beta targets do not.

java/private/selenium_test.bzl

suite.bzlSupport per-file tags in Java Selenium suites +8/-1

Support per-file tags in Java Selenium suites

• Adds an extra_tags map to java_selenium_test_suite and applies each entry to its corresponding generated test. Rejects map keys that are not in the suite's sources.

java/private/suite.bzl

Other (5) +39 / -13

ci-java.ymlSelect tagged Java tests in both OS jobs +21/-13

Select tagged Java tests in both OS jobs

• Replaces hand-picked target lists with an os-sensitive filter over Java targets. Adds build and execution options for continued testing, serialized local jobs, and Selenium Manager browser downloads instead of pinned browsers.

.github/workflows/ci-java.yml

BUILD.bazelTag window geometry and screenshot tests +5/-0

Tag window geometry and screenshot tests

• Marks PositionAndSizeTest, TakesScreenshotTest, and WindowTest as OS-sensitive without tagging the rest of the suite.

java/test/org/openqa/selenium/BUILD.bazel

BUILD.bazelTag Firefox full-page screenshot test +3/-0

Tag Firefox full-page screenshot test

• Marks TakesFullPageScreenshotTest as OS-sensitive within the Firefox browser suite.

java/test/org/openqa/selenium/firefox/BUILD.bazel

BUILD.bazelTag remote-driver download test +3/-0

Tag remote-driver download test

• Marks RemoteWebDriverDownloadTest as OS-sensitive within the router's large-test suite.

java/test/org/openqa/selenium/grid/router/BUILD.bazel

BUILD.bazelTag input-action browser tests +7/-0

Tag input-action browser tests

• Marks combined actions, keyboard, mouse, wheel, and drag-and-drop tests as OS-sensitive. Other interaction tests remain outside the filter.

java/test/org/openqa/selenium/interactions/BUILD.bazel

Description

resetWindow() calls setPosition() and changeSizeTo() before every WindowTest method,
excluding Android but not other mobile platforms whose window cannot be moved or resized. On a
mobile session that supports reading its window size but not changing its window rectangle, even
testGetsTheSizeOfTheCurrentWindow now fails during setup before its read-only assertion executes.

Code

java/test/org/openqa/selenium/WindowTest.java[R41-44]

+    if (!TestUtilities.getEffectivePlatform(driver).is(ANDROID)) {
+      driver.manage().window().setPosition(new Point(50, 50));
+      changeSizeTo(new Dimension(600, 500));
+    }

Evidence

The superclass initializes the driver before this BeforeEach method. The test's existing comments
document that window movement and resizing are unsupported on most mobile platforms, while the new
guard checks only Android; the first test merely reads window size.

java/test/org/openqa/selenium/testing/JupiterTestBase.java[52-65]
java/test/org/openqa/selenium/WindowTest.java[38-60]
java/test/org/openqa/selenium/testing/TestUtilities.java[150-159]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The new per-test window reset executes unsupported resize and move commands before read-only tests on non-Android mobile sessions.
## Fix Focus Areas
- java/test/org/openqa/selenium/WindowTest.java[38-45]
## Recommended Fix
Restrict the reset to sessions that support window movement and resizing, including an appropriate guard for non-Android mobile platforms, without changing the read-only tests' setup.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

Description

Adding @Ignore(SAFARI) to testShouldCaptureScreenshotAtIFramePageAfterSwitching disables its
last executable default browser variant: the method already ignores Chrome, Edge, and Firefox. The
default browser targets exclude Internet Explorer, so no standard target now exercises a screenshot
after switching into an iframe; the other new Safari ignores also remove Safari coverage from four
screenshot checks.

Code

java/test/org/openqa/selenium/TakesScreenshotTest.java[R262-264]

+  @Ignore(
+      value = SAFARI,
+      reason = "Safari applies the display color profile, so colors do not match exactly")

Evidence

The method ignores Chrome, Edge, and Firefox as well as the newly ignored Safari. The browser
macro's default set excludes Internet Explorer, and matching Ignore annotations disable test
execution.

java/test/org/openqa/selenium/TakesScreenshotTest.java[256-290]
java/private/selenium_test.bzl[19-79]
java/test/org/openqa/selenium/testing/SeleniumExtension.java[232-237]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The new Safari ignore makes the switched-iframe screenshot test disabled for every default browser target, while four other screenshot checks also stop running on Safari.
## Fix Focus Areas
- java/test/org/openqa/selenium/TakesScreenshotTest.java[110-112]
- java/test/org/openqa/selenium/TakesScreenshotTest.java[152-154]
- java/test/org/openqa/selenium/TakesScreenshotTest.java[192-194]
- java/test/org/openqa/selenium/TakesScreenshotTest.java[225-227]
- java/test/org/openqa/selenium/TakesScreenshotTest.java[262-264]
## Recommended Fix
Replace the unconditional Safari skips with a Safari-compatible screenshot assertion, such as a color-profile-tolerant comparison, so the screenshot operations remain tested.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

Description

The new os-sensitive filter excludes ChromeDriverFunctionalTest and FirefoxDriverBuilderTest
because neither source file receives the tag in its suite. The Windows job previously ran both
targets and the macOS job ran the Firefox target, so those browser-startup checks are no longer
selected on those platforms.

Code

.github/workflows/ci-java.yml[46]

+        --test_tag_filters=os-sensitive

Evidence

The workflow replaces its explicit target lists with a positive tag filter. The Chrome suite
includes ChromeDriverFunctionalTest.java through its source glob but does not tag it; the Firefox
suite lists FirefoxDriverBuilderTest.java but tags only the screenshot test. The suite macro
applies per-file tags to generated targets, so neither former target matches the filter.

.github/workflows/ci-java.yml[37-47]
.github/workflows/ci-java.yml[63-73]
java/test/org/openqa/selenium/chrome/BUILD.bazel[27-41]
java/test/org/openqa/selenium/firefox/BUILD.bazel[4-11]
java/test/org/openqa/selenium/firefox/BUILD.bazel[48-61]
java/private/suite.bzl[64-79]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The new positive tag filter drops formerly explicit Chrome and Firefox browser-startup checks from the platform jobs.

## Fix Focus Areas
- .github/workflows/ci-java.yml[44-47]
- .github/workflows/ci-java.yml[70-73]
- java/test/org/openqa/selenium/chrome/BUILD.bazel[27-41]
- java/test/org/openqa/selenium/firefox/BUILD.bazel[48-61]

## Recommended Fix
Add `os-sensitive` per-file entries for `ChromeDriverFunctionalTest.java` and `FirefoxDriverBuilderTest.java` so their stable local targets remain selected by the new filter.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

Description

In throwErrorWhenElementOriginIsOutOfViewport and throwErrorWhenOriginOffsetIsOutOfViewport, the
PR deletes .hasMessageContaining("out of bounds") for every browser, not only for Safari, whose
wording is presumably the one that differs. Chrome, Edge and Firefox now pass even if their error
text changes or stops explaining the failure, and nothing records which browser needed the looser
check.

Code

java/test/org/openqa/selenium/interactions/DefaultWheelTest.java[102]

+        .isInstanceOf(MoveTargetOutOfBoundsException.class);

Evidence

The patch removes the message assertion with no browser condition at both sites (old lines 97-98 and
136-137), even though the rest of this change scopes Safari differences with @Ignore(SAFARI) or
@NotYetImplemented(SAFARI).

java/test/org/openqa/selenium/interactions/DefaultWheelTest.java[90-142]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The `hasMessageContaining("out of bounds")` assertions were removed for all browsers, so Chrome, Edge and Firefox no longer have their error message checked.

## Fix Focus Areas
- java/test/org/openqa/selenium/interactions/DefaultWheelTest.java[90-142]

## Recommended Fix
Keep the message assertion and skip it only when the current browser is Safari, e.g. capture the thrown exception and assert on its message when `!Browser.detect().equals(Browser.SAFARI)`. Another option is a Safari-specific assertion with a short comment saying why the wording differs.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

💡 Did you know, you can turn these tips off under Display > Comment parts

More tips ↗ | Customize Qodo ↗ | Qodo docs ↗

Review updated until commit df01e45

Results up to commit 9a88d74 ⚖️ Balanced

🐞 Bugs (0) 📘 Rule violations (0) 📎 Requirement gaps (0) 🎨 UX issues (0) 🔗 Cross-repo conflicts (0) 📜 Skill insights (0)

1. OS jobs no longer check browser builders ✗ Dismissed

Description

The new os-sensitive filter excludes ChromeDriverFunctionalTest and FirefoxDriverBuilderTest
because neither source file receives the tag in its suite. The Windows job previously ran both
targets and the macOS job ran the Firefox target, so those browser-startup checks are no longer
selected on those platforms.

Code

.github/workflows/ci-java.yml[46]

+        --test_tag_filters=os-sensitive

Evidence

The workflow replaces its explicit target lists with a positive tag filter. The Chrome suite
includes ChromeDriverFunctionalTest.java through its source glob but does not tag it; the Firefox
suite lists FirefoxDriverBuilderTest.java but tags only the screenshot test. The suite macro
applies per-file tags to generated targets, so neither former target matches the filter.

.github/workflows/ci-java.yml[37-47]
.github/workflows/ci-java.yml[63-73]
java/test/org/openqa/selenium/chrome/BUILD.bazel[27-41]
java/test/org/openqa/selenium/firefox/BUILD.bazel[4-11]
java/test/org/openqa/selenium/firefox/BUILD.bazel[48-61]
java/private/suite.bzl[64-79]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The new positive tag filter drops formerly explicit Chrome and Firefox browser-startup checks from the platform jobs.

## Fix Focus Areas
- .github/workflows/ci-java.yml[44-47]
- .github/workflows/ci-java.yml[70-73]
- java/test/org/openqa/selenium/chrome/BUILD.bazel[27-41]
- java/test/org/openqa/selenium/firefox/BUILD.bazel[48-61]

## Recommended Fix
Add `os-sensitive` per-file entries for `ChromeDriverFunctionalTest.java` and `FirefoxDriverBuilderTest.java` so their stable local targets remain selected by the new filter.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

Results up to commit 3364267 ⚖️ Balanced

🐞 Bugs (0) 📘 Rule violations (0) 📎 Requirement gaps (0) 🎨 UX issues (0) 🔗 Cross-repo conflicts (0) 📜 Skill insights (0)

1. Mobile window-size checks fail during setup ✗ Dismissed

Description

resetWindow() calls setPosition() and changeSizeTo() before every WindowTest method,
excluding Android but not other mobile platforms whose window cannot be moved or resized. On a
mobile session that supports reading its window size but not changing its window rectangle, even
testGetsTheSizeOfTheCurrentWindow now fails during setup before its read-only assertion executes.

Code

java/test/org/openqa/selenium/WindowTest.java[R41-44]

+    if (!TestUtilities.getEffectivePlatform(driver).is(ANDROID)) {
+      driver.manage().window().setPosition(new Point(50, 50));
+      changeSizeTo(new Dimension(600, 500));
+    }

Evidence

The superclass initializes the driver before this BeforeEach method. The test's existing comments
document that window movement and resizing are unsupported on most mobile platforms, while the new
guard checks only Android; the first test merely reads window size.

java/test/org/openqa/selenium/testing/JupiterTestBase.java[52-65]
java/test/org/openqa/selenium/WindowTest.java[38-60]
java/test/org/openqa/selenium/testing/TestUtilities.java[150-159]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The new per-test window reset executes unsupported resize and move commands before read-only tests on non-Android mobile sessions.
## Fix Focus Areas
- java/test/org/openqa/selenium/WindowTest.java[38-45]
## Recommended Fix
Restrict the reset to sessions that support window movement and resizing, including an appropriate guard for non-Android mobile platforms, without changing the read-only tests' setup.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

2. The switched-iframe screenshot check never runs ✗ Dismissed

Description

Adding @Ignore(SAFARI) to testShouldCaptureScreenshotAtIFramePageAfterSwitching disables its
last executable default browser variant: the method already ignores Chrome, Edge, and Firefox. The
default browser targets exclude Internet Explorer, so no standard target now exercises a screenshot
after switching into an iframe; the other new Safari ignores also remove Safari coverage from four
screenshot checks.

Code

java/test/org/openqa/selenium/TakesScreenshotTest.java[R262-264]

+  @Ignore(
+      value = SAFARI,
+      reason = "Safari applies the display color profile, so colors do not match exactly")

Evidence

The method ignores Chrome, Edge, and Firefox as well as the newly ignored Safari. The browser
macro's default set excludes Internet Explorer, and matching Ignore annotations disable test
execution.

java/test/org/openqa/selenium/TakesScreenshotTest.java[256-290]
java/private/selenium_test.bzl[19-79]
java/test/org/openqa/selenium/testing/SeleniumExtension.java[232-237]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The new Safari ignore makes the switched-iframe screenshot test disabled for every default browser target, while four other screenshot checks also stop running on Safari.
## Fix Focus Areas
- java/test/org/openqa/selenium/TakesScreenshotTest.java[110-112]
- java/test/org/openqa/selenium/TakesScreenshotTest.java[152-154]
- java/test/org/openqa/selenium/TakesScreenshotTest.java[192-194]
- java/test/org/openqa/selenium/TakesScreenshotTest.java[225-227]
- java/test/org/openqa/selenium/TakesScreenshotTest.java[262-264]
## Recommended Fix
Replace the unconditional Safari skips with a Safari-compatible screenshot assertion, such as a color-profile-tolerant comparison, so the screenshot operations remain tested.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---