Lesson 03 · Reports, Artifacts & Test Selection

Beyond the 1Z0-830 exam

A green checkmark isn't enough — when CI fails you need to see what failed and why, and you need the slow tests not to block every push. This lesson covers publishing reports/artifacts and selecting which tests run when (fast vs full), which is what the module's lab models.

Objectives

After this lesson you will be able to:

• Publish test reports (Surefire/JUnit XML, Allure) from CI.
• Save build artifacts (logs, screenshots, coverage) for diagnosis.
• Tag tests and select subsets with Surefire groups or Maven profiles.
• Design a fast PR build vs a full nightly build.

Reports — make failures legible

Surefire writes JUnit-format XML to target/surefire-reports/ on every run. CI turns that into a readable summary:

• JUnit XML → most CI systems render pass/fail per test from it; GitHub Actions has test-reporter actions that annotate the PR.
• Allure (Module 15) → a richer HTML report with steps, attachments, and history/trends.

Publish the report so a reviewer sees the failing test name and message without leaving the PR.

Artifacts — capture the evidence

An artifact is a file the job uploads for later download — the things you need to debug a failure you can't see:

- name: Run tests
  run: mvn -f labs/pom.xml test
- name: Upload reports on failure
  if: failure()                                  # only when something broke
  uses: actions/upload-artifact@v4
  with:
    name: surefire-reports
    path: labs/target/surefire-reports/

Typical artifacts: Surefire reports, UI screenshots/video/traces (Module 18), coverage HTML, application logs. The if: failure() guard avoids uploading noise on green runs.

Test selection — fast vs full

Running the entire suite (including slow integration/UI/Testcontainers tests) on every push is too slow. Tag tests and run subsets:

@Test @Tag("fast")  void parsesInput() { … }
@Test @Tag("slow") @Tag("integration") void fullCheckoutFlow() { … }

Then filter at the command line — Surefire reads JUnit tag expressions:

mvn test -Dgroups=fast                    # PR build: only fast tests
mvn test -DexcludedGroups=slow            # everything except slow
mvn test                                  # nightly: the full suite

Maven profiles (Module 13) do the same at a coarser grain — this course uses one to keep infra-heavy Part B labs out of the default run.

The lab's TestSelector is this exact logic: excluding("slow") for the PR build, including("smoke") for a smoke gate, all() for nightly — with the rule that an excluded tag wins even if the test also carries a wanted tag:

TestSelector.excluding("slow").select(suite);   // → only the fast tests
TestSelector.including("smoke").select(suite);   // → only smoke-tagged tests

A two-tier pipeline

Trigger	Tests	Goal
Pull request / push	fast only (unit, excludes slow/integration/ui)	feedback in minutes
Nightly (cron) / pre-release	full suite incl. Testcontainers & UI	thorough coverage

This keeps everyday feedback quick (Module 15's pyramid: lots of fast unit tests) while still running the expensive tests regularly.

Gotcha — don't let "fast" hide real failures

Selection is a speed optimization, not an excuse to skip tests that matter. If a slow test only ever runs nightly, a breakage can sit for a day. Keep a meaningful smoke subset on every push, and make the nightly failures visible (notify, don't bury) so the full suite stays trusted.

SDET note

Selection, reports, and artifacts together make CI trustworthy: fast feedback that's quick to act on, failures that explain themselves, and evidence to debug them. A suite people trust is one they keep green; a slow, opaque suite is one they learn to ignore — and an ignored suite catches nothing.

Key Takeaways

• Publish test reports (Surefire JUnit XML, Allure) so failures are legible on the PR.
• Upload artifacts (reports, screenshots, logs) — guard with if: failure() — to debug what you can't see.
• Tag tests (@Tag) and select with Surefire -Dgroups/-DexcludedGroups or Maven profiles.
• Run a fast PR build (unit/smoke) and a full nightly build (integration/UI/Testcontainers).
• An excluded tag wins over an included one; keep a smoke subset per push and make nightly failures visible.

Lesson Quiz

Lesson Quiz · Reports, Artifacts & Test Selection0 / 5

1. Why upload artifacts like screenshots/reports from a CI job?

   • ATo slow the build
   • BTo diagnose a failure you can't reproduce locally, from the CI run itself
   • CThey replace the tests
   • DOnly for legal reasons

2. How do you run only fast tests in Maven/Surefire?

   • ADelete the slow tests
   • BTag tests with @Tag and use -Dgroups=fast (or -DexcludedGroups=slow)
   • CRename the test files
   • DSet a longer timeout

3. A sensible two-tier pipeline is…

   • AFull suite on every keystroke
   • BFast subset on each push/PR; full suite (incl. integration/UI) nightly or pre-release
   • COnly nightly runs, never on PRs
   • DRandom selection each run

4. In the lab's TestSelector, excluding('slow') applied to a test tagged both 'slow' and 'smoke'…

   • ARuns it because of 'smoke'
   • BExcludes it — an excluded tag wins (the conservative choice)
   • CThrows an error
   • DRuns it once per tag

5. A risk of relying only on a fast PR build is…

   • ABuilds get too slow
   • BA breakage caught only by a slow/nightly test can sit unnoticed for a day
   • CReports stop generating
   • DTags stop working

---

Next: Module 21 · AI-Assisted Development & Testing. This module's lab is in labs/src/main/java/com/jse21/m20_cicd/.