Guide
Show Cucumber test results on Jira stories
By Miftode Vasilica, maker of Gherkin Studio ·
To see Cucumber results on a Jira story, your pipeline needs to do two things after the tests: write a report in the Cucumber JSON format, and send that file to an app in Jira that knows which scenario belongs to which story. The link between the two is a tag with the issue key, such as @PROJ-123. This page shows the whole chain, with a GitHub Actions example you can copy.
"Is it done?"
Friday, stand-up. The product owner asks whether the password reset story is done. The developer says the pipeline is green. The tester says she has not looked yet. The product owner opens Jira, where the story says "In review", which answers nothing.
The information exists. It is in a CI log three clicks away, in a format nobody outside the team reads. Putting pass or fail on the story itself removes the question.
Step 1: make your test run write Cucumber JSON
Most BDD tools can write this format, either built in or through a plugin (Cucumber's list of reporters).
cucumber-js:
npx cucumber-js --format json:cucumber.json
Cucumber-JVM, in the runner's options:
@CucumberOptions(plugin = {"json:target/cucumber.json"})
Reqnroll, SpecFlow, behave and pytest-bdd reach the same format through their own plugins or converters.
Step 2: tag scenarios with the issue key
@PROJ-12
Scenario: Link expires after one hour
Given I requested a reset link 61 minutes ago
When I open the link
Then I see "This link has expired"
Gherkin Studio looks for the story in this order: a key tag on the scenario, a key tag on the feature, then a file named like PROJ-12.feature. A scenario tagged with two keys counts for both stories.
Step 3: send the report after the tests
In Jira, a Jira admin opens Apps → Gherkin Studio and creates a CI token. Save the address and the token as secrets in your CI. Then add one step.
GitHub Actions:
- name: Run Cucumber
run: npx cucumber-js --format json:cucumber.json
- name: Send results to Jira
if: always()
env:
GHERKIN_STUDIO_URL: ${{ secrets.GHERKIN_STUDIO_URL }}
GHERKIN_STUDIO_TOKEN: ${{ secrets.GHERKIN_STUDIO_TOKEN }}
BUILD_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
run: |
curl -X POST "$GHERKIN_STUDIO_URL?build=$BUILD_URL" \
-H "Authorization: Bearer $GHERKIN_STUDIO_TOKEN" \
-H "Content-Type: application/json" \
--data-binary @cucumber.json
The line people forget is if: always(). Without it, a failing test stops the job before the report is sent. Jira then shows the last green run, on exactly the day you need to see red.
The build address is optional. When you send it, it has to start with https://.
What you see on the story
- Failing if any of the story's scenarios failed, with the step that failed.
- Passing if all of them passed.
- Not run otherwise.
A story with nine passing scenarios and one failing is failing. That is the point.
When nothing shows up
The answer to the curl call tells you why. It has three fields:
| Field | What it means | What to fix |
|---|---|---|
updated |
Stories that received results | Nothing |
unknownIssues |
Keys in your tags that do not exist in Jira | A typo in a tag, or a story that was moved to another project |
unmappedScenarios |
Scenarios with no key anywhere | Add an @KEY-123 tag |
Print the answer in your CI log for the first few runs. Most first-day problems are a tag written in lower case: @proj-12 is not read as an issue key, @PROJ-12 is.
Short answers
Do I need Cucumber itself?
No. Any tool that writes Cucumber JSON works.
Can several jobs send results for the same story?
Yes, when each job covers different stories. Each report replaces the result of the stories it mentions, so if one story's scenarios are split across two jobs, merge the two files before sending. How to merge them.
Who can see the results?
Anyone who can see the issue.
Where Gherkin Studio fits
Gherkin Studio is a BDD app for Jira Cloud, built on Atlassian Forge. It is on the Atlassian Marketplace, with a 30-day free trial. See the test results section of the docs and how to tag scenarios with issue keys.