In our previous blog post about Cypress, we highlighted the importance of reporting test automation results. This post takes it a step further by using the previously generated report.json file compiled from all the Mochawesome JSON files to create comprehensive Slack messages with our test results. These messages allow teams to be swiftly notified about potential issues with their web applications and react quickly to prevent further escalations. This post will walk you through, step by step, how to incorporate Slack messages into your Cypress automated tests.
Slack Channels and Webhooks
The first necessary step to sending Slack messages is to create Slack channels in your workspace (for example: cypress-slack-results). It’s advisable to create a separate Slack channel for each environment you want to report results for.
If you have two environments where your tests are running (e.g., DEV and ACC), you can create two corresponding channels: cypress-slack-results-dev and cypress-slack-results-acc.
For both of these channels, you’ll need to generate incoming webhooks. We won’t go into the details of creating these webhooks here, as Slack provides official documentation on how to set them up:
- Create Webhook URL for channel at https://api.slack.com/apps
- For more info read this article: https://api.slack.com/messaging/webhooks
Slack Messages
As previously mentioned, to successfully send Slack messages with accurate test results, a previously generated report.json file containing all accumulated results is required.
To generate this file, please follow the steps outlined in our previous blog post: https://www.north-47.com/reporting-in-cypress.
This file is then used to parse and send a Slack message containing the complete results of the test run. The analysis is performed using the following part of the code:
function analyzeReport(report) {
const builder = new SlackBlocksBuilder();
builder.add(
`Cypress test results for pipeline <${process.env.CI_PIPELINE_URL}|${process.env.CI_PIPELINE_IID}>. Report: <https://gitlab/{pathToPipeline}/-/jobs/${process.env.CI_JOB_ID}/artifacts/download| Download the full report>`
);
let totalPassedSteps = report.stats.passes;
let totalFailedSteps = report.stats.failures;
let totalSkippedSteps = report.stats.pending;
const totalSteps = totalPassedSteps + totalFailedSteps + totalSkippedSteps;
builder.add(
`*Summary*\nTotal tests: ${totalSteps}\n:white_check_mark: *${totalPassedSteps}* | :x: *${totalFailedSteps}* | :grey_question: *${totalSkippedSteps}*`
);
if (totalFailedSteps === 0 && totalSkippedSteps === 0) {
builder.add(`All tests have passed :dancing_parrot:`);
} else if (totalFailedSteps > 0) {
builder.add(
`We have failing tests :fire: \nPlease check the reports :point_up:`
);
}
if (totalSkippedSteps > 0) {
builder.add(
`We have skipped tests. :disappointed: \nPlease make sure to fix and enable them, or delete obsolete automated tests.`
);
}
return builder.build();
}
What is displayed in the slack message is the following:
- Download link for the html report
- Summary of the passed/failed/skipped tests
- Custom message depending of what the results are. (if there are failed tests a warning message is displayed to check the results)
The full script for sending slack messages looks as follows:
import dotenv from "dotenv";
import https from "https";
import fs from "fs/promises";
dotenv.config();
const reportDir = "cypress/results";
const reportPath = `./${reportDir}/report.json`;
const webHookURL = process.env.SLACK_WEBHOOK_URL;
class SlackBlocksBuilder {
constructor() {
this.sections = [];
}
add(section) {
this.sections.push(section);
}
build() {
return this.sections.map((section) => ({
type: "section",
text: {
type: "mrkdwn",
text: section,
},
}));
}
}
function analyzeReport(report) {
const builder = new SlackBlocksBuilder();
builder.add(
`Cypress test results for pipeline <${process.env.CI_PIPELINE_URL}|${process.env.CI_PIPELINE_IID}>. Report: <https://gitlab/{pathToPipeline}/-/jobs/${process.env.CI_JOB_ID}/artifacts/download| Download the full report>`
);
let totalPassedSteps = report.stats.passes;
let totalFailedSteps = report.stats.failures;
let totalSkippedSteps = report.stats.pending;
const totalSteps = totalPassedSteps + totalFailedSteps + totalSkippedSteps;
builder.add(
`*Summary*\nTotal tests: ${totalSteps}\n:white_check_mark: *${totalPassedSteps}* | :x: *${totalFailedSteps}* | :grey_question: *${totalSkippedSteps}*`
);
if (totalFailedSteps === 0 && totalSkippedSteps === 0) {
builder.add(`All tests have passed :dancing_parrot:`);
} else if (totalFailedSteps > 0) {
builder.add(
`We have failing tests :fire: \nPlease check the reports :point_up:`
);
}
if (totalSkippedSteps > 0) {
builder.add(
`We have skipped tests. :disappointed: \nPlease make sure to fix and enable them, or delete obsolete automated tests.`
);
}
return builder.build();
}
function prepareSlackMessage(messageBlocks) {
return {
username: "Cypress",
icon_emoji: ":banana:",
blocks: messageBlocks,
};
}
function sendSlackMessage(webhookURL, messageBody) {
return new Promise((resolve, reject) => {
try {
messageBody = JSON.stringify(messageBody);
} catch (e) {
reject(new Error("Failed to stringify messageBody: " + e.message));
return;
}
const requestOptions = {
method: "POST",
headers: {
"Content-Type": "application/json",
},
};
const req = https.request(webhookURL, requestOptions, (res) => {
let response = "";
res.on("data", (d) => {
response += d;
});
res.on("end", () => {
resolve(response);
});
});
req.on("error", (e) => {
reject(new Error("Slack request failed: " + e.message));
});
req.write(messageBody);
req.end();
});
}
(async function () {
if (!webHookURL) {
console.error("Error: SLACK_WEBHOOK_URL environment variable is not set.");
process.exit(1);
}
try {
const messageBlocks = await analyzeReport();
const message = prepareSlackMessage(messageBlocks);
const response = await sendSlackMessage(webHookURL, message);
console.log("Slack message sent successfully:", response);
} catch (e) {
console.error("Error in main function:", e);
}
})();
Further on to POST to Slack, the slack incoming webhook is needed which can be set in the .env file as SLACK_WEBHOOK_URL.
To post messages to slack the following command needs to be added to package.json:
"slack": "node post-to-slack.js",At the end to run the whole test suite and report the results, run the following command:
cypress run --browser chrome --headless; npm run merge:slack:reports; npm run slackConclusion
Promptly reporting test results to Slack and making them available to all team members: QAs, developers, and product owners increases the visibility of issues and improves our response time in resolving them. This leads to significantly higher software quality, boosts team productivity, and helps ensure that bugs no longer cause unnecessary headaches.
