Overview

The Mutation Test Report provides a historical view of mutation testing results executed for different projects. It helps users evaluate the effectiveness of their test cases by displaying key metrics such as mutation score, execution duration, and technology used. This report enables teams to monitor test quality, track previously executed mutation tests, and identify areas where test coverage can be improved.

Pre-requisites

The Cavisson IDE Plugin installation and configuration must be completed prior to report generation. Follow the below-mentioned steps for installing and configuring the Cavisson plugin. The plugin is supported on IDEs such as Visual Studio, Visual Studio Code, IntelliJ IDEA, and Eclipse.

1. Installation

Install the Cavisson Mutation Testing extension in Visual Studio Code.

• Open the Extensions view in VS Code (Ctrl + Shift + X).
• Click the More Actions (⋯) menu in the top-right corner.
• Select Install from VSIX.
• Choose the file Cavisson Mutation Testing to install the extension.

2. Configuration

Configure the mutation testing profile.

• Open the Command Palette (Ctrl + Shift + P).
• Run the command Cavisson Mutation Tester: Configure Project.
• Provide the following details:
  • Select Technology: Choose the required technology (e.g., .NET).
  • Profile Name: Enter a profile name (e.g., PaymentService_Tests).
  • Target Files / Namespace: Specify the namespace or use * to scan all files.
  • Tags: Enter labels for categorization (e.g., sprint:24, build:102).
  • Server URL: Enter the Cavisson Controller URL. For example: http://<controller-host>:8080
  • Auth Token: Provide the Cavisson API authentication token, which can be fetched from the Cavisson Controller UI under User Profile/API Token section.

3. Running Tests

• Open the Command Palette (Ctrl + Shift + P).
• Run Cavisson Mutation Tester: Run and Upload Report.
• Select the configured profile to start the mutation testing process.
• After execution completes, the extension automatically opens the generated report in the default web browser for quick local review.

For viewing all the generated mutation test reports, you can log in the Product UI and access the mutation test report from Configurations > Advanced

Global Action Icons

The Global Action Icons available at the top-right corner of the table allow users to quickly manage and analyze the mutation test report data.

• Search: Enables users to quickly locate specific records by entering relevant keywords in the search field.
• Refresh: Updates the table to display the latest available mutation test report records.
• Column Filter: Enables users to show or hide table columns based on their preference, helping customize the table view for focused analysis.
• Download: Allows users to export the mutation test report data for offline analysis or record keeping.

Mutation Test Report History

This section displays a list of previously generated mutation test reports along with their execution details.

The table contains the following columns:

• Status: Indicates the current status of the mutation test execution (for example, completed or running).
• Project Name: Displays the name of the project for which the mutation test was executed. This value is derived from the project specified during the Cavisson IDE plugin configuration.
• Generated At: Shows the date and time when the mutation test report was generated.
• Mutation Score: Represents the effectiveness of the test cases by showing the percentage of mutations detected by the tests. A higher score indicates better test effectiveness. As a general guideline, a mutation score above 60% is considered acceptable, above 75% is good, and above 85% is considered excellent. Taken to execute the mutation testing process.
• Technology: Indicates the technology stack used for the project (such as .NET, Java, etc.).
• Tags: Displays labels associated with the mutation test report to help categorize and organize reports.
• Action Options: The Action column provides the following options to manage the mutation test report:
  • View mutation report: Opens the detailed mutation test report for further analysis.
  • Delete: Removes the selected mutation test report from the history.

Mutation Test Report - Mutants

The Mutants tab displays mutation testing results for all project files and directories. It helps evaluate how effectively test cases detect introduced mutations. A summary bar at the top shows the distribution of Killed, Survived, and No Coverage mutants for quick analysis. Below it, a table lists files and directories, allowing users to drill down into folders and individual files to view mutation results in detail.

Mutants Table Columns

• File / Directory: Displays the name of the folder or source file. Users can expand folders to navigate deeper into the project structure and view mutation results at the file level.
• Mutation Score Of Total: Represents the percentage of mutations killed out of the total number of generated mutants.
• Mutation Score Of Covered: Shows the mutation score calculated only for the code that is covered by tests.
• Killed: Indicates the number of mutants that were successfully detected and killed by the test cases.
• Survived: Displays the number of mutants that were not detected by the tests, indicating potential weaknesses in the test suite.
• Timeout: Shows the number of mutants that caused tests to run longer than the allowed execution time.
• No Coverage: Represents mutants generated in parts of the code that are not covered by any tests.
• Ignored: Displays mutants that were intentionally ignored during mutation testing.
• Runtime Errors: Shows the number of mutants that caused runtime errors during execution.
• Compile Errors: Represents mutants that failed during compilation.
• Detected: Indicates the total number of mutants detected by the test suite.
• Undetected: Displays mutants that were not detected during mutation testing.
• Total: Shows the total number of generated mutants for the respective file or directory.
• Mutation report theme button: The Theme icon allows users to switch the report interface between Light and Dark themes for better readability and viewing preference.

Drill Down View

The Mutation Report supports drill-down analysis for deeper insights. By clicking on a File or Directory in the table, a detailed view opens displaying mutation statistics for the selected folder or file. This allows users to analyze mutation results at a more granular level, helping identify files where test coverage is effective or needs improvement.

Code Level View

The Code Level View displays the source code of the selected file along with mutation results. It highlights the code where mutations were applied and shows the corresponding mutation status (such as Killed). This view helps users analyze how test cases interact with specific code lines and identify areas that require improved test coverage.

Mutation Tests

The Tests tab shows how test cases interact with generated mutants during mutation testing. It helps identify which tests successfully detect mutants and which parts of the code are not adequately covered. A summary bar at the top displays mutation results such as Killed, Survived, and No Coverage mutants for quick analysis. Below it, a table lists test directories and files, allowing users to analyze test effectiveness and drill down into folders or individual test files for more detailed insights.

Mutation Tests Table Columns

• File / Directory: Displays the name of the test folder or test file. Users can expand folders to navigate deeper into the test structure.
• Killing: Indicates the number of mutants that were successfully detected and killed by the corresponding tests.
• Covering: Represents the number of mutants that are covered by the tests but not necessarily killed.
• Not Covering: Displays the number of mutants that are not covered by the tests.
• Total Tests: Shows the total number of tests present in the respective folder or test file.

Drill Down View (Tests)

The Tests tab also supports drill-down analysis. By clicking on a test folder or file, a detailed view opens showing mutation statistics and the corresponding test case code. This helps users analyze how specific test cases kill or cover mutants and evaluate test effectiveness.

Test Case Code View

The Test Case Code View displays the source code of the selected test file along with mutation testing results. It highlights the tests responsible for killing or covering mutants, allowing users to understand how specific test cases validate the code. This view helps analyze the effectiveness of individual test cases in detecting mutations.