An innovative Machine learning based automated bug triaging, filing and notification system
‘Machine learning based automated bug triaging, filing and notification system’ developed by Sudhanshu Gaur and Mayank Mohan Sharma.
This system is based on harnessing Machine Learning (Supervised Learning) to make a software defect or bug finding, triaging, filing, and notification system.
This system automatically picks up results of the software test automation reports on a regular (which could be hourly/nightly/weekly etc.) basis. It parses the current failures (or errors) from the report and then compares the failures/errors to the failures/errors that are known or present in the bug/defect tracking system. Here, machine learning will help the system to make the decision based on the failure logs that the bug has to be considered as a new, deferred, known issue or if it is already existent in the defect tracking system.
Once the decision has been made by the system, the ticket’s respective status will be logged or changed respectively. If the bug/defect is new, it will create a new ticket for the same in the defect tracking system. If the defect is previously filed then it will change the status of the ticket to Open (or reopen if it was closed). Once the ticket is created or reopened the system will automatically notify the stakeholders via email or instant messaging about the respective ticket.
A feedback mechanism is devised in the system by which a software test engineer can make the system adjust the decision making to be more precise. All the components and the whole process will be automated i.e. there will be no human interference unless the algorithm or the decision making engine has to be adjusted via feedback. This would make the system more precise and more accustomed to the respective quality and testing standards followed by the user or organization respectively.
Present Day Drawbacks:
Manual Process
- Test Management is a manual process currently.
- It takes significant amount of Man Hours in Bug Triaging, Bug Filing which is costly and not efficient.
- Test Automation’s full potential is not utilized when the test results are looked and analyzed manually.
- Performance edge given by automation gets neutralized due to manual intervention.
Not Scalable
- Scaling of present day Test Management process means more man hours.
- It is not completely automated so execution, reporting, triaging has to be done for each different types of test automation systems like Android Espresso, XCuitest for mobile, Selenium for Front End systems’ automation etc. and that too manually.
Time and Quality
- To find (from the log files), triage and file a bug takes at least 20-30 minutes.
- Plus overhead like finding if the bug has been previously filed or not. Then send mails or tag developers for communicating the bug discovery etc.
- Quality can take a hit, if there are too many bugs/exceptions.
Visibility
- Current systems does not provide the visibility/ability to find out which test suite(s) is more buggy/ erroneous, so that immediate action can be taken.
- Creation of Dashboard from the current process is difficult as it will have to be updated manually as the final decision of filing or not filing a bug is taken manually.
Highlights of the new system:
- Autonomous Learning, Prediction and Decision Making
- Automatic Process as Machine Learning based system finds, triages and files the bugs
- Communication i.e. notifications to all stakeholders is inbuilt
- Highly Scalable
- Integration is easy as it runs when the test automation already has completed the execution
- Centralized ML(Machine Learning) Engine which needs not to be changed for any Test Automation Framework or testing type i.e. Mobile Apps, Application Programming Interface, Front End systems etc.
- Feedback Refined Quality + Fully Machine Operated System
- All Automated Process significant time saving
- Quality controlled by algorithms and programs so any number of defects can be handled
- Dashboards based Deep Insights
- It would be very easy to glance through the quality of the automated test suites via dashboards.
- Overall quality can be checked and immediate action can be taken based on the observations.
- Creation of dashboards will be easy, as the whole process will be automated end to end.
Sudhanshu Gaur and Mayank Mohan Sharma has also filed for the patent for the technology.
