EXANTE, a leading finance company, has recently undertaken a significant transformation in its continuous integration (CI) practices. Anatoly Bobunov, a seasoned Software Development Engineer in Test (SDET), has been at the forefront of this evolution. By systematically overhauling the CI pipeline, Bobunov aimed at creating a more robust, efficient, and user-friendly automated testing framework. His efforts have not only improved the existing processes but have also set a foundation for future advancements in CI and automated testing within the company.
Initial State of CI at EXANTE
Identifying the Rudimentary Design
At EXANTE, the CI pipeline initially featured a simplistic single-stage called “tests.” While this setup served its purpose, it had notable limitations. Execution times varied greatly from 5 minutes to an hour, making the process unpredictable. Moreover, the lack of comprehensive status notifications meant that detailed log spelunking was required to diagnose issues. The rudimentary design hindered the team’s ability to maintain consistent performance across various services, leading to inefficiencies that called for an overhaul of the entire CI pipeline.
Team Structure and Skill Disparities
EXANTE’s testing teams were small and segmented by service groups. This structure, although organized, created skill discrepancies. Experienced testers often juggled framework development, while junior members struggled to keep pace. This imbalance necessitated a more streamlined and skills-inclusive CI pipeline. The segmentation of teams by services meant that expertise was often siloed, which further complicated efforts to standardize testing practices across the organization. Addressing these skill disparities became a key focus to ensure that both seasoned and new testers could contribute effectively to framework development.
Standardizing the Test Environment
Role of Docker in Ensuring Consistency
To alleviate compatibility issues, the team turned to Docker. By creating a standardized Dockerfile, they encapsulated all necessary dependencies, ensuring a uniform testing environment across various operating systems. Despite the Docker image bloating to 1.3 GB, it proved invaluable for maintaining consistency. This consistency was crucial for avoiding the “works on my machine” syndrome, which often plagued the development and testing process. The uniform environment created by Docker ensured that tests would run the same way regardless of where they were executed, leading to more reliable and reproducible results.
Automated Docker Image Rebuilds
Docker image rebuilds were automated to reflect changes in configurations, libraries, or the Dockerfile. This automation, albeit adding a few extra minutes to the build time, eliminated manual rebuilds. It significantly improved efficiency and ensured that the test environment was always up-to-date. With automatic rebuilds, the team could be confident that any changes in dependencies or configurations were immediately incorporated into the test environment. This not only reduced the time spent on manual updates but also minimized the risk of errors that could arise from outdated environments, further streamlining the CI process.
Enhancing Reporting Capabilities
Integration with Allure TestOps
EXANTE integrated Allure TestOps to enhance reporting and data visualization. This tool allowed automatic login, test run creation, and data submission to Allure. Implementing a “Prepare allure run” stage helped organize interactions with Allure, passing data as job artifacts and streamlining the reporting process. With Allure TestOps, the team could generate comprehensive and visually appealing reports that provided deep insights into test runs. This integration addressed the previous limitations in report clarity and accessibility, making it easier for testers to analyze results and identify issues without delving into extensive logs.
Detailed Insights and Feedback
The detailed insights provided by Allure TestOps enabled better visualization of test results. Integrating this stage in the CI pipeline addressed previous limitations in report clarity, ensuring that testers had access to comprehensive data without needing exhaustive log examinations. The tool’s ability to present data in a user-friendly format improved communication within the team and facilitated quicker decision-making. With clear and detailed insights, testers could focus on resolving issues more effectively, thus enhancing the overall quality and reliability of the software being developed.
Incremental Enhancements in CI Stages
Introduction of the Collect Stage
The introduction of a “collect” stage addressed early coding errors. Using the --collect-only option with pytest, this stage verified the collectability of tests, preemptively catching simple coding and configuration issues. This update reduced unnecessary delays caused by non-runnable tests. By implementing a preemptive check, the team could ensure that only valid tests were executed, thus saving valuable time and resources. The collect stage acted as a preliminary filter, allowing testers to focus on more complex issues rather than wasting time on basic errors that could have been easily avoided.
Implementing Static Code Analysis
To standardize code quality, EXANTE added a static code analysis stage. Initially, tools like flake8 and isort were introduced, allowing for gradual adaptation. Over time, the stage became mandatory, enforcing strict coding standards that improved code consistency and MR review efficiency. Static code analysis helped in identifying potential issues early in the development process, thus preventing them from becoming major problems later. By enforcing coding standards, the team could ensure a higher level of code quality and maintainability, ultimately contributing to a more stable and reliable software product.
Early Error Detection and Rapid Feedback
Defining Smoke Tests for Critical Functionality
EXANTE defined smoke tests to quickly verify critical functionalities. Marked with @pytest.mark.smoke, these tests were isolated in the CI pipeline. Any failure in smoke tests immediately halted the pipeline, thus preventing further unnecessary executions and facilitating quicker fixes. By focusing on essential functionalities, smoke tests provided a rapid feedback loop that allowed testers to identify and address significant issues early in the pipeline. This approach minimized the impact of critical failures and ensured that fundamental system operations were validated before proceeding with more extensive testing.
Health Checks for Service Availability
A “health checks” stage was included to verify service availability post-deployment. By running automated health check tests, the team ensured that services were operational before proceeding. This stage provided immediate feedback on infrastructure issues, streamlining the troubleshooting process. Health checks acted as a safety net, confirming that all essential services were up and running before further tests were executed. This immediate validation helped maintain the integrity and reliability of the CI pipeline, ensuring that subsequent tests were conducted under optimal conditions and reducing the risk of cascading failures.
Ensuring Internal Script Integrity
Introduction of Internal Tests Stage
An “internal tests” stage was incorporated to validate the functionality of test preparation scripts and utilities. These tests, triggered by changes in designated directories, ensured that auxiliary scripts remained functional, thereby reducing potential disruptions in test data preparation. Internal tests played a crucial role in maintaining the integrity of the test environment by verifying that all supporting scripts were operational. By ensuring the reliability of these internal tools, the team could avoid unexpected issues during test execution, thus enhancing the overall efficiency and effectiveness of the CI pipeline.
Improving Debugging Efficiency
To simplify debugging, EXANTE enabled a custom-run mechanism in GitLab CI. This feature allowed users to run specific jobs with customized settings, bypassing certain checks when necessary. Parameter-based runs offered flexibility and facilitated an efficient troubleshooting process. Custom runs enabled testers to isolate issues more quickly by focusing on specific segments of the pipeline. This targeted approach to debugging not only expedited the resolution of problems but also provided valuable insights into potential improvements for the CI pipeline, thus contributing to its ongoing optimization and refinement.
Visualization and Pipeline Flow
Using Pipeline Flowcharts
The pipeline’s sequential execution was illustrated through flowcharts. These visual aids showcased the stages involved in typical use-case pipelines, such as nightly runs and acceptance tests. Flowcharts helped team members grasp the pipeline’s structure and interactions, promoting a better understanding of the CI process. By providing a clear and concise visual representation, flowcharts enabled the team to identify bottlenecks and areas for improvement more easily. This holistic view of the pipeline facilitated better planning and coordination, ultimately enhancing the overall efficiency and effectiveness of the CI process.
Adoption of Best Practices
By visualizing and documenting the pipeline stages, EXANTE promoted the adoption of best practices. This structured approach ensured that all team members were on the same page, driving consistency and alignment in CI processes. Documenting the pipeline stages provided a valuable reference for both current and future team members, fostering a culture of continuous improvement and collaboration. This commitment to best practices not only improved the immediate performance of the CI pipeline but also laid the groundwork for sustainable long-term success.
Conclusion
EXANTE, a premier finance company, has recently undergone a major transformation in its continuous integration (CI) practices, marking a pivotal shift in its technological landscape. Spearheading this effort is Anatoly Bobunov, a seasoned Software Development Engineer in Test (SDET), whose expertise has been crucial in this endeavor. Bobunov’s mission was clear: overhaul and refine the CI pipeline to enhance robustness, efficiency, and user-friendliness in the automated testing framework. By methodically revamping the CI system, Bobunov has significantly upgraded the company’s existing processes, ensuring that they are more streamlined and effective. This transformation not only elevates the current state of CI and automated testing at EXANTE but also lays a strong foundation for future technological advancements in these areas. His dedication and strategic approach have set a new benchmark for CI practices within the organization, driving innovation and excellence. The long-term benefits of this overhaul will undoubtedly position EXANTE at the forefront of technological innovation in the financial sector, making it a model for others to follow.
