When Anatoly Bobunov joined EXANTE as a Software Development Engineer in Test (SDET), the company’s Continuous Integration (CI) pipeline appeared simple but inefficient. A single “tests” stage encapsulated jobs for each service, with durations ranging from five minutes to a daunting hour. Notifications exclusively went to Slack, providing limited insights unless one dug into GitLab CI logs. Small teams of testers concentrated on particular services, leading to unbalanced skill sets and uneven focus.
State of CI Upon Arrival
Upon Anatoly’s arrival, the CI pipeline’s elementary nature quickly emerged as a critical bottleneck. Each job’s execution time varied widely, hampering coordination and efficiency. Moreover, the exclusive reliance on Slack notifications was insufficient for comprehensive monitoring, necessitating manual log checks. This setting perpetuated an uneven distribution of skills among small, service-specific testing teams, exacerbating the challenges.
Dockerfile for Test Environment
To address cross-platform compatibility issues, Anatoly introduced Docker to isolate the test environment. The Dockerfile detailed steps for creating essential directories, updating packages, installing dependencies, and downloading tools. Incorporating an automated Docker image rebuild when configuration files or libraries changed ensured a stable development environment. The implementation significantly reduced manual effort, although the average build time of two minutes pointed out room for further optimization.
Integration with Allure TestOps
Integrating Allure TestOps into the CI pipeline marked a substantial upgrade in test reporting and analytics. A preparatory stage was instituted to handle Allure interactions and save necessary data as job artifacts. These artifacts, subsequently utilized in test jobs, enhanced the organization and granularity of test result analysis, driving improved debugging and performance monitoring.
Test Collection Stage
The introduction of a “collect” stage marked a pivotal enhancement designed to validate code correctness before executing tests. By implementing the pytest --collect-only command, tests could be listed without being run, enabling quick detection of coding errors. This preliminary check successfully reduced time spent on correcting simple issues, thus streamlining the development pipeline.
Static Code Analysis Stage
In a move to standardize coding practices, a static code analysis stage introduced tools like isort and flake8 for maintaining code quality. These linters initially operated in a non-blocking mode to allow testers to rectify issues without failing jobs. Eventually, stricter enforcement helped in maintaining consistent coding styles and caught more subtle errors, further elevating code integrity.
Smoke Test Stage
A dedicated smoke test stage aimed at swiftly validating fundamental functionality became another significant addition. By marking critical tests with @pytest.mark.smoke and segregating them, this stage provided rapid feedback on crucial operations. When smoke tests failed, subsequent stages were halted, thereby conserving resources and time by pinpointing major flaws early.
Health Checks Stage
Implementing a health checks stage addressed one of the persistent issues—service infrastructure problems. This stage involved making API requests to verify service availability before executing tests. Ensuring that tests ran only when services were live minimized disruptions due to infrastructure issues, thus improving overall efficiency.
Internal Tests Stage
Enhancing the reliability of scripts used for test data preparation, an internal tests stage was introduced. Unit tests for these scripts were run whenever relevant changes occurred in the scripts folder within Merge Requests (MRs). This stage ensured that foundational utilities were functioning correctly, thereby supporting more dependable test setups.
Debugging in GitLab CI
To aid debugging efforts within GitLab CI, predefined variables were aligned with job customization options. This flexibility allowed team members to bypass certain checks during custom runs, making the debugging process more manageable. The ability to pass tailored parameters for test runs further empowered the team to test hypotheses effectively and efficiently.
Pipeline Flowcharts
Conclusively, flowcharts illustrating the pipeline’s full structure and specific examples of nightly and acceptance test runs offered visual clarity. The full pipeline diagram highlighted all checks implemented to prevent poor-quality code from merging. Concurrently, nightly and acceptance run illustrations focused on service status checks, Allure report preparation, and comprehensive service test execution.
Summary
When Anatoly Bobunov became a Software Development Engineer in Test (SDET) at EXANTE, he noticed the company’s Continuous Integration (CI) pipeline was complex yet lacking in efficiency. The pipeline’s single “tests” stage combined jobs for every service, with job durations varying widely from just five minutes to as long as an hour. All notifications were routed to Slack, which offered limited insights unless team members delved deeply into GitLab CI logs. This setup often left essential details buried and hard to find. Additionally, the team’s structure presented its own challenges. Small groups of testers focused narrowly on specific services, leading to imbalances in skill distribution and inconsistent attention to different areas of the codebase. The fragmentation in focus and uneven expertise distribution among teams contributed to the inefficiency and complexity of the pipeline, making it clear that an overhaul was necessary to streamline operations and improve overall effectiveness.
