CIGRATE & CIPilot

Two connected contributions

CIGRATE · Research framework

I researched and developed an LLM-based framework for CI configuration migration between Travis CI and GitHub Actions. The work compared zero-shot, few-shot, and fine-tuned settings using GPT-4 and open-source models.

CIPilot · Implementation and extension

CIPilot builds on that work toward broader CI/CD migration scenarios, including CircleCI and Azure DevOps. The tooling was delivered as a browser extension and web portal.

The research question

How can migration output be generated and assessed in a way that combines configuration similarity, syntax validation, and practical usability?

Generation needs a validation and correction path

1. Establish the migration inputStart from the source CI configuration and target service. The configuration is the artifact being transformed.
2. Generate the target configurationUse the selected model and prompting or fine-tuning setting to produce the migrated YAML.
3. Check syntax, configuration, and usabilityAutomated checks assess generated configurations alongside the research's similarity measures.
4. Detect errors and refine the configurationAgentic workflows support iterative correction and configuration refinement, extending the basic generation step.

This workflow distinguishes producing configuration text from evaluating whether the result is structurally valid and useful.

Evaluation scope and what each check measures

CrystalBLEU

A similarity measure used to compare generated configurations with reference configurations.

Cosine similarity

A complementary similarity measure in the evaluation.

Syntax and configuration validation

Automated checks of the generated configuration's structure and service-specific requirements.

Usability assessment

Checks intended to assess the readiness of the generated output for practical CI/CD use.

These methods describe different dimensions of output quality. Similarity scores and configuration validation are complementary evidence.

Publication and product evidence

The public CIgrate registered report, co-authored with Taher A. Ghaleb, records the study design and comparison with CIMig. It appears in the ICSME 2025 Registered Reports program.

The subsequent research and evaluation work resulted in a publication in Empirical Software Engineering. The practical tooling can be explored through the CIPilot website.

My contribution connects experimental design, framework implementation, automated checks, and developer-facing tooling.