Overview

FlyCI Wingman is a DevTools startup dedicated to reducing the time developers spend in debugging failing CI builds, allowing them to focus on more impactful work.

Through extensive customer interviews, the founders of FlyCI, known for its fast and cost-effective macOS runners for GitHub Actions, identified that the most time-consuming and tedious task for developers is debugging failing CI builds. The process involves inspecting the logs of the failing build, making code changes, and then waiting to see if they resolved the issue. This repetitive cycle, regardless of whether it involves a one-line change or a more complex fix, distracts developers from their current tasks and might take hours to resolve.

This insight led to the creation of FlyCI Wingman. FlyCI Wingman is an AI-driven agent that analyzes failing workflows in GitHub Actions and suggests fixes to the failing builds. It collaborates with a large language model (LLM) optimized to address issues like lint errors, typos, failing tests, and more. Once Wingman identifies a fix, it makes code suggestions directly to the GitHub pull requests. The developer's role is then to review the suggested changes and merge them into the codebase.

The Solution

FlyCI Wingman relies on FlyCI, the macOS runners where the CI builds are executed. It extends the the runners capabilities by integrating the AI agent in the runners themselves. The key aspect of developing Wingman is to find the right LLM and continuously prompt it in an interactive manner, ensuring that the suggested fixes are both functional and aligned to the best software development practices. This process of continues prompting and interactive with the outside environment is know as agentic behavior.

Additionally, it is crucial to maintain the security and privacy provided by FlyCI runners, making sure no code leaves the ephemeral environment where the builds are executed.

How it works?

Usage of FlyCI Wingman is pretty straight-forward.

Prerequisite: FlyCI runners need to be set up in the repository. See how.

  1. Open FlyCI Dashboard and re-run the failing build with AI. You can see that a little FlyCI Wingman icon is added to mark that this new build is run with AI.

  2. When the build finishes, review the comments added to the pull request by FlyCI Wingman and merge the suggested code changes

Technical Details

Let's see what happens behind the scene.

When the developer re-runs the build with AI, FlyCI Wingman is called to action. The build is executed again, but this time, if a step fails, FlyCI Wingman identifies the logs related to the failure and inputs them into the LLM to get code change suggestions. The entire process occurs at the "Complete runner" step in the GitHub Actions workflow where the developer can also review the FlyCI Wingman logs.

It is important to note that FlyCI Wingman ensures the build passes with the proposed changes applied. This assurance provides developers with confidence that the suggested code changes effectively resolve the issue, eliminating the need for them to manually implement and test the fixes.

Conclusion

Today, the AI hype is undeniable, but beneath the surface there are practical applications of this technology that enable us to focus on what truly matters. FlyCI Wingman is a good example for AI agentic behavior[1] transforming repetitive build failures into valuable actions, thereby saving developers' time and maintaining their focus. Such kind of agentic behavior is not limited to developer workflows but can be used in any information workflow including querying data from databases, connecting to external services or orchestrating complex human-machine interactions. With the advancements in AI and Large Language Models(LLMs) we will witness more and more AI agents integrating into our daily work.

Read more about our experience with AI.

Do you need an AI expertise at your company?

Check out the AI services we offer and don't hesitate to contact us for a free consultation.

[1] AI agentic behavior refers to AI solutions that automate complex workflows and work autonomously with minimum to no human supervision.