The challenges of using AI in software development [Q&A]

Artificial intelligence has found its way into many areas, not least software development. But using this technology isn't without problems around security, code quality and more.

We talked to Vibor Cipan, senior manager -- developer relations, community and success at AI coding agent Zencoder to find out more about the challenges of AI development and how to address them.

BN: What are the current challenges in using AI for software development today?

VC: The first challenge is the issue of hallucinations and incorrect code generation. We've found that while large language models can produce syntactically correct code, they often struggle with context, sometimes hallucinating functions or API calls that don't actually exist in the codebase. This is particularly challenging when the generated solutions don't align with the existing architecture. At Zencoder, we've addressed this through what we believe is a crucial multi-layered approach -- first using our repository grokking technology to understand the entire codebase context, then implementing agentic feedback loops to verify and validate code against existing patterns.

Another is context awareness. We've seen firsthand how first-generation coding assistants struggle with complex codebases, especially in enterprise environments where you're dealing with tens of thousands of files and millions of lines of code. In our view, this requires a deep understanding of not just the code itself, but the relationships and dependencies between files and the underlying Abstract Syntax Tree. That's why we've developed what we call the 'Goldilocks context' approach -- providing just the right amount of context to LLMs for high-quality code generation.

We've also observed how rapidly the software development landscape evolves. It's not just about keeping up with new frameworks and libraries -- we're seeing how this evolution affects everything from architectural patterns to security best practices and performance optimizations. This is particularly challenging when dealing with enterprise codebases that often contain legacy code and technical debt. We believe the solution lies in sophisticated approaches like continuous model retraining and dynamic fine-tuning that can adapt to project-specific patterns.

Finally, we've recognized that tool fragmentation and workflow integration present real challenges for developers. In our experience, developers are already juggling multiple tools -- from version control systems to CI/CD pipelines and testing frameworks. That's why we've focused on integrating directly into popular IDEs like VS Code and JetBrains products, making our AI assistant a natural extension of the developer's existing workflow rather than another tool to manage.

BN: There are many coding agents and assistants out there. What are the characteristics of a good one compared to an average one?

VC: In our research and practical working with AI coding assistants, we've come to understand what truly sets exceptional solutions apart from average ones. We believe it starts with deep context integration. From our perspective, superior coding assistants need to move beyond simple single-file understanding to achieve comprehensive repository analysis. We've found that this means not just understanding multiple repositories and supporting various programming languages, but truly grasping the nuances of coding styles and patterns that make each development team unique.

When it comes to reinforcement learning and adaptation, we've seen how leading solutions leverage sophisticated models with reasoning capabilities. We've been particularly impressed by developments like DeepSeek's R1, which has shown how even relatively simple reward mechanisms can dramatically improve code quality and reliability.

Regarding customization and control, we've learned that providing teams with granular control over their AI assistants is crucial. This isn't just about basic settings -- we're talking about the ability to leverage various AI models based on specific needs, orchestrate multiple models working in concert, and define custom AI instructions that truly reflect a team's development practices. We've found this particularly important when working with enterprise clients who often have very specific requirements and established workflows.

In our experience, security and reliability have proven non-negotiable. We've seen how enterprise adoption demands comprehensive security, privacy, and compliance infrastructure. Organizations need formal certifications like ISO and SOC2, along with rigorous compliance processes. This is particularly crucial when dealing with sensitive codebases and intellectual property.

BN: There have been issues around LLMs being unreliable and creating bugs. What are your thoughts on this?

VC: The challenges with LLMs in code generation are very real, but we've found innovative ways to address them, that is what really differentiates us from the others where we address the hallucinations. Yes, these models can be non-deterministic by nature, but we believe the key lies in how you handle and refine their output rather than just accepting what they initially generate.

In our experience we've discovered that the secret to reliable code generation isn't in trying to make LLMs perfect -- it's in building sophisticated systems around them. We've developed an approach using agentic feedback loops coupled with real-time code repair mechanisms. Think of it as having multiple expert reviewers constantly checking and improving the code as it's being generated, significantly reducing both errors and developer frustration.

What we've found particularly effective is moving beyond the traditional single-model approach. Instead, we orchestrate multiple specialized models and agents working together, each with their own expertise. It's similar to how a development team works -- you might have an architect drawing up the blueprints, developers implementing the code, and QA engineers verifying everything works correctly. Our reasoning models create the high-level architectural plans, while specialized models handle the actual code generation, and separate verification models ensure everything meets quality standards. When issues are spotted, our repair agents automatically jump in to fix them.

We've structured this as what we like to call a 'methodical journey' rather than a single leap. The code doesn't just appear -- it evolves through a sophisticated pipeline, starting with careful architectural planning and moving through multiple validation cycles. Our system maintains constant awareness of code quality, much like a senior developer would, ensuring everything stays consistent with existing patterns in your codebase.

BN: The development stack can be complex. How do these coding solutions integrate with that stack, and a company's codebase?

VC: Successfully integrating AI into these environments requires a deep understanding of this complexity while staying focused on what matters most -- developer productivity.
At Zencoder, we've approached this challenge through our Repo Grokking technology. Rather than just scanning code, we've developed a system that truly understands a codebase's DNA. It ingests the entire codebase to understand not just the technical elements -- file structures and dependencies -- but also the human aspects, like developer coding styles and team conventions.

We see a lot of value in embedded coding agent's ability to handle complex workflows. Instead of generating isolated snippets, we've built it to break down complex tasks into manageable pieces while maintaining a holistic view. This means it can work across multiple files and different programming languages while keeping everything consistent with your existing codebase.

A core component is Zencoder's agentic loop system. We don't just accept the first output from an LLM -- every piece of generated code goes through multiple refinement cycles. Think of it as having a team of expert reviewers constantly checking and improving the code, ensuring it aligns perfectly with your established patterns and practices.

Through this approach, we've created what we believe is a truly integrated solution that feels like a natural extension of your development process, not just another tool to manage.

BN: Where do you see the future of AI in coding?

VC: We envision AI evolving into something far more meaningful than just a code generation tool -- it will become your coding buddy, a trusted sidekick that drastically improves developer productivity while fostering growth and innovation. Our philosophy centers on empowerment rather than replacement, seeing AI as an amplifier of human creativity and capability in software development.

Moving forward, we expect AI to transform into a true pair programming partner -- the kind of colleague you've always wanted by your side. It won't just write code; it will engage in meaningful dialogue about architectural decisions, suggest optimizations, and help you grow not just as an engineer but as a professional. This AI companion will understand your coding style, anticipate your needs, and challenge you to explore better solutions while respecting your expertise and judgment.

We're pragmatic about this evolution. AI will undoubtedly transform existing development workflows and create entirely new ones that we can't yet imagine. We'll see enhanced capabilities in understanding complex codebases, more sophisticated architectural planning, and seamless integration across all aspects of software development. However, what excites us most is the potential for AI to become a catalyst for developer growth and learning.

Throughout all of this our vision remains steadfastly human-centric. The future we're building isn't about replacing developers -- it's about creating an environment where AI and human developers work in harmony, each contributing their unique strengths to the development process. This will not only boost productivity but also foster innovation and professional growth, creating new possibilities in software development that we're just beginning to imagine.

Image credit: BiancoBlue/depositphotos.com