Software architecture in the age of AI [Q&A]

Does AI impact software architecture? Or does software architecture impact AI? The answer of course is both, but how much of an overlap is there between the two?

To find out we spoke to Mike Loukides, vice president of emerging tech content at O'Reilly, about how software architects can work with AI and avoid its more negative effects.

BN: How will AI change software architecture?

ML: To talk about the impact of AI on software architecture, you have to consider both the practice of software architecture and the software that's architected. I've argued that AI won't change the practice of software architecture much. There may be better tools for analysis and design, but the substance of the job will remain unchanged. The hard parts of software architecture are understanding the context into which the software will fit; thinking about the trade-offs that are essential to good engineering; and collecting the information needed to make these decisions. We haven't yet seen AI tools that can perform these tasks. That doesn't mean those tools don't exist, but they're not here yet. The other side of this question is more important: how does AI change the software that architects are responsible for designing?

BN: How will AI change the applications that software architects design?

ML: We're going to see many new kinds of software: for example, systems that predict flows of products through supply chains so that a company can maintain good stocking levels, or systems to digest and summarize all the documents relevant to a legal case. Architects will have to do what they usually do: talk to the stakeholders, determine what the requirements are, and design software that will operate safely and correctly. But 'safely and correctly' means something new when AI is part of the product. We all know that AI can produce answers that are incorrect; we all know that AI can hallucinate, or things up out of nothing. That's fine when you're playing with ChatGPT, but it's certainly not fine when a business's future is at risk.

BN: What can software architects do to design applications that use AI responsibly?

ML: Nothing I've said means that we shouldn't use AI, but it does mean that we shouldn't use AI naively --– and that gets to the heart of the software architect's responsibilities. What does security mean in the context of AI? We're finally recognizing that, for traditional software at least, security needs to be designed in, not added on. How do you design security into an AI-driven application, where new kinds of attacks are possible? That's part of the architect's responsibility. How do you keep AI from generating inappropriate or incorrect responses? That's a matter of building 'guardrails' into the system. Designing guardrails is part of an architect's responsibility. How do you evaluate the system’s performance? Someone has to determine what it means for the application to be performing properly, and that task will fall to the architects. And we haven't mentioned considerations like operating costs and regulatory compliance that are important to any software project.

What makes this difficult is that AI security, guardrails, and evaluations are all still research topics. We're still working out how to do this. AI systems are complex, and we're still working on understanding the complexity. Those are some of the challenges that architects will need to deal with.

BN: How do architects guard against naïve and harmful uses of AI?

ML: Given the complexity that AI adds to software systems, I’d caution anyone building software against throwing in AI purely for buzzword compliance. Here's how I'd defend against naïve uses of AI. First, focus on real business goals. How will this application move the needle on key business metrics? Stay focused on that -- know what you're trying to accomplish. Don’t throw in AI just because you can. Second, keep humans in the loop, and think carefully about the humans’ role. It's not just to say yes or no, or to provide responses when an AI fails. That's likely to a new kind of soul-crushing drudgery. The best applications will combine human creativity with AI's ability to digest a huge body of information. Third, think about appropriate interfaces for the humans who are in the loop. We've all been impressed by ChatGPT, but I'm not convinced that chats are the ideal interface between humans and machines. Fourth, use AI to augment human capabilities rather than minimize head count. Jobs will change -- some will disappear, new ones will arise -- but using AI to reduce payroll rather than to maximize productivity is a path to failure.

BN: Are there any lessons that you can share from your own experience?

ML: One of O'Reilly’s AI projects is generating multiple choice quiz questions and answers. Even though the AI-generated questions and answers are rarely useable without editing, we’ve found that giving our developers a set of questions and answers to edit and improve is much more effective than having them write quizzes from scratch. Anyone who's had to develop a quiz knows that writing good quizzes is difficult and time-consuming. Combining humans and machines yields results that are better than either human or machine could produce on their own. We’re augmenting the capabilities of our talent, eliminating drudgery while helping our talent to be more creative. And we're generating higher quality quizzes faster. The speedup isn't miraculous -- it's more like 50 percent than 1000 percent -- but it's significant.

BN: Will software and software architecture evolve together?

ML: AI will undoubtedly change what we do and how we work. We will have to learn how to collaborate with machines just as much as we need to develop machines that know how to collaborate with us. Ethan Mollick encourages us to embrace AI's weirdness. Embracing the weirdness is a necessary part of developing new kinds of software, new applications that we can't yet imagine. However, we also need to make AI reliable and trustworthy–those are older, less buzz-worthy goals, but they are essential to building enterprise-ready AI applications. Software architects will need to keep both weirdness and trust in focus. They will be the key players in determining how AI and humans work together.

Image credit: everythingposs/depositphotos.com