<QA_Blog/>

As 2025 comes to a close, I have been reflecting on the current state of software quality. In 2025, we saw exponential growth in AI adoption within engineering teams. "Big Tech" companies have been bullish in their AI adoption strategies. AI tools replaced thousands of employees, and the remaining staff were told to use AI in their workflows or look for work elsewhere. Like many others, I can't help but question whether or not this transition has resulted in positive outcomes for building software.

• The biggest challenge for engineering teams isn't building software; it's often managing the expectations of management, who expect features to be delivered yesterday. AI vendors have irresponsibly hyped improved software delivery with "new and improved" tools released every few months. In reality, the promised productivity gains are unproven, and according to some researchers, there could actually be productivity losses. The reach of AI isn't just limited to writing code; it's being used throughout the software development life cycle. AI is being used to write requirements, tickets, tests, and documentation. A multi-tool approach is typical, with each tool having a narrow understanding of the software, the needs of end-users, and the business.

So, How Has AI Affected Software Quality?

Have you noticed the devices and services you use seem buggier and maybe more sluggish than usual? Well, you're not alone. This year, we've seen problematic system updates that have caused system crashes, combined with the forced and invasive addition of AI features, which have led to the biggest shakeup in desktop OS market share we've seen in years. We've also seen some of the buggiest smartphone software releases in recent times. There have been multiple major outages bringing down significant chunks of the internet. Planes have been grounded due to software issues, which caused nosedives. Multiple AAA video games have had terrible releases with crippling performance issues, bugs, and have completely missed the mark with their audiences. Motor vehicles are plagued with dangerous and annoying software glitches.

• This cannot be a mere coincidence.

There are flaws in AI tooling, with a significant one being the training data. AI vendors have scraped the internet, pulling our code (both good and bad), documentation, posts, bad habits, and biases. We will never know what filtration techniques AI companies use to ensure that only good-quality data is used. There is a certain smell to AI-generated code. When an agent is working on an existing codebase, it becomes clear very quickly that it doesn't understand the project. Too often, an agent will fail to detect existing components, hooks, and utility functions that are suitable to reuse and instead will create massive files that lack modularity and that break every software design practice you've read about. This leads to greater maintenance burdens and adds unnecessary complexity, which leads to greater risk of defective software and ultimately higher delivery costs.

While AI models can now leverage Model Context Protocol (MCP) and Retrieval-Augmented Generation (RAG) to enrich AI outputs based on the context provided from other sources, it isn't a silver bullet. A common example of using MCP is to provide a model with documentation about the product or feature under development. Anyone who has worked in a "fast-paced lean-agile startup" knows that documentation is often scarce and low quality. High-quality output from AI tooling requires high-quality, well-structured information for it to be beneficial. Even with integrations into your Confluence and Jira spaces, you can still find that outputs from AI can be unsatisfactory. If you are using AI to generate documentation, your workflow may end up in a hallucination loop. A hallucination loop occurs when incorrect information generated by AI makes it through your entire workflow, resulting in incorrect and defective software. If your workflow consumes additional content as part of its thinking process, ensure the content is accurate first. Be aware that using RAG can also lead to poor quality outcomes, as in most cases, it is not guaranteed which sources a model will choose to use for enrichment. There is no control over what information the model decides to retain or defer. Models typically use a first-in-first-out (FIFO) approach with context tokens, so over time, your session will become less useful and will eventually deteriorate to the point where you will need to start a new one. Using MCP or RAG can have adverse effects on the model's context window. Feeding a model with several documents and a large codebase will fill the context window rapidly, so you need to be selective about what you give the model access to. Heavy scrutiny is a must when using an agentic workflow to build software.

• Human beings have a natural inclination to find the quickest way to complete a task as an energy-saving measure. When we find a more efficient way to meet a goal, complacency can kick in quickly. With agentic workflows, it is too easy to become dependent. Research into AI consumption has shown evidence that we are using less of our brains. This could mean we are falling into habits where we rely on the machine to do our work and not scrutinise AI output as much as we should. I also think complacency is a significant issue when it comes to software performance. Devices are more powerful than ever, giving the illusion that optimizing software is less important. Devices may have significant resource overheads available, which is fine in isolation, but once deployed to an end-user's device running everyone else's poorly performing software, the illusion vanishes. Businesses favour feature delivery due to it being a more valuable use of time. Too often, performance issues raised are left unfixed because features are a priority.

How Can QA Professionals Adapt?

Being a QA professional can often be a thankless job, and often others within the business will question whether or not we are adding value. A well-oiled and integrated QA team provides value to the business with sound domain knowledge, the ability to foresee issues before development begins, and to find problems during development before they reach production. At times, some may treat QA as a "roadblock" in delivering features. The fact is, we often need that "roadblock" to ensure we mitigate issues and deliver a product that our users truly deserve. There are a few things I think we can do as QA professionals to improve software quality.

Slow down and intervene earlier to speed up later.
Everyone else might be in a hurry, but you don't have to be. Adopt a shift-left approach. If you are not already involved in the discovery and work creation steps in your workplace, put your hand up and get involved. As a tester, you should be able to foresee any potential risks, missing acceptance criteria, and holes in user stories, as you should know every inch of your product. As per the 1-10-100 rule in quality management, investing in prevention by getting involved in discovery and planning, you will save your company money.

Adopt a quality-first, automation-second mindset.
Just like everyone else, we are human and can fall prey to the complacency trap. Test automation is a tool, not a silver bullet. Passing tests does not guarantee quality software, even with a high degree of code or feature coverage. Quality is about much more than software being technically correct. Take ownership of quality over the entire development life-cycle. Look for gaps in features before they are developed, review documentation, and tickets; these will only become defects later. When you are testing, cast a wider net, as just testing the new feature isn't enough. The more you explore, the more you will discover. The work you put in now will save your team time, money, and headaches later on.

Be cautious with AI tool use.
There is no going back now. Whether you like it or not, AI is here to stay. The point of this blog post isn't to deter you from adopting AI in your practice; it's to highlight that there are problems with the way software is written today. AI can provide value, but you must remain focused on ensuring the validity of outputs as you would when testing an application. Someone who can use AI tooling efficiently to enhance their work will stand out from others.

Advocate for a culture of quality. Quality isn't the sole responsibility of QA professionals.
It's a value that your entire company should work by. Collaborate closely with your team, peer up with engineers during development, talk to users, work closely with decision-makers, and always challenge assumptions. The end-user should be at the forefront of all decision-making. Not only will this lead to better quality outcomes, but it will also enhance your product in a way that will leave a lasting impression on users.

Are We in a Full-Blown Software Quality Crisis?

Given the recent friction and changes in software development, as well as the increasing number of critical issues and poor releases across various software categories, I believe that overall software quality has deteriorated. Some may argue that software is just more complex, and while true, it's inexcusable that key indicators of quality software are being ignored. Based on what we know of the practices of big tech companies, it's fair to say that AI adoption (and misuse) is a contributing factor. As well as this, companies are spending less time on refinement and optimisation as it is seen as less valuable work. Leveraging automation hasn't been good for quality either, as teams can fall prey to complacency, allowing issues to fall through the cracks. Teams may also be cutting corners, not taking the time to properly plan and evaluate requirements. The business necessity to deliver fast and the combination of these factors are leading to poor quality outcomes. You don't have to dig very far to find a group of users more frustrated than ever with popular tools, systems, websites, and games. One can hope that we can learn from our mistakes in 2025 and that 2026 will be a new year with improved software quality for all.

For those interested in some further reading, I have compiled a table of the most notable software glitches and defects of 2025, spanning across several industries and software genres. You can also find sources used to write this blog below.

Merry Christmas and a happy New Year!

Sources

Software is getting worse:
https://www.youtube.com/watch?v=FI5ba4RRE8U
https://medium.com/@shivamyaduvanshi435/the-software-industry-is-broken-in-2025-and-developers-are-paying-the-price-1ab6d6dfbdf5
https://dev.to/esha_suchana_3514f571649c/the-hidden-24-trillion-crisis-why-software-quality-cant-wait-57ei

AI is destroying productivity:
https://hbr.org/2025/09/ai-generated-workslop-is-destroying-productivity

AI hallucination loop:
https://www.missingheader.com/017-the-hallucination-loop-how-ai-risks-reinforcing-its-own-errors/

Human complacency trap:
https://www.forbes.com/councils/forbestechcouncil/2025/04/23/the-complacency-paradox-trusting-ai-without-losing-your-edge/
https://link.springer.com/article/10.1007/s10676-024-09788-0

Quality assurance strategies:
https://www.browserstack.com/guide/what-is-shift-left-testing
https://www.makingstrategyhappen.com/the-cost-of-quality-the-1-10-100-rule/