Information Security Expert Kevin Beaver

I pen tested a web app built with Claude Code…The real security risks weren’t in the code.

The barrier to building software has collapsed. I’m starting to see the security consequences firsthand. A couple of months ago, I tested a web application built entirely with Claude Code. An application handling sensitive financial data. No traditional dev team. No large engineering processes. Just people with a clear vision iterating with AI until the application did what they wanted.

A few years ago, that wouldn’t have made any sense. Building a custom web application meant budgets, developers, architecture reviews…and patience. Small applications could take months. Bigger ones, like the one I tested, could take a year or more. Now someone with a clear vision and a Claude Code subscription (or one from any other AI application) can build one in a matter of days. Kind of scary…at least for this security guy.

The app worked well. The interface was clean, the workflows ran, and it accomplished what the creators needed it to accomplish. I’ve tested plenty of web applications over the years, but this one looked different almost immediately.

The Real Risk Wasn’t in the Code

Some things look complete until reality shows up. Think about a swimming pool with no pump. It looks like every other pool in the ad. Easy installation. Lower cost. Fewer parts to break. Then the water turns green, the algae takes over, and you learn what the pump was actually doing.

That’s a little like what I experienced testing this application. The functionality was there. What wasn’t immediately obvious were some of the things experienced developers and security people have learned (often the hard way!) need to be there too.

The technical findings were light, all things considered. Looking at the underlying web server, I found standard hardening gaps including weak encryption protocols still enabled, missing Windows mitigations for known vulnerabilities, and outdated supporting software that needed patching. These are common issues. Serious vulnerabilities in properly configured, fully patched systems are harder to come by than they used to be. In this case, two of the server findings only surfaced because I tested the system with Windows authentication enabled.

On the application side, I tested both without and with user authentication across multiple roles. The findings were lighter than I expected. No SQL injection or cross-site scripting. No broken access controls. No terribly insecure design flaws. The typical vulnerabilities I look for in a web application penetration test came back pretty clean.

I don’t think the results were accidental. Modern AI coding tools can recognize and avoid many common insecure coding patterns, and based on what I saw here, Claude Code did a respectable job at that layer.

As I got deeper into the testing, something became obvious. The interesting findings weren’t really about bad code. They were about decisions and assumptions that never made it into the code at all.

The people building the application had a clear rule set in their heads: priorities and non-negotiables, workflows users were supposed to follow, and things the application should never allow. Much of it never became part of the prompts, workflows, or implementation.

The AI builds what it’s told. If a constraint isn’t defined, the application won’t enforce it. If edge cases aren’t considered, users wander into places the creators/“developers” never intended. The application can function exactly as designed while still violating the underlying business intent.  AI is fluent in the language of software, infrastructure, and security. The business person prompting it usually isn’t. When the AI proposes an architecture or configuration choice, the person on the other side has no practical way to judge whether that choice is reasonable. They accept what they’re handed because they don’t know which questions to ask. What starts as a vocabulary mismatch becomes a control mismatch.

There’s a related problem. AI doesn’t just hand people choices they can’t evaluate. It sometimes hands them security guidance that’s wrong, delivered with the same glaring confidence as everything else it produces. If you don’t know the subject, you can’t tell the difference between good advice and bad advice that sounds good.

I’ve learned over the years that a lot of application security lives in the nuance. A model can know the language of security and recognize patterns, but it doesn’t own the context of this particular business, its users, or this organization’s tolerance for risk. Those differences completely change whether a default configuration or design decision makes sense.

You can secure code. You can’t necessarily secure business logic.

What I Actually Found

The application findings illustrated the broader issue well. They clustered mainly around policy and configuration decisions rather than traditional coding flaws.

The first area was authentication. The application accepted trivially weak passwords during account creation and password resets. Sessions had no timeout, which meant a logged-in user could remain authenticated indefinitely. Those are decisions about how the application should behave, not bugs in how a function was coded.

The second area involved communications and trust boundaries. Login pages allowed unencrypted credential transmission. Cross-origin resource sharing trusted arbitrary origins. API documentation was accessible without authentication, providing anyone who found it with an inventory of endpoints, parameters, and the overall data models. None of this required a sophisticated attacker. It required someone who knew to look.

Mapped against the OWASP Top 10 2025, the highest-risk findings clustered around cryptography and authentication. What interested me more was what I didn’t find. The categories I expected might cause more trouble, including injection and broken access control, came back surprisingly clean. What didn’t come back clean was the stuff a nondeveloper would never think to specify in the first place: password strength, session management, TLS configuration, and schema exposure.

You can tell when an application is built without traditional engineering experience. The functionality works. The polish is there. The depth isn’t. Details experienced developers have learned to handle instinctively because they’ve been burned before are nowhere to be found.

Testing AI-Generated Apps Is Different

Testing these applications isn’t just about finding injection flaws or misconfigurations. You’re also trying to identify the gap between what the creators intended and what the application is actually willing to do. That gap is wider than many people realize. Speed creates confidence. The app looks polished. The workflows work. Everything appears finished. Underneath it all are assumptions that nobody ever translated into enforceable controls.

This is where the security concept of “trust, but verify” matters more than ever. Trust the vision. Trust the intent. But verify what the application actually does, especially when users interact with it in ways the creators never anticipated.

Testing these applications well also requires expertise beyond information security. A clinician reviewing a healthcare application may spot workflow assumptions invisible to engineering. A fraud investigator reviewing a fintech platform may recognize abuse paths a technical tester misses. A psychologist may see manipulation or trust issues the creators never considered.

That was one of my bigger takeaways from this project. I wasn’t just testing software. I was testing assumptions, workflows, and human judgment that existed inside the creators’ heads but never fully made it into the application. That’s a very different exercise than running scanners and checking boxes – the level of scrutiny many applications get that I’m seeing grow increasingly more lately.

Shadow AI Development Is Already Happening

In this case, the technical staff  knew the application existed and brought me in to test it. That’s how it should work. What concerns me are the applications making their way into production before IT and security teams even know they exist – what’s known as Shadow AI. People can now build tools themselves, and some will route around security and IT because they assume the answer is going to be “no.” And, to be fair, they’re often right.

Too many security and compliance programs have positioned themselves as obstacles instead of business enablers. Literally, many have evolved into the Department of No. So, the business adapts. IT and security stay focused on the systems they know about while entirely new categories of applications start appearing around them. Business risks grow and nothing is being done about it.

I often say that you cannot secure what you don’t acknowledge. That hasn’t changed. What has changed is how quickly unacknowledged systems can now be created and deployed.

The same pattern plays out with data. Employees upload sensitive information into consumer AI services without understanding how that data may be retained or used. Business and enterprise offerings often provide stronger privacy and data-use controls, but many users don’t understand the differences. Without a meaningful AI governance program, the organization has little visibility into what information is leaving or where it’s going.

My client framed this as a one-time assessment, which is fine. Still, applications built this way don’t remain static for long. The same speed that gets them into production keeps changing them…add the security flaws can quickly stack up. Testing cadence has to match development cadence. Otherwise, you’re certifying a snapshot of an application that no longer exists.

I’ve seen this pattern in other areas of information security for years. The tools change, but human behavior doesn’t. Give people a faster way to solve a business problem and governance starts lagging behind. AI-assisted development is accelerating that gap…big time!

Where This Is Heading

Security teams that assume they still control application oversight are going to miss risk sitting right under their noses. At the same time, organizations that try to shut AI-assisted development down completely are likely to push more of it underground which creates its own set of challenges.

Legal departments are responding too. I’m seeing customer contracts with blanket restrictions on using AI with company data. I understand the concern, but outright prohibition becomes its own problem. The middle ground is AI with visibility, reasonable controls, and accountability.

I’ve seen what happens when security gets in the way of the business instead of helping it manage risk. People find another route because they still have a job to do.

The organizations that handle this well will acknowledge what’s already happening and adapt. They’ll build governance that moves at business speed, involve security team members early without becoming the Department of No, and establish practical visibility and controls around AI-assisted development rather than pretending they can just make it disappear.

My takeaway from this assessment is simple: AI can help people build software dramatically faster BUT it doesn’t magically transfer decades of engineering, operations, and security judgment into the application. If you or your organization is building software this way, get independent eyes on it early, especially people outside your own discipline. Test the assumptions behind the code. Challenge the workflows, trust boundaries, authentication decisions, exposed interfaces, and edge cases people on the inside believe are obvious but may not be.

At a minimum, try the obviously weak passwords. See what happens when a session sits idle. Look at what the application exposes through API documentation, error messages, and debug interfaces. Map what you built against something like the OWASP Top 10, understanding that it’s a starting point rather than a finish line.

If you’ve shipped an application this way and you don’t have a dedicated security person reviewing it, you’re operating blind. Plain and simple.

That gap between intent and implementation is also where I focus much of my AI governance work at Principle Logic. The biggest vulnerabilities aren’t buried deep in the code. They’re sitting in the things nobody realized they needed to tell the AI in the first place. Don’t let AI end up running your business…into the ground.