507개 테스트를
AI가 만들었다 507 Tests
Written by AI

"What happens if you don't write tests?"

Honestly, I didn't even know what test code was at first. If the code runs, it's fine, right? Click around in the browser, check the output, done. In 12 years of working in the recruiting industry, "test automation" was an interview question I asked candidates -- not something I ever expected to do myself.

But as features piled up, things got scary. Fix the JD parsing logic and resume analysis breaks. Modify the matching algorithm and the report format goes haywire. Fix one thing, three things break. Developers call this "regression bugs."

A Non-Developer's Testing Fear

Convince-X's Candidate Analyzer runs on a 3-stage AI pipeline: resume extraction → JD structuring → matching analysis. Each stage depends on the others, so a change in one triggers a chain reaction.

In the early days, I manually tested every feature by opening the browser -- uploading a JD, uploading a resume, eyeballing the results. This worked with 10 features. Past 20, it became impossible.

"Well, just write tests then" is the obvious response. But I didn't know how to write test code. Jest? Mocha? Assertions? For 12 years as a recruiter, "What's your test coverage?" was a question I asked candidates -- and suddenly I had to do it myself.

That's where AI re-enters the picture.

414 Security Tests

Security tests came first. The reason is simple: running a SaaS means handling customer data, and a security breach means game over.

There's a well-known list called OWASP Top 10 -- the ten most common security vulnerabilities in web applications. I asked AI to "create tests for each OWASP Top 10 category against our codebase."

The first batch of AI-generated security tests was about 280. Running them revealed actual vulnerabilities -- 3 endpoints that weren't properly sanitizing inputs, 2 routes missing CSRF validation. Building tests actually caught bugs.

That experience completely changed my perception of testing. Tests aren't "verification" -- they're a "discovery tool."

93 Functional Tests

After security came functional tests -- verifying the core features of Candidate Analyzer: JD parsing, resume analysis, and matching logic.

Functional tests were harder to build than security tests. Security tests have clear pass/fail criteria ("this input should be blocked"), but functional tests require defining "what result should come out of parsing this JD?" -- and that's hard.

This is where my domain expertise mattered. When AI was building "JD tech stack extraction tests," I provided real-world JD patterns from the field. AI can't easily distinguish between "Python experience preferred" and "Python required" on its own. 12 years of reviewing thousands of JDs directly improved the quality of test cases.

How AI Writes Tests

Many people imagine "AI writes tests" means pressing a button and getting perfect output. Reality is different.

Here's the actual process:

The key is iteration in Step 3. AI's first draft of tests only works about half the time. Wrong import paths, misconfigured mocks, missed async handling. You send these errors back to AI for 2-3 rounds before getting stable tests.

Building 507 tests took about 2 weeks. Not pure coding time, but the result of this iterative process done a little each day.

Why Deployments Aren't Scary

What changes when you have 507 tests?

Deployments aren't scary anymore. That's the biggest change.

Before tests, every deployment came with anxiety. "Will this break existing features?" I'd pray while hitting the deploy button. I actually had to rollback from production issues two or three times.

Now it's different. After modifying code, one npm test command runs all 507 tests in under 30 seconds. If anything fails, you know exactly what broke.

The peace of mind from 414 security tests is especially valuable. Every time I add a new API endpoint, I don't need to manually check "Is there an XSS vulnerability? Can authentication be bypassed?" -- the tests verify automatically.

The Circular Validation Trap

Here's where I need to be honest.

AI-written code tested by AI. This structure has a fundamental limitation. When the same AI writes the code and validates it, both can be wrong in the same direction. This is called "circular validation."

After recognizing this problem, we introduced an AI cross-review system. A separate AI model independently reviews code changes. Every modification triggers multi-stage AI verification covering security, performance, and code quality.

Even this isn't perfect. The ultimate conclusion is that "real checks" can only come from a human. A key insight from the AI C-Suite system: "An AI wearing different hats is not real oversight. Real oversight can only come from the CEO."

607 Tests and Beyond

The test count that started at 507 has grown to 607. Each new feature -- the email sequence engine, candidate pipeline, i18n for 3 languages -- added more tests alongside it.

607 tests, 100% pass rate, and zero production incidents. The confidence these numbers provide is bigger than you'd expect.

But don't get intoxicated by the numbers. Even with 607 tests, there are areas they can't cover. Testing the "quality" of AI output remains inherently difficult. "Is this matching analysis correct?" is a question that ultimately requires human judgment.

That's why tests are a "safety net," not a "guarantee." What 607 tests tell you is "basic functionality and security work" -- not "the product is flawless." Maintaining that humility is important.

A non-developer building 607 automated tests with AI assistance. That's the reality of 2026. And this is just the beginning.