Verilay and a different AI tool checked the same app for dependency risks and didn’t fully agree. Tracing the gap made the scoring meaningfully more precise for every app Verilay checks.
This week I tightened up one of the more important numbers Verilay shows you: how many of your app’s dependencies actually put your users at risk. Here’s the improvement, and the slightly unusual way I found it.
I run Verilay on my own other apps as a matter of habit — it’s the fastest way to catch something before a real user does. This time, on LogInsight, I also did something I don’t normally bother with: I asked Lovable, the AI tool LogInsight is actually built with, to run its own independent check on the exact same code.
Two different tools, same repository, checking for the same thing. I wanted to see whether they’d agree.
They mostly did — both found the same one library, react-router, as the one genuine issue that could actually touch a real visitor. Good sign.
Where they diverged was in how loudly each one described everything else. Lovable was calm about it: most of the remaining flags were tools like vite and vitest, things that only run on my own computer while building the app, nowhere near anything a visitor could reach. Verilay’s own count was treating all of it as equally urgent.
That gap was worth closing properly rather than shrugging off, since the whole point of Verilay is that the number means something.
Rather than guess, I ran Verilay’s own dependency-checking logic directly against the real repository, outside the app, so I could watch exactly what it was doing with each finding.
Two adjustments made the biggest difference.
Severity levels needed to line up more precisely with the public vulnerability database Verilay checks against. That database rates each issue as Critical, High, Moderate, or Low — Verilay was treating Critical and High as the same tier. High is genuinely worth fixing, but it isn’t the same level of urgency as Critical, and folding them together made the headline number louder than it needed to be.
Build-only tools needed their own lane. A tool your app relies on while you’re building it — but which never ships to a real visitor’s browser — is a real thing to eventually clean up, but it isn’t the same category of risk as something a stranger could actually exploit. The interesting part: even deeply buried, indirect tools (the ones your app pulls in automatically through other tools, several layers removed from anything you chose directly) already carry an honest label inside your project’s own lockfile, saying exactly whether they’re build-only or genuinely shipped. Verilay was reading that file for version numbers and skipping the one field that actually mattered.
| Before | After | |
|---|---|---|
| How severity is weighted | Critical and High combined | Matched to the real database, tier by tier |
| Build-only tools | Counted the same as shipped ones | Reported, but no longer inflate your score |
| Cross-checked against an independent AI’s own audit | Not yet | Yes — same conclusion, same real issue found |
On LogInsight specifically, the one real, user-facing issue was still flagged correctly the whole way through. What changed is that the dozens of build-only tools sitting around it no longer drown it out.
Full technical detail is in the changelog entry for this update.
If you’re building anything that scores or ranks things automatically, it’s worth occasionally handing the same input to a second, independent tool and seeing where the two disagree. Not because your first tool is untrustworthy — because a disagreement between two honest attempts is one of the fastest ways to find the specific spot worth double-checking, out of everything you could have double-checked.
For your own AI-built app, the same idea applies at a smaller scale: if Lovable or Replit tells you something is fine and a separate check says otherwise (or the reverse), that disagreement is the interesting part, worth a few minutes to resolve — not something to average away.