You just watched a candidate sail through a take-home, talk beautifully in a whiteboard round, and nod through your follow-up. Two weeks later, the truth lands on your desk. The code was mostly AI-generated, the reasoning was thin, and your team is now cleaning up a hire who looked strong until the job got real.
That's the hiring problem in the era of AI. Polished output is cheap. Real judgment is still hard to fake. If you want to know how to test IT candidates in the era of AI, stop asking whether they can produce something impressive and start asking whether they can explain, defend, debug, and adapt under real constraints.
The classic stack, generic LeetCode, long unsupervised take-homes, and a behavior interview detached from the actual artifact, breaks the moment AI tools become part of the workflow. A candidate can now turn a half-baked idea into a clean-looking submission fast, and that means you're no longer measuring only skill. You're measuring access to fluent generation.
This is why the old process feels fine until it fails catastrophically. A candidate can submit a tidy solution, talk through it confidently, and still have no real command of the edge cases, tradeoffs, or debugging steps that matter once they're inside your codebase. In AI-enabled hiring, the bottleneck isn't typing speed anymore, it's separating genuine problem-solving from generated surface area.
Practical rule: If your process rewards only the final answer, you're hiring for presentation, not performance.
The market has already moved in this direction. By 2025, 51% of U.S. organizations were using AI to support HR activities, and 64% of those AI-enabled HR users applied it to recruiting, interviewing, or hiring, with about one-third using AI to screen or review resumes, according to High5Test's recruiting statistics. Employers were also planning more AI at later stages, with 69% intending to use it for candidate assessment analysis and 83% for resume review in 2025, which is exactly why candidate testing now has to account for AI at every step.
The fix is not to panic-ban tools and hope for purity. The fix is to design tests that expose judgment. That means shorter tasks, explicit rules about AI use, live defense of submitted work, and follow-ups that force candidates to show how they think when the answer isn't sitting in a model output. If you're running a startup, that's the difference between a strong senior hire and a very expensive illusion.
Underdog.io's take on why recruiting didn't get easier lands on the same reality, your process has to work harder because candidates can now work faster.
Resume review should stop being a keyword contest. AI can rewrite a LinkedIn summary in seconds, so the signal now lives in verifiable artifacts, shipped products, public technical writing, GitHub commits, and evidence that someone built, debugged, and maintained something. If the resume is polished but the trail is thin, treat that as a warning, not a win.
I'd rather see a candidate with one ugly but real shipped project than a bullet list that reads like a brand deck. Ask what they owned, what broke, what they had to roll back, and what they learned. That's harder to fake than a stack of buzzwords.
Take-homes need a reset too. Stop assigning overnight marathons. Give candidates a 60 to 90 minute exercise, hand them a half-built repo, and ask them to extend or debug it. Force a short written rationale for each design choice so you can see whether the work is theirs and whether they understand the tradeoffs.
Live coding should not be a trivia show. Replace algorithm puzzles with guided walkthroughs of unfamiliar code, then ask the candidate to explain what they would change and why. System design should also become concrete. Use operational scenarios, cost spikes, partial outages, migration debt, and real constraints from the candidate's own past work.
If you want a useful way to compare stages, use the table below and decide what signal each one must produce before it earns a slot in your funnel.
| Stage | Old Tactic | AI-Aware Redesign | Signal Captured |
|---|---|---|---|
| Resume screening | Keyword matching | Verify shipped work, commits, and technical writing | Authenticity and scope |
| Take-home | Long unsupervised build | Short repo extension with a rationale note | Problem solving and ownership |
| Live coding | LeetCode-style puzzles | Walkthrough of unfamiliar code | Debugging and communication |
| System design | Abstract architecture talk | Scenario based on real incidents | Judgment under constraints |
If you want to research how interviews can scale without turning into theater, the guide to instant UX insights is a useful reference because it treats structured evaluation as a design problem, not a vibe check. That's the mindset you want here.
The output of each stage should be different. Resume screening should tell you whether the candidate is real. The take-home should tell you whether they can build. The live round should tell you whether they can reason. The system design round should tell you whether they can make tradeoffs under pressure.
If you hire mid-level engineers, live walkthroughs of unfamiliar code are the cleanest signal. They're hard to bluff because the candidate has to read, interpret, and explain in real time. Preparation is light on the candidate side, moderate on the interviewer side, and the failure mode is obvious. People who lean too hard on AI get lost when the code doesn't match their assumptions.
Modified take-homes with checkpoints work better for junior hires. Candidates need more prep time because the assignment is more hands-on, but the interviewer can score it faster if the checkpoints are explicit. This format exposes weak debugging and shallow understanding, especially when the candidate can produce something that compiles but can't explain the edges.
Scenario-based reviews are the right choice for senior people. Give them a production incident, an architecture change, or a migration constraint, then ask them to defend the decision path. Interviewer effort is higher, but the test reveals whether the candidate can think beyond syntax and into consequences.
The best senior candidates don't just defend what they built. They explain what they would not build and why.
For teams that need to move fast, use the decision rule I trust most. If volume is high and interviewer time is tight, choose a modified take-home with a short defense round. If you're hiring for depth and leadership, prioritize scenario reviews. If you're trying to verify hands-on competence without giving away the answer, use a live walkthrough.
Underdog.io's discussion of using LLMs in coding interviews fits that same logic, use the tool if you want, but make the candidate show judgment, not just output. That's the dividing line now.
| Format | Candidate Effort | Interviewer Effort | Best For | Weakness |
|---|---|---|---|---|
| Live walkthrough of unfamiliar code | Low to moderate | Moderate | Mid-level engineers | Can under-sample deep build skills |
| Modified take-home with checkpoints | Moderate | Low to moderate | Junior hires | Easy to over-scope if you're sloppy |
| Scenario-based review | Low | High | Senior and staff roles | Requires strong interviewer calibration |
If you're choosing between formats, don't ask which one is “fair.” Ask which one reveals the most about how the person will behave on Monday morning when the code is messy and the deadline is real.
Start with the minimum control that still protects the signal. A locked browser, tab-switch tracking, webcam proctoring, and copy-paste tracking all have a place, but none of them are magic. HackerRank's write-up on tab proctoring makes the point clearly, it catches distraction and context switching, but it doesn't prove whether the answer is good or genuine on its own.
Behavioral signals matter because they catch the gaps that environment controls miss. Watch for paste-volume anomalies, impossible response speed, long pauses followed by polished output, and camera-off requests that don't fit the assessment stage. Then verify what you saw with a live code walkthrough, a follow-up debugging round on the same problem, or a system design defense.
That layered approach matters because technical cheating is rising in plain sight. One 2025 to 2026 dataset reported cheating attempts on proctored technical assessments rising from 16% in 2024 to 35% in 2025, with entry-level roles jumping from 15% to 40% and software engineering flagged at 48% versus 12% in sales, according to HR Tech Edition. Another report said 61% of flagged cheating attempts still cleared the pass threshold when no behavioral-detection layer was present, which is why answer correctness alone is a weak filter.
A sane policy is simple. Use light monitoring early, stronger verification later, and reserve heavy proctoring for finalists who've already passed a real conversation. If you're hiring senior people, don't force surveillance theater too early or you'll push good candidates away. Transparency works better than surprise.
Tell candidates up front what's monitored, what's allowed, and what stage-specific rules apply. Some companies now treat AI as off-limits unless explicitly invited, and Cisco's published guidance says that if a candidate isn't explicitly invited to use AI during the assessment process, it should be considered off-limits. That stage-specific policy is the right model because a take-home and a live round do not need the same rules.
Keep the experience human. Disclose monitoring in the job posting, store recordings only as long as your local employment rules allow, and give candidates a friction-light opt-out path that doesn't destroy the integrity signal. Heavy-handed control is not a virtue if it scares off the exact talent you want to hire.
A good rubric stops the loudest interviewer from winning the room. It also protects you from the classic failure mode where a candidate sounds sharp but can't ship. Sackett et al. 2022 reported operational validities of .42 for structured interviews, .33 for work samples, .19 for unstructured interviews, and .07 for years of experience, as summarized in Sapia's candidate screening automation tips. That's the case for structure, not intuition.
Use four dimensions. Technical depth asks whether the solution works, including edge cases. Reasoning and tradeoff articulation measures whether the candidate can explain performance, readability, and scope choices. Collaboration and communication covers clarity of thought in live rounds, responsiveness to hints, and debugging behavior. AI-augmented judgment checks whether they used tools appropriately and visibly, or hid obvious generation.
Here's the template I'd use in every interview loop.
| Dimension | 1 - Red Flag | 3 - Competent | 5 - Strong Hire | Weight |
|---|---|---|---|---|
| Technical depth | Fails on edge cases or can't explain core logic | Works on the happy path, misses some nuance | Robust, handles edge cases, makes the system safer | 40% |
| Reasoning and tradeoffs | Vague, hand-wavy, or defensive | Can explain some choices | Clear, specific, and honest about constraints | 25% |
| Collaboration and communication | Hard to follow, resists feedback | Adequate live discussion | Precise, responsive, and improves with hints | 20% |
| AI-augmented judgment | Hides tool use or can't defend it | Uses tools transparently, unevenly | Uses AI selectively and explains the boundaries | 15% |
Strong rule: If a candidate can't explain a decision, they don't own it.
The scoring sheet should also force evidence. Name the exact moment that led to each score, the line of code, the tradeoff they named, the bug they found, the hint they used. That's how you stop a strong-on-paper candidate from skating through on charisma. It also helps interviewers calibrate across teams, because disagreement becomes a discussion about evidence instead of tone.
If you need a training reference for how to measure whether a process is working, how to prove training value is useful for thinking about outcomes, even though the context is different. The same rule applies here, if you can't measure the quality of the signal, you're guessing.
A candidate gets a deliberately underspecified task. Build a small rate limiter for a webhook ingestion service. The repo already has a stale code stub with a subtle bug, and the candidate has 30 minutes to work. They can use any tooling they want, including AI assistants, but they must document what they delegated, what they kept manual, and why.
A strong candidate keeps the API clean, writes a short test file, and explains which AI-generated snippets they rejected because they did not fit the design. They add one tight paragraph on memory ceiling and backpressure, then justify the simpler limiter strategy instead of over-engineering the path. They also point out the subtle bug in the stub and why it matters.
The weak candidate generates code that compiles, ignores the rate-limit semantics, and drops in a copy-pasted README. The artifact looks polished, but the reasoning is thin or borrowed when you ask why the design works. That is the failure pattern you need to catch.
The next round makes the format useful. First, the candidate defends the design for ten minutes. Then the interviewer injects a regression and asks them to pair-debug it. If they understood the first artifact, they recover quickly. If they only produced output, the cracks show immediately.
A startup hiring team can use Underdog.io as one source of curated tech candidates, then apply a process like this to separate real operators from polished generators. The marketplace gives you access to candidates, the assessment tells you who can genuinely own the work.
The decision is straightforward. Strong submission plus clear defense plus fast debugging equals hire. Clean-looking artifact without reasoning equals no. In the AI era, the artifact is just the opening move. The signal is whether the candidate can stand behind it.
Consent comes first. If you use webcam monitoring, tab tracking, or any similar proctoring, get explicit written consent before the assessment starts. Spell out which stages allow AI and which do not, store candidate data under the GDPR or the employment rules that apply to you, and give every flagged submission a clear appeal path.
Candidate experience matters just as much. Send the rubric before the interview. Cap the total assessment time at three to four hours. Pay for take-homes that run longer than ninety minutes. Return feedback within five business days. If you cannot do that, the process is too heavy.
Candidate experience guidance for hiring teams supports that approach. Clear expectations and low-friction communication keep strong candidates engaged. The point is not suspicion. The point is a process strict enough to trust.
Days 1 to 30: Audit your current assessments, pick two pilot formats, and train interviewers on anchored rubrics. Days 31 to 60: Run pilots on five to eight candidates, log signal strength against time cost, and tighten the rubric where interviewers disagree. Days 61 to 90: Lock the new pipeline, publish an honest candidate FAQ, and review fairness and pass-through quality every quarter.
If your rollout needs a heroic launch, it is already too big. Start with one role family and one hiring team.
The founder checklist stays short. Get consent. Disclose AI rules. Pay fairly for long work. Store data carefully. Give feedback fast. Do those five things, and you get a process that is strict where it should be and humane where it matters.
