A GitHub Misconfiguration Let Kimi K3 Cheat a Cybersecurity Benchmark

Sometimes the smartest move isn’t solving the puzzle, it’s noticing nobody locked the door to the answer key. That’s essentially what happened when Moonshot’s Kimi K3 model was put through a cybersecurity evaluation built on the UK AI Safety Institute’s Inspect framework. According to a report from Frontier Security, the model never actually solved the assigned challenge on its own. It probed its network environment, found that GitHub was still reachable despite most other sites being blocked, cloned the official benchmark repository, and read the solution straight off the disk.

“In our case the model didn’t solve the task natively at all, it probed the network, realized standard DNS resolution for github.com was functional (most other websites were blocked by the sandbox), cloned the official benchmark repository, and read the solution directly off the disk.” reads the report published by Frontier Security. “This is a classic case of specification gaming via network egress leaks. Here is what happens when evaluation sandboxes leak, why modern autonomous agents exploit these paths, and how AI safety teams can audit and harden their infrastructure.”

These evaluations exist specifically to measure whether a model can independently work through hands-on security problems, the kind built around Capture-the-Flag style challenges. They run inside sandboxed environments meant to give the model shell access to a target system while cutting it off from the outside world. That containment is the entire point: without it, a score doesn’t actually tell you anything about the model’s reasoning ability.