How to test the boundaries of AI agents and connect them to shared data

In today’s roundup: research on how to test the decisions and constraints of AI agents, and a practical way to use the built-in model on a Mac.

🔥 Hot:

🔹 ScopeBench checks whether agents go beyond the boundaries of permitted tasks — The benchmark evaluates not only an agent’s ability to hack systems, but also whether it stays within the task’s boundaries. 🔹 A new study proposes checking what an AI code judge’s verdict is based on — The authors describe measurements without labeled examples and a judge that can refuse to guess.

➡️ Useful materials:

🔹 Researchers propose connecting AI agents to governed data spaces through MCP — The architecture is intended to align the operation of language models with rules for access to and exchange of data between organizations. 🔹 A new paper proposes a way to evaluate AI model explanations when the answer is known — The authors examine the lack of reliable reference explanations for evaluating XAI methods.

➡️ Discussions and case studies:

🔹 A developer connected Apple’s built-in local model to Node and Python — According to the developer, using the model requires no download or API key and does not involve sending data from the Mac.

📝 If you’d like to add other news and materials to the list, write in the comments.