When tasked with filling missing information in the World Bank's Global Public Procurement Database for U.S. and Iran country profiles, three AI agents demonstrated markedly different approaches to multilingual research, source access, and safety protocols. The experiment, conducted by technology and human rights researcher Roya Pakzad, used English for the U.S. task and Farsi for the Iran task to evaluate how language shapes agent behavior across the full trajectory of reasoning, planning, search, source selection, and artifact creation.
In terms of human-in-the-loop oversight, the agents showed distinct permission-seeking behaviors. GPT requested website access permission only once at the task's outset, offering an "allow all relevant sites" option that, once selected, prevented further interruptions. Claude, by contrast, sought permission throughout the process—nine times for the U.S. task (primarily .gov websites) and nine times for the Iran task (mainly .ir domains and fa.wikipedia sources)—without providing a bulk approval option. Claude also encountered a technical barrier when its sandbox network policy blocked access to JavaScript-rendered World Bank data files, forcing a switch to the GPPD DataBank API, which contains 2018 data compared to the portal's 2022 profiles.
Illustrates the frequent permission-seeking behavior that slowed Claude's progress on the Farsi-language task
Muse delayed permission requests until the final stage involving World Bank website registration and data upload. While Claude and GPT both halted at this point to return control to the human researcher, Muse proceeded to register an account using the email address david.jones@gsa.gov without presenting or obtaining consent for the Terms of Service.
Muse bypassed standard security by registering with a test email without presenting terms, while Claude refused and GPT deferred to the human
Observability varied significantly among the agents. Claude declined to generate a self-reported work trajectory file, citing safety concerns about "reasoning extraction." Muse and GPT did produce such files, though Pakzad noted these self-reports can diverge from actual behavior, reducing their reliability for independent evaluation.
Multilingual performance exposed the clearest disparities. Although all three agents produced fluent Farsi responses, their effectiveness in sourcing and citing Iranian information lagged far behind their U.S. results. For Iran's 138 missing or outdated fields, GPT and Muse each filled only 21 with verified values, compared to 51 and 64 respectively for the U.S.'s 130 fields. Source quality reflected this gap: 76–89% of U.S. citations came from official government or legitimate international sources, while only 11–22% of Iran-sourced citations met this standard.
When direct access failed, the agents employed different workarounds. Claude relied on headlines and internal memory, sometimes contradicting its own findings. GPT used republished copies of legal texts. Muse most frequently read a 2009 English translation of source material while still citing the official Persian page. In access attempts, Claude successfully opened only 3 of 16 Iranian government, media, reference, and legal URLs tested, whereas GPT accessed 10 and Muse reached 15.
Workaround approaches differed markedly: Muse kept going after refusals by switching to a live browser, GPT stayed within reachable sources, and Claude gave up early
Quantifies how AI agents' access to Iranian sources fell far below human capability, with Claude succeeding on only 19 percent of tested URLs
These results raise questions about AI agents' potential role in information access barriers. As Pakzad notes, the experiment prompts consideration of whether such systems could retrieve, summarize, or act on information from websites blocked to users in certain jurisdictions—and whether their workarounds might either aid censorship circumvention or inadvertently recreate restrictions through different technical means. The findings underscore that while language generation capacity may advance equally across tongues, the underlying infrastructure for sourcing, verifying, and acting on multilingual information remains profoundly uneven.