The rapid integration of artificial intelligence into scientific research has sparked a debate over the definition of progress. On one side, proponents of 'output-centric' science argue that progress is defined by verifiable results: a cured disease or a proven mathematical conjecture is a success regardless of whether a human understood the underlying mechanism. This perspective suggests that if an AI can produce a correct result that passes a formal certificate of correctness or a phase 3 clinical trial, the human element becomes a secondary concern. However, critics argue that this view overlooks the fact that science is a human practice of building insight, which is what allows new findings to be integrated into existing knowledge and inspires future generations of researchers.

This push toward automated science is rooted in a 'value-free ideal' that emerged in the mid-20th century. This philosophy posits that science should be an impersonal application of methods, stripped of ethical or social values to ensure objectivity. In the age of AI, this ideal finds a technological fulfillment: AI agents can enact the scientific method at scale, potentially reaching a point where human judgment is no longer required to produce 'truth.' Yet, the author of the essay argues that this substitution of human trust with procedural trust is a mistake. When a scientific claim is produced by an agent whose logic is opaque, there is no one to consult if those claims eventually fail or if the AI exploits a hidden weakness in the testing process.

True scientific knowledge relies on 'epistemic taste' and sociality—the human capacity to decide which problems are worth solving and to build trust through shared language. Because there are an infinite number of possible truths, human values are necessary to sift through them and prioritize discoveries that align with human aspirations. Furthermore, scientific objectivity is not a solo achievement but a social process of 'transformative interrogation.' Peer review and public critique are not just automated fact-checks; they are mechanisms for community members to hold one another accountable and negotiate the boundaries of personal preference. By delegating these roles to AI, we risk a 'drift' where we produce answers without corresponding questions, eventually losing the human capacity to judge what is meaningful or how to defend the results we produce.