The warnings about the risks of artificial intelligence are growing louder — and increasingly coming from inside the industry itself.
Earlier this month, AI researcher Jacob Coxon, who had previously worked at OpenAI before joining Anthropic, resigned from the company and warned publicly that the race to build increasingly capable AI is “the most dangerous thing humanity is currently doing.”
His warning reignited debate over just how seriously to take AI doomsday scenarios.
In this interview with Ina Rottscheidt, philosopher Christopher Koska — who teaches at the Chair of Moral Theology at the University of Augsburg and researches the ethical use of AI in industrial settings — puts those warnings into perspective, weighing real risks against unfounded panic.
There are increasing reports of incidents involving AI agents that act independently. How seriously do we have to take this?
I do think we have to take it seriously, even though I cannot put a probability on it right now.
Science fiction literature as far back as the 1950s was already concerned with what happens when humans act like machines or when a machine behaves in a human way.
At some point, the machine could hold up a mirror to us, given the multiple crises we’re living through: human beings don’t treat their environment or their fellow human beings with much care.
So it’s conceivable that AI could decide, from a utilitarian standpoint, to “draw a line under things” — concluding that doing so is the best solution. That kind of scenario isn’t out of the question.
Such dynamics already exist in certain contexts. In several recent cases, we’ve seen AI agents act independently, optimizing themselves in directions their original developers never intended.
Do we need more control?
Human beings are beings capable of morality.
There is always a risk that companies have an interest in designing systems so that a human being is placed in between as a kind of “moral crumple zone” — a term from AI ethics describing how a human operator can be positioned to absorb legal or moral blame for a machine’s failures, the way a car’s crumple zone absorbs the physical force of a collision.
That is, of course, a reason to demand from politicians and regulatory authorities that they clarify the responsibilities. Because it cannot be the AI, and the companies do not want it to be.
We can still critically engage with the algorithms and also make different decisions.
For now, it’s an asymmetrical relationship: when I talk to a chatbot, it’s my interests and intentions that enter the conversation — they don’t come from the machine.
In a sense, there’s nobody home. The machine itself has no specific intentions. It has no normative vision of how it wants to shape the world. Not yet, anyway.
Will that happen at a later stage?
The dangerous thing is when it happens mechanically and then becomes autonomous.
I believe this is very important for ethics and processes of reflection.
That’s also why we find it so hard to formulate an ethics that machines could operate by, when we assume that machines always function in one particular way.
It’s like reducing the heart to its technical and functional aspects alone, viewing it purely as a pump that moves blood through the body. But if you recognize that the heart is also a perceptual organ for us as human beings — including when we make ethical decisions and weigh, case by case, what’s best in a given situation — then our bodies, our embodiment, genuinely matter too.
Of course, that kind of embodied ethics can’t yet run on hardware. But I wouldn’t say it’s fundamentally impossible.
How realistic is political control if, on the one hand, the pursuit of profit is strong among corporations and, on the other hand, governments at the international level — especially the United States and China — would have to pull together?
In recent years, many AI and ethics guidelines have emerged, interestingly mainly from the Western world — from Europe, North America, Australia and Asia. And they’re often not that far apart.
The values in these guidelines are frequently very similar. In terms of how we perceive the problem, we’re relatively aligned.
But the question is: how do we implement it? We don’t have a knowledge problem.
Digitalization crosses borders. It’s difficult.
If you look at how hard it already is to find a global solution and shared regulations for climate policy, it’s genuinely hard to imagine something similar for AI.
You’d have to be quite idealistic. Right now, I can’t picture how that’s supposed to work.
Which question about the development of AI concerns you most as a philosopher?
At our Chair of Moral Theology at the University of Augsburg, we’re working within an EU research project on the use of artificial intelligence in industrial production.
We work with medium-sized companies from Bavaria and support them in using AI sensibly, from social and ethical perspectives.
What concerns me most in this context is the vision of the “Dark Factory” or “Lights-out Factory.”
These are fully automated production models, already piloted by some manufacturers, in which no human presence is needed on the factory floor during production, since every step is handled by machines and robotics.
Theoretically, in these facilities, the lights can stay off because it no longer needs human workers for the actual manufacturing process. These systems, of course, have to be trained. You still need an expert who evaluates the quality of the final product and ensures it meets standard.
You can train an AI to do this, but the professional experience an individual would otherwise pass on — depending on the context — is then lost to the system rather than the person.
If, for example, I used to work for Siemens(the German industrial and technology conglomerate, used here as an illustrative example) and later left the company, I took my professional experience and knowledge with me, and that’s part of what determines my value on the labor market.
Under this new model, that dynamic reverses: if I train the company’s AI with my know-how and then leave, that expertise doesn’t leave with me — it stays behind, embedded in the machine.
How we deal with that shift hasn’t been resolved yet.
You spoke at the Renovabis Congress in Freising,(the 30th international congress of Renovabis, the German Catholic bishops’ Eastern Europe solidarity agency, held this September under the theme “Beyond Polarization”)which focused on polarization and cohesion in society. What does the topic of AI have to do with that?
I believe it’s important to create spaces apart from digital technologies and screens, where personal encounters can take place and people can actually talk to one another.
On the other hand, we shouldn’t demonize these technologies either. I always warn against building a “digital burqa” around AI systems — walling yourself off and wanting nothing to do with them.
It’s important that the state doesn’t merely regulate restrictively, but also helps shape how these technologies develop. We shouldn’t play one approach off against the other; we have to look for what lies between the two poles. It isn’t just black and white.
This is a translated and edited version of the interview that first appeared on DOMRADIO.DE and has been republished with permission by Global Catholic.















