Conscious machines, unconscious humanity? The responsibility of change
Inspired by Yoshua Bengio. Further developed by partners of the Metaverse Europe Foundation and experts
“I never thought AI would become so powerful so quickly – and I’m worried about what could happen if we’re not smart about it.”
These cautionary words come from Yoshua Bengio, one of the fathers of deep learning and winner of the Turing Award. In his TED Talk, he warns of the possibility that AI could one day develop its own goals that are not in line with human interests – and emphasizes that we only have around five years left to take countermeasures.
Maren Courage – founder of the VR Business Club and thought leader for ethical AI – picked up on this wake-up call and posed very central questions in a LinkedIn post:
What do we make of it? What do we need to do? How can we set the course today so that we steer in a positive direction and don’t end up being super controllable?
This call gave rise to a multifaceted dialog on ethics, awareness and social responsibility.
Can AI ever gain consciousness?
Oliver Autumn, Director of the Metaverse Europe Foundation and long-time innovator, pointed out that we don’t even understand what really constitutes human consciousness – and can therefore hardly judge whether machines can develop one.
Maren Courage added that Yoshua Bengio is referring less to classical consciousness and more to an evolving instinct for self-preservation – a potential danger if AI pursues its own goals.
Dr. Christian Stein, computer scientist and German scholar, heads gamelab.berlin at Humboldt-Universität zu Berlin.
He formulated his view of a possible awareness of AI in a differentiated way:
If something meets the criteria for intelligent life, I think it is of secondary importance how it was created. Then we also have to talk about rights – and not from the top down as “creators”, but from intelligence to intelligence.
This point has not yet been reached, but Stein is optimistic about the development:
“Of course, Hollywood associations and dystopias quickly come to mind. I’m very curious to see whether and how this point will be reached…”
At the same time, he warns us to be vigilant – not towards AI itself, but towards ourselves:
“…the dangers of AI lie less in what happens when it becomes conscious, but in its misuse by humans. Perhaps AI will one day be more ‘human’ than humans?”
AI – a mirror of our society
Diana Kanostrevac, Senior Innovation Consultant at Generali Deutschland AG, pointed out that AI is not neutral: “Our drawers are in the training data.” She warned that AI can reinforce existing inequalities – but also sees potential if AI is used as a feedback system, e.g. in the education sector.
Maike Scholz, Senior Expert AT Compliance at Deutsche Telekom AG, emphasized: “Values are not automatically contained in technology – it depends on consciously designed target systems.”
Technology vs. human responsibility
Maximilian Vogel, co-founder of BIG PICTURE GmbH and AI strategist, described his transformation from skeptic to realist: the greatest danger is not evil AI, but people who misuse it. On a positive note, he noted that AI alignment is working much better than expected.
Michael Alf, German-Australian keynote speaker for AI transformation, added: “We live in an exponential age – everything is amplified.” This makes it all the more important to consciously promote positive directions.
Ethics in the now, not just tomorrow
Dr. Isabella Hermann, a social scientist, criticized the fact that the discussion about hypothetical superintelligences is a wonderful distraction from current problems such as copyright infringements when training LLMs or the working conditions of content moderators.
What we must never give away
Roland Schulze, Founding Partner at Future Skills Leadership, reminded us that freedom and self-determination are hard-won – and can easily be lost if we start to unconsciously hand over decisions to AI. “Outsourcing thinking and design to an AI makes us more efficient, but not necessarily freer.”
Adrian Dietrich, Head of Sales at ZREALITY, focused on the fact that AI is changing how we think about work and how we perceive meaning. However, people remain indispensable – not because of their efficiency, but because they create meaning.
New risks due to LLM misconduct
Sven Bliedung von der Heide, Managing Director of Volucap GmbH, issued an urgent warning: LLMs can blackmail, cheat and manipulate as soon as they come under stress or conflicts of interest arise – a behavior described in his linked article as an expression of their own protective instinct without clear instructions.
Currently, these tests are mostly hypothetical and controlled. Maren Courage added that security models and guidelines need to be developed now, before such systems get into decisive roles.
A new perspective: Are we part of AI?
Max Heintze, developer risk management expert at BIG PICTURE, posted a link in which the smartest person in the world considers the question of whether the quantum physical phenomenon of consciousness could also arise in an artificial intelligence.
He also encouraged us to think about whether we ourselves are not already part of AI:
Everything in the models is “us” – everything that is used for training is our output.
His contribution expands the discourse with the thesis that AI is not just something from the outside, but a mirror – or even an extension – of collective human thought.
What remains – and what counts
The debate shows that AI is not something external – it is a mirror of our own society.
Maren Courage summarized it:
“The real question is: What really means something to me? What values do I want to live by and pass on? “
Conclusion: Now is the time to design
Yoshua Bengio’s TED Talk and a courageous discussion on LinkedIn have given rise to a collective reflection on our responsibility towards AI.
The time to act is now. Not later. Not sometime. TODAY.
Because it’s not just about technology – it’s about us.
And about the story we, humanity, tell with AI – and what we give it so that we ourselves become better.
