This year’s AI Congress, the leading artificial intelligence event promoted by CIDAI and organised by technology centre Eurecat, highlighted the importance of aligning the automation capabilities of agentic AI with human judgement in order to address key challenges related to the security and governance of this technology from business, societal and environmental perspectives.

The congress, taking place this week in Barcelona, pointed to a future in which AI deployment increasingly focuses on customised models that run locally and independently on organisations’ own infrastructure. This approach helps maintain control over data, safeguard privacy, reduce dependence on external providers and improve efficiency, offering a competitive alternative to cloud-based frontier AI models.

According to Ariadna Font, Co-founder and CEO of Alinia AI, “agentic AI multiplies not only productivity, but also the volume and speed of errors. A person makes a mistake once, but an agent may repeat the same mistake a thousand times before anyone realises it”.

For this reason, she stressed that “the most urgent priority is to invest in robust governance and control frameworks that enable the safe and responsible use of the latest technological advances”. While “there is currently a race to see who gets there first in AI, I do not believe the winner will be the organisation with the most AI, but the one that maintains human oversight and judgement”.

In this regard, Joan Mas, Director of the AI Congress and Scientific Director of Eurecat’s Digital Area, highlighted that “agentic AI has rapidly emerged over the past year and offers tremendous opportunities to achieve genuinely significant levels of automation. At the same time, it raises important questions about the safeguards and protections that must be built into these agents to ensure their decisions remain aligned with human judgement”.

During the institutional opening session of the AI Congress, Daniel Altimiras, President of Eurecat, emphasised that “Catalonia has the knowledge, talent, technological capabilities and business ecosystem needed to play a significant role in this new phase of artificial intelligence”. The challenge, he added, “is no longer simply understanding what AI can do, but integrating it effectively, securely and responsibly into our organisations and turning it into competitiveness, productivity and new growth opportunities”.

“For the Government of Catalonia, we advocate purpose-driven artificial intelligence focused on solving real-world challenges, creating public value, strengthening the competitiveness of our businesses and improving the services we provide to citizens,” said Albert Tort, Secretary for Telecommunications and Digital Transformation of the Government of Catalonia.

Towards an AI model that delivers real value

According to Jezabel R. Garcia, Chief AI Scientist at Multiverse Computing, “the next revolution in AI is not a larger model, but a model that operates where it creates value: on a device, within a company, inside a hospital or across a public administration. Compression enables us to maintain a model’s capabilities while reducing its size and energy consumption by more than 50 percent. This is what makes sovereign, efficient and sustainable AI systems possible in Europe.”

The AI Congress, now in its twelfth edition, also highlighted the importance of ensuring that AI adoption starts with a clear definition of the outcomes organisations aim to achieve. In this context, international digital transformation expert Pooja Munshi underscored the concept of intentional AI, defined as artificial intelligence aligned with specific objectives and with the needs of people, organisations and the environment.

Drawing on her recent work with the United Nations Environment Programme, Pooja Munshi explored how leaders can develop intentional AI solutions that transform technological capabilities into meaningful and lasting impact. She also encouraged leaders to ask better questions and listen carefully to people, evidence and the world affected by the technologies they develop.

For her part, Meritxell Borràs, Director of the Catalan Data Protection Authority (APDCAT), addressed the challenges that agentic AI poses for data protection when agents process personal data, as well as the need to establish effective human oversight mechanisms throughout their operation.

During the event, participants also presented examples of agents and multi-agent systems applied across different sectors and shared recommendations for their deployment to ensure technical security against emerging threats and risks, while also complying with the requirements of the Artificial Intelligence Act (AI Act).

Aligned with the Catalonia AI 2030 Strategy promoted by the Government of Catalonia, the AI Congress received support from CaixaBank Tech, Conversia, Forvis-Mazars-Infini, Orange Empresas, Raona, Softeng, APDCAT and Laboral Kutxa. The event brought together more than 50 speakers and attracted more than 2,200 registered participants.