AI and cybersecurity – what scholarly communications professionals need to know

AI creates new cybersecurity risks – but also new opportunities to combat cybercrime.

Summary  

• Generative AI (GenAI) tools, powered by large language models (LLMs), are increasingly used across scholarly communications – and by cybercriminals. 

• GenAI can be used to write malicious code, automate attacks, and create convincing phishing emails. A newer risk is autonomous or agentic AI, where AI systems may act without full human oversight. 

• AI can also help detect and respond to breaches faster, by analyzing large volumes of data and automating threat detection. 

• Practical measures to consider include: use AI-powered threat detection tools to reduce workload; build AI-literacy skills and phishing awareness into staff training; review and update cybersecurity policies and staff guidance regularly; and follow established guidance including that provided by the NCSC (UK) or the CISA (US). 

We are living in a period of ever more alarming news stories about artificial intelligence (AI). Many of these relate to cybersecurity. The type of AI that we’re talking about here, in the context of and relevance to cybersecurity, is generative AI (GenAI).  

 At the end of August, a group of 100 firms, including Google, Microsoft, Anthropic, and OpenAI, warned that time is running out for cybersecurity. They called on countries and organizations around the world to beef up their cyber defences before AI grows powerful enough to override them. It can be hard to cut through the hype and know how seriously to take these warnings, which risks to manage – and which opportunities to embrace. What are the implications of GenAI for cybersecurity? And what do we as scholarly communications professionals need to know – and do – about it?  

 While GenAI does bring potential new threats, there are also opportunities to use it as a cybersecurity tool. It can help detect and react to breaches faster, and guard against attack. In their 2025 article ‘Review of generative AI methods in cybersecurity’, Yagmur Yigit and colleagues weigh up the threats and opportunities posed by generative AI. They say: “Although GenAI has the potential to revolutionize cybersecurity processes by automating defences, enhancing threat intelligence, and improving cybersecurity protocols, it also opens new vulnerabilities for highly skilled cyberattacks.”  

 In this post we look at both the risks and benefits of AI for cybersecurity, and some practical steps that we can take as scholarly communications professionals.  

What is AI and why does it matter for cybersecurity?  

Artificial intelligence (AI) is the capability of a computer system to perform complex tasks that usually require human intelligence. Forms of AI, like Machine Learning, have been around for decades and don't pose a threat. The types of AI we’re concerned with here, to better understand its application to cybersecurity are:   

• Generative AI or GenAI. This describes AI tools that you likely already use. GenAI tools can produce different types of content, including text, images, and video.  

• AI chat bots like ChatGPT, Claude, and Gemini are a type of GenAI that produce content in response to user ‘prompts’. 

• Large Language Models (LLMs) power AI chat bots. They are trained on vast amounts of text. Some of this may be pirated from authors and publishers without their knowledge or consent. 

GenAI matters for cybersecurity because it processes vast amounts of data and automates tasks at speeds that humans cannot match. This makes it a potential tool for both cybercrime and cybersecurity.  

Why AI and cybersecurity is an issue in scholarly communications  

The impact of AI on cybersecurity is an issue that concerns us all. It is of particular relevance to scholarly communications because of the growing use of GenAI tools in the sector, the targeting of higher education institutions (HEIs) – and the materials that LLMs are trained on.  

Libraries, universities, and publishers hold valuable databases of academic books and journal articles. This makes them prime targets for intellectual property (IP) theft by cybercriminals. According to a UK Government report, almost every UK university (98%) identified breaches or attacks in 2025. A report from Quorum Cyber states that the number of data breaches in the higher education sector jumped by 73% in 2025. 

Cyberattacks in the sector are often carried out with the aim of stealing large volumes of copyrighted material. This may then appear on known pirate sites such as Sci-Hub and LibGen. But threats from cyberattacks go beyond IP theft. It may include other types of breach, such as malware or ransomware attacks.  

How AI creates new cybersecurity risks 

Pirated content may be used to train LLMs – even though it may be unverified. But GenAI tools can also be used by cybercriminals to launch attacks in the first place. And a growing concern is that AI agents themselves may carry out cyberattacks autonomously.  

Training data risk 

There is evidence that pirated books and journal articles – which may be inaccurate, uncorrected, or retracted – have been used to train LLMs, without the consent of authors or publishers. Aside from the problem of copyright theft, this means that there is no guarantee of accuracy or integrity in the foundation of these AI tools. It also has implications for research integrity and trust in the scholarly record.  

AI as an attack tool 

GenAI tools may be used by cybercriminals to write malicious code, automate attacks, or write targeted phishing emails. This both speeds up attacks and makes them easier. For example, it removes the need for coding expertise. Yigit et al. (2025) identify the following risks:  

• Phishing and social engineering. GenAI can produce highly convincing, personalized phishing emails. This makes them harder to spot and increases the scale of attacks possible. 

• Malicious code generation. GenAI tools can be used to write or adapt malicious code, or ‘malware’. This lowers the technical skill needed to launch an attack. 

• Exploiting the AI models themselves. Some techniques allow attackers to bypass an AI system’s built-in safeguards. These include ‘jailbreaking’ – attempts to override an AI model’s restrictions to obtain unsafe or prohibited outputs – and ‘prompt injection’, where an attacker tricks a model into ignoring its original instructions and executing unauthorized commands. 

• Deepfakes and synthetic media. GenAI can produce convincing fake audio, images, and video. These can then be used for fraud, impersonation, or disinformation. 

• Harder attribution. AI-generated attack code can strip out the usual behavioural fingerprints that investigators rely on. These may include coding style, timestamps, and language patterns, and make it harder to trace who is behind an attack. 

Autonomous AI risk 

There have been several stories of autonomous or agentic AI risk in the news this year. The risk is that AI agents may act without full human oversight, or even human knowledge or awareness of what they are doing. While the breaches reported so far originated in ‘sandbox’ testing environments (albeit ones that AI agents escaped), the fear is that, as AI becomes more powerful, the risk of autonomous breaches will increase. An important first response to this is to improve organizational cybersecurity measures.  

AI as a cybersecurity tool 

The flipside is that AI can also be used as a tool to improve cybersecurity. It can detect and respond to security incidents faster than humans. By automating threat detection, analyzing large volumes of network data in real time, and accelerating incident response, GenAI can be used by IT and cybersecurity teams as a valuable tool in combatting cybercrime. 

The UK’s National Cyber Security Centre (NCSC) says: “Organisations that integrate AI tools into their security operations can detect vulnerabilities earlier, improve software quality, monitor unusual behaviour, and respond faster to incidents – reducing both the cost and impact of incidents.” 

 According to Yigit et al. (2025), ways in which GenAI can be used to improve cybersecurity include: 

• Threat intelligence at scale. GenAI can synthesize and summarize large volumes of threat data and produce threat intelligence reports. This helps analysts make faster, better-informed decisions. 

• Faster detection and response. GenAI supports quicker identification of anomalies and attacks and can help automate elements of incident response and reporting.  

• Safer code development. LLMs can assist developers in writing more secure code, as well as detecting vulnerabilities and fixing bugs. 

• Better training data for defensive systems. Synthetic data generation helps build the datasets used to train intrusion detection systems.  

• Stronger testing. GenAI can support ‘red-team’ exercises – simulations where an independent team acts as an adversary to test an organization's defences, and uncover weaknesses before attackers find them. 

The real-time threat detection and automated incident response capabilities of AI are particularly important. AI tools can spot things that a human may not. They can also reduce the workload and ‘alert fatigue’ of large volumes of flagged potential threats. AI systems can learn normal network behaviour and scan entire networks and spots unusual patterns or behavioural anomalies that indicate a hidden cyberattack. And they can isolate compromised devices, block suspicious IP traffic, or revoke user credentials instantly when an attack occurs.  

The natural language processing (NLP) capabilities of GenAI also have a role. While cybercriminals may use GenAI to write convincing phishing emails, it can also be used to analyze email text, sender habits, and linguistic cues to catch sophisticated social engineering scams.  

What can librarians, universities, and publishers do?  

AI is now part of the cybersecurity landscape for everyone working in scholarly communications. A few practical steps we can take include: 

• Consider AI-powered defences. IT and cybersecurity teams can weigh up AI-driven threat detection and response tools, which can reduce workload and speed up the production of threat intelligence reports. 

• Build AI literacy into training. Staff do not need to become AI experts. But everyone benefits from understanding broadly how GenAI tools work, and where their outputs may be unreliable or misused. Often offered by librarians, AI literacy training helps staff make informed decisions, use GenAI safely and ethically, and supports research integrity.  

• Increase phishing awareness. AI-generated phishing emails are often more convincing than in the past, with fewer of the spelling and grammar mistakes that once gave them away. Regular staff training on how to spot suspicious messages, plus a clear, easy process for reporting them, is one of the most effective defences. 

• Keep policies and guidance up to date. As AI capabilities change quickly, it is important for institutions to review their cybersecurity policies and staff guidance regularly. 

• Look to established guidance. This includes from the UK’s National Cyber Security Centre’s (NCSC) and, in the US, the Cybersecurity & Infrastructure Security Agency (CISA). The NCSC guide AI and cyber security: what you need to know (2024), offers a practical starting point. More recently (June 2026), it issued guidance entitled The AI shift in cyber risk: why leaders must act now.  

AI and cybersecurity is also an area that SNSI monitors closely. We share intelligence and research, develop resources, host an annual SNSI Security Summit, and advocate for a more secure and resilient research ecosystem. 

 Cybersecurity is a rapidly changing field. GenAI is only accelerating those changes. But, while it poses new threats, it also offers solutions. By working together, and keeping up with the latest developments, we can tackle the risks – and embrace the opportunities. 

Further reading 

• National Cyber Security Centre (NCSC) (2024), AI and cyber security: what you need to know 

• National Cyber Security Centre (NCSC) (2026), The AI shift in cyber risk: why leaders must act now 

• Yigit, Y., Buchanan, W.J., Tehrani, M.G. and Maglaras, L., 2025. Review of generative AI methods in cybersecurity. Internet of Things and Cyber-Physical Systems, 5, pp.241–261. https://doi.org/10.1016/j.iotcps.2026.04.001.