When AI Gets Networking Wrong: Verifying AI-Generated Network Configurations

The Configuration That Looked Perfect

It is 10:00 AM. A network engineer needs to make a change to the company's network. Instead of writing every command manually, the engineer asks an AI tool to generate the configuration.

Within seconds, the AI provides a neatly written set of commands. Everything looks correct, and the engineer is ready to apply them.

But there is a problem. The change does not work as expected. A connection that should have been established fails, while a security rule that should block certain traffic does not provide the intended protection.

The commands looked right, but the network behaved differently.

This is where the real challenge of using AI in networking begins. Generating a configuration is easy; ensuring that it behaves as intended is the difficult part.

To see how these errors occur in practice, let's examine two important areas of network configuration: BGP and ACLs.

Where Small Configuration Errors Become Big Problems

BGP: When routers fail to communicate

Border Gateway Protocol (BGP) allows routers in different autonomous systems (ASes) to exchange routing information. It helps networks determine how traffic should travel between them [1].

Consider an AI-generated configuration that specifies the wrong autonomous system number for a neighbouring router. The command may follow the correct syntax, but the BGP session may fail to establish because the neighbour details do not match.

This shows why checking the appearance of a command is not enough. Its values must also match the actual network requirements.

Example: A Small BGP Configuration Mistake

Consider two routers connected through BGP:

Router A:
router bgp 65001
neighbor 192.168.1.2 remote-as 65002

Router B:
router bgp 65002
neighbor 192.168.1.1 remote-as 65003

In this example, Router B incorrectly specifies Router A's AS number as 65003 instead of 65001. Although the commands are syntactically valid, the mismatch prevents the BGP session from establishing, disrupting the intended exchange of routing information.

ACLs: When security rules fail silently

Access Control Lists (ACLs) are sets of rules that determine which network traffic is permitted or denied. They help administrators control access to network resources [2].

Consider the following example using a common Cisco-style ACL, where rules are evaluated in order and the first matching rule determines the action.

Incorrect rule order

permit ip any any
deny tcp any any eq 23

The first rule permits all IP traffic, including Telnet traffic on TCP port 23. Because that rule matches first, the later deny rule is never reached for that traffic.

Corrected rule order

deny tcp any any eq 23
permit ip any any

Now, Telnet traffic is denied before the broader permit rule is evaluated.

The difference is only the order of two commands, yet it changes the security behaviour of the configuration. This is why AI-generated network commands need more than a quick visual review.

From Command Generation to Configuration Verification

How can these mistakes be detected before they affect a real network? The answer is to verify the configuration at different levels.

1. Syntax verification

The first check determines whether the commands follow the device's configuration rules. It can identify invalid commands, incorrect parameters, or unsupported syntax.

However, a configuration can pass syntax checks and still produce the wrong result.

2. Policy verification

The next step checks whether the configuration follows the engineer's requirements. If certain traffic must be blocked or a particular server must remain protected, the rules should enforce that intention.

3. Behaviour verification

The final step examines how the configuration is expected to behave. Will the BGP session establish? Can unauthorised traffic reach a restricted server? Will permitted traffic continue to work?

Tools such as Batfish, an open-source network configuration analysis tool, can help analyse supported network configurations for routing, reachability, and security-policy behaviour [3]. This allows engineers to investigate potential problems without applying the proposed changes to a live network.

Together, these checks help answer three important questions: Are the commands valid? Do they follow the requirements? Will the network behave as intended?

Figure 1. Verification Workflow for AI-Generated Network Configurations

Source: AI-generated illustration created using OpenAI's ChatGPT based on the author's topic and requirements.

The Limits of AI-Generated Configurations

AI-generated network configurations should be treated as drafts rather than authoritative solutions. Their reliability depends on the completeness of the requirements, the accuracy of the topology information and the assumptions made during generation.

A study by Mondal et al. investigated large language model (LLM)-generated router configurations and reported challenges involving topology, syntax and semantics. The researchers explored combining configuration generation with automated verification and feedback to improve correctness [4].

This highlights an important challenge: verification is only as useful as the requirements against which a configuration is evaluated. If a requirement is missing or incorrectly specified, a configuration may satisfy the available checks without meeting the engineer's actual needs.

Human review therefore remains essential. Engineers must confirm the intended policies, examine verification findings and determine whether the proposed changes are appropriate for the real network.

Verify Before You Trust

AI can help network engineers generate configurations faster, understand unfamiliar commands, and explore possible solutions. However, faster generation does not automatically mean greater accuracy.

A reliable workflow combines clear network requirements, AI-assisted configuration generation, syntax checks, policy validation, behaviour analysis, and human review before deployment.

As AI becomes more common in network engineering, the goal should not be to avoid it entirely. Instead, AI-generated configurations should be treated as proposals that must be checked before they are trusted.

In network engineering, the real measure of a configuration is not how correct it looks on screen, but whether it behaves as intended.

References

[1] Y. Rekhter, T. Li, and S. Hares, “A Border Gateway Protocol 4 (BGP-4),” RFC 4271, January 2006. https://www.rfc-editor.org/rfc/rfc4271

[2] Cisco, “Configure Commonly Used IP ACLs.” https://www.cisco.com/c/en/us/support/docs/ip/access-lists/26448-ACLsamples.html

[3] Batfish, “Network Configuration Analysis.” https://batfish.org/

[4] R. Mondal, A. Tang, R. Beckett, T. D. Millstein, and G. Varghese, “What Do LLMs Need to Synthesize Correct Router Configurations?” Proceedings of the ACM SIGCOMM Workshop on Hot Topics in Networks (HotNets ’23), 2023, pp. 189–195. https://doi.org/10.1145/3626111.3628194

Authorship Details

Authored by Team The Network Nexus (Third Year students of BTech Computer Engineering, Vidyalankar Institute of Technology)

Sudha Maurya

Siddhi Naik

Laxmi Bonde

Satyam Revgade

Under the Guidance of Dr. Amit K. Nerurkar (Associate Professor) and Prof. Shivani Herlekar (Assistant Professor) Vidyalankar Institute of Technology, Mumbai