Firewall reviews sound straightforward: export the configuration, find risky rules, and send the network team a list.
The real work is harder.
A rule can look narrow until its address or service objects are expanded. A policy can say inspection is enabled while no inspection profile is attached. A rule may appear unused in a static export, but only traffic logs can establish whether it is actually unused. A naming convention may suggest that a path crosses from development into production, but names are not evidence of a business boundary.
The difficult part is not producing more findings. It is producing findings that are technically correct, traceable to source evidence, honest about what cannot be concluded, and usable by the people who must review and remediate them.
That is the problem we set out to solve. The latest work is now available in the open-source firewall-review skill in the Transilience community tools repository.
The skill is a reusable knowledge and workflow layer for Claude, Codex, and other agent-based security workflows. It documents how to approach firewall configuration review, rule analysis, evidence handling, custom policy checks, and the final network-team handoff.
The Problem Statements
We started with six recurring problems.
1. Firewall Rules Hide Their Effective Behavior
Firewall policy tables are full of indirection. A rule references address groups, nested objects, service aliases, schedules, zones, interfaces, and inspection profiles. The name shown in the policy table is not always the behavior enforced by the device.
A custom service object with a reassuring name may resolve to every TCP port. A policy marked for UTM inspection may have no profile attached. A privileged administrator may be restricted to a “trusted” network that is actually an entire private address block.
A useful review must resolve those relationships and preserve the original source evidence.
2. Static Configuration Is Not Runtime Evidence
Configuration exports can prove what is configured. They cannot prove everything about what happened.
A static review cannot safely conclude that a rule had zero traffic, that a path was never reachable, or that a policy is operationally unnecessary unless the required runtime evidence is available. Traffic logs, routing state, session data, object state, and other live evidence may change the conclusion.
This distinction matters. “No hit-count evidence was provided” is defensible. “This rule is unused” may not be.
3. Business Intent Is Usually Missing
The configuration can show source, destination, service, action, and device context. It often cannot show why the rule exists.
Questions such as these require business context:
- Is development allowed to initiate connections into production?
- Which systems are part of the cardholder data environment?
- Is this partner path approved and monitored?
- Does this rule match the service communication matrix?
- Is there a valid exception, owner, and expiration date?
Object names can help generate candidates, but they cannot replace an authoritative scope map or approved policy source.
4. Free-Form AI Review Is Difficult to Reproduce
An unconstrained model can produce a useful review, but the coverage may vary from one run to the next. One pass notices management-plane exposure. Another focuses on broad egress. A third spends its attention on naming quality.
That variability is a problem when someone asks, “What exactly did you check on every firewall?”
The review needs a stable catalogue of named questions, explicit evidence requirements, and a recorded outcome for every check—including when the answer is CANT-TELL.
5. Device Hardening and Rule Quality Are Different Reviews
A clean rulebase does not mean the firewall itself is hardened. Password settings, administrative timeouts, SNMP configuration, VPN behavior, logging, insecure protocols, and other system settings need their own review.
The reverse is also true: a hardened device can still contain dangerously broad or misleading rules.
The workflow has to keep rule analysis, contextual policy review, and product-hardening benchmarks separate while presenting them coherently.
6. A Finding List Is Not a Collaboration Workflow
The deliverable must work for network teams, security teams, auditors, and reviewers.
People need one place to see the observation, severity, affected rules, source evidence, recommendation, owner response, justification, closure status, and evidence gaps. They also need filterable rule-level detail, even when several rules share the same observation.
Presentation cannot come at the cost of traceability.
What We Solved
The open-source skill turns those problem statements into a repeatable review method.
A Structured Detector Catalogue
The skill documents 22 detector patterns: 17 vendor-agnostic checks plus five FortiGate-specific checks introduced through the recent firewall-review work.
The FortiGate checks cover:
- Administrative idle timeouts above the policy ceiling.
- SSL-VPN timeout and reauthentication weaknesses.
- Overly broad trusted-host ranges for privileged administrators.
- UTM enabled without an attached inspection profile.
- Custom service objects that span all ports.
These checks focus on issues that are easy to miss in a spreadsheet review because the effective risk sits outside the visible rule row.
See the firewall-review detector references.
A Repeatable Semantic Review
The semantic-check catalogue defines 15 named questions across seven categories, including trust boundaries, management-plane exposure, regulated paths, egress monitoring, segmentation, multi-tenant boundaries, and service-object hygiene.
Instead of asking an agent to “review this firewall,” the workflow asks the same questions for each applicable device and records one of four outcomes:
- Pass.
- Fail.
- Cannot tell from the available evidence.
- Not applicable.
The result is a coverage matrix, not just a variable list of findings.
An Evidence-State Contract
The evidence-state contract separates:
- Confirmed static facts.
- Deterministic derivations.
- Conservative candidates.
- Runtime evidence gaps.
- Business-context gaps.
- External evidence.
- Unassessed scope.
This is one of the most important parts of the work. It keeps a useful observation from becoming an unsupported conclusion.
If the configuration proves that a service object spans ports 1–65535, the workflow can say so directly. If traffic logs are missing, it does not convert absence of evidence into a zero-use claim. If a name suggests a production boundary but no scope map exists, the result remains a candidate or CANT-TELL.
Data-Driven Custom Policy Checks
The custom policy benchmark provides a consistent way to evaluate organization-specific requirements such as:
- Development or UAT access into production.
- PCI or regulated environment boundaries.
- Partner connectivity.
- Approved service communication matrices.
- Exception ownership and expiration.
The policy source is treated as data. Naming conventions may help identify what to inspect, but the final verdict depends on authoritative context.
That makes the workflow reusable without embedding any organization’s names, network ranges, rules, or exceptions into the skill.
A Separate FortiGate Hardening Layer
The CIS FortiGate benchmark reference separates checks that can be resolved from configuration from checks that require runtime validation.
The documented benchmark profile contains 64 checks: 45 with deterministic configuration logic and 19 that remain warnings because static evidence alone cannot settle them.
This preserves a clean boundary between what the exported configuration proves and what still needs live verification.
A Network-Team Review Workbook Contract
The network-team review workbook profile defines a single-file handoff with complete rule coverage, evidence gaps, editable response fields, and quality gates.
It supports both:
- Filterable row-level findings.
- A grouped presentation where one approved observation and severity can span several affected rules.
Grouping is limited to presentation fields. Every rule keeps its own evidence, recommendation, and response data, and the workbook preserves separate filterable source sheets.
That solves a practical tension: executives and reviewers want consolidated observations, while engineers need the exact rule-level detail required to act.
What the Open-Source Skill Is—and Is Not
The repository is intentionally clear about the capability boundary.
The firewall-review package is an open-source methodology and reference specification. It contains the operating guidance, detector definitions, evidence rules, review catalogue, benchmark profile, and reporting contract. It does not ship the external Python runtime or claim that every documented detector and renderer is executable from this repository.
The implementation status documents that boundary and the acceptance gates required before describing a runtime capability as implemented.
That distinction is part of evidence-safe engineering too. Documentation should not make stronger product claims than the available code and tests support.
A Better Firewall Review Outcome
The goal is not a larger spreadsheet.
The goal is a review in which every conclusion has a known evidence state, every finding traces back to the configuration, every unresolved question stays visible, every custom policy decision uses authoritative context, and every affected rule reaches the network team in an actionable format.
That changes the central question from:
What looks wrong in this firewall?
to:
What can we prove, what can we derive, what still needs context, and what should the network team do next?
If you are building cloud security, compliance, or agent-driven firewall review workflows, start with the open-source firewall-review skill. Use the pieces you need, contribute improvements, and keep the evidence boundary explicit.
That is how firewall review becomes repeatable—and how an AI-assisted review becomes something an engineer or auditor can actually defend.
