Troubleshooting Cloud Firewall
This document provides basic instruction on the process of troubleshooting the Cloud Firewall (FWAPI) in Triton.
Firewall API architecture and troubleshooting
This document is intended to give an overview of the Firewall API architecture: its various pieces, and how to interact and troubleshoot them.
Components
These are the components that propagate changes to firewalls in Triton:
On the compute node
- rules are managed using the
fwadmtool fwadmconverts high-level FWAPI rules into an ipf ruleset for each zone- rules and remote instances get stored in
/var/fw/ - ipf policies are stored in
/zones/:uuid/config/ipf.conf - logs from each fwadm action are stored in
`/var/log/fw/logs
Interaction with ipfilter
fwadm controls its own ipf instance, which is separate from the one visible in the zone. The zone can't see or modify the GZ-controlled ipf rules. When filtering, the rulesets are evaluated like so:
Add / update / delete rule workflow
Modifying a firewall rule goes to the Firewall API directly, which pushes the change to the firewaller agents on the compute nodes:
Provision / update / destroy instance workflow
Modifying any of the instance parameters that can affect firewalls (tags and NICs) causes an update to be posted to FWAPI (in the fwapi.update workflow task), which is then pushed to the firewaller agents on the compute nodes:
Troubleshooting
Is the firewall enabled? Is firewall_enabled set (VMAPI)?
Via VMAPI:
$ sdc-vmapi /vms/<uuid> | json -Ha firewall_enabled
trueOn the compute node:
# vmadm get <uuid> | json firewall_enabled
true
# fwadm status f7dc262f-dd0b-475f-9461-aefb0b6cb2b1
runningWhat instances does a rule apply to?
Via FWAPI (in the FWAPI zone):
# fwapi vms <uuid> | json -a uuid
ae513916-4ea3-cece-dfc0-e28b51cb74faOn the compute node:
# fwadm vms <uuid>
ae513916-4ea3-cece-dfc0-e28b51cb74faWhat rules apply to an instance?
Via FWAPI:
# fwapi rules <uuid>
[
{
"enabled": true,
"owner_uuid": "e6fcbc64-3f32-11e2-a144-bf78292e9628",
"rule": "FROM any TO vm ae513916-4ea3-cece-dfc0-e28b51cb74fa ALLOW udp PORT 54",
"uuid": "c21912b0-e2b7-471d-aee2-a024d82621ba",
"version": "1386926482712.093012"
},
{
"enabled": true,
"owner_uuid": "e6fcbc64-3f32-11e2-a144-bf78292e9628",
"rule": "FROM any TO vm ae513916-4ea3-cece-dfc0-e28b51cb74fa ALLOW tcp PORT 80",
"uuid": "e449bf0a-a3f6-4b2f-a426-73bd16b8219a",
"version": "1386926716940.093012"
},
{
"enabled": true,
"global": true,
"rule": "FROM any TO all vms ALLOW icmp TYPE 8 CODE 0",
"uuid": "27775f65-d377-4979-9c7c-63c9d4f98525",
"version": "1386743867795.003240"
}
]On the compute node:
# fwadm rules <uuid>
UUID ENABLED RULE
27775f65-d377-4979-9c7c-63c9d4f98525 true FROM any TO all vms ALLOW icmp TYPE 8 CODE 0
e449bf0a-a3f6-4b2f-a426-73bd16b8219a true FROM any TO vm ae513916-4ea3-cece-dfc0-e28b51cb74fa ALLOW udp PORT 54
c21912b0-e2b7-471d-aee2-a024d82621ba true FROM any TO vm ae513916-4ea3-cece-dfc0-e28b51cb74fa ALLOW tcp PORT 80What rules are being hit on a instance?
On the compute node:
# fwadm stats <uuid>
0 pass out quick proto tcp from any to any flags S/SA keep state
0 pass out proto tcp from any to any
0 pass out proto udp from any to any keep state
0 pass out quick proto icmp from any to any keep state
0 pass out proto icmp from any to any
0 pass in quick proto icmp from any to any icmp-type routerad
0 pass in quick proto tcp from any to any port = smtp
0 pass in quick proto icmp from any to any icmp-type echo code 0
130 block in allWhat remote instances are stored on a compute node?
On the compute node:
# fwadm list-rvms | json -a uuid
6b1db4be-63bb-11e3-bf59-db53ba2a8cefSomething has gone wrong - what information should I gather?
Logs
- In the FWAPI zone:
svcs -L fwapi - On the compute node:
svcs -L firewaller - On the compute node: logs in
/var/log/fw/and/var/log/fw/logs
Rule and remote instance data
The output of:
# fwadm list-rvms# fwadm list
Other data to gather
If appropriate:
- A gcore of the
firewallerprocess
