Setup Azure CloudShell

1. Setup your AzureCloud Shell

• Login to Azure Cloud Portal https://portal.azure.com/ with the provided login/password

  

• Click the link “Skip for now (14 days until this is required)” do not click the “Next” button

  

• Click the “Next” button

  

• Click on Cloud Shell icon on the Top Right side of the portal

  

• Select Bash

  

• Click on Mount Storage Account

  

• Select

  • Storage Account Subscription - Internal-Training
  • Apply

• Click Select existing Storage account, Click Next

  

• in Select Storage account Step,

  • Subscription: Internal-Training
  • Resource Group: Select the Resource group from the drop down: K8sXX-K8s101-workshop
  • Storage Account: Use existing storage account from dropdown.
  • File share: Use cloudshellshare
  • Click Select

  

• After 1-2 minutes, You should now have access to Azure Cloud Shell console

  

Introduction to FortiWeb and its functionalities

What is FortiWeb

FortiWeb is a web application firewall (WAF) that protects hosted web applications from attacks targeting known and unknown exploits. Using multi-layered and correlated detection methods, FortiWeb defends applications from known vulnerabilities and zero-day threats. The Web Application Security Service from FortiGuard Labs utilizes information based on the latest application vulnerabilities, bots, suspicious URL and data patterns, and specialized heuristic detection engines to keep your applications safe.

FortiWeb also offers a machine-learning function that enables it to automatically detect malicious web traffic. In addition to detecting known attacks, the feature can detect potential unknown zero-day attacks to provide real-time protection for web servers.

Configurable Features

FortiWeb allows you to configure these features:

• Vulnerability scanning and patching
• IP reputation, web application attack signatures, credential stuffing defense, anti-virus, and FortiWeb Cloud Sandbox powered by FortiGuard
• Real-time attack insights and reporting with advanced visual analytics tools
• Integration with FortiGate and FortiSandbox for ATP detection
• Behavioral attack detection
• Advanced false positive and negative detection avoidance

FortiWeb hardware and virtual machine platforms are available for medium and large enterprises, as well as for service providers.

Benefits

FortiWeb is designed specifically to protect web servers. It provides specialized application layer threat detection and protection for HTTP and HTTPS services, including:

• Apache Tomcat
• nginx
• Microsoft IIS
• JBoss
• IBM Lotus Domino
• Microsoft SharePoint
• Microsoft Outlook Web App (OWA)
• RPC and ActiveSync for Microsoft Exchange Server
• Joomla
• WordPress

FortiWeb’s integrated web-specific vulnerability scanner drastically reduces challenges associated with protecting regulated and confidential data by detecting your exposure to the latest threats, especially the OWASP Top 10.

FortiWeb’s HTTP firewall and denial-of-service (DoS) attack-prevention protects your web applications from attack. Using advanced techniques to provide bidirectional protection against sophisticated threats like SQL injection and cross-site scripting (XSS) attacks, FortiWeb also helps you defend against threats like identity theft, financial fraud, and corporate espionage.

FortiWeb provides the tools you need to monitor and enforce government regulations, industry best practices, and internal security policies, including firewalling and patching requirements from PCI DSS.

Features

FortiWeb’s application-aware firewall and load balancing engine can:

• Secure HTTP/HTTPS applications.
• Prevent and reverse defacement.
• Improve application stability.
• Monitor servers for downtime & connection load.
• Reduce response times.
• Accelerate SSL/TLS.*
• Accelerate compression.
• Rewrite content on the fly.

*On VM models, acceleration is due to offloading the cryptography burden from the back-end server. On hardware models, cryptography is also hardware-accelerated via ASIC chips.

FortiWeb significantly reduces deployment costs by consolidating WAF, hardware acceleration, load balancing, and vulnerability scanning in a single platform with no per-user pricing. These features:

• Reduce the total resources required to protect your regulated, Internet-facing data.
• Ease the challenges associated with policy enforcement and regulatory compliance.

Introduction to Kubernetes Ingress Controllers and FortiWeb Integration

Overview of Ingress Controller

An Ingress Controller is a component in Kubernetes that manages the routing of external HTTP and HTTPS traffic to services within a Kubernetes cluster. It listens for Ingress resources created by users and configures the load balancer accordingly. Essentially, it acts as an entry point for external traffic, routing requests to the appropriate services based on the defined rules.

Why Ingress Controller is Needed

1. Centralized Traffic Management: It centralizes the traffic management, allowing for easy routing rules setup.
2. Load Balancing: Distributes incoming traffic across multiple pods, ensuring high availability and reliability.
3. SSL Termination: Manages SSL/TLS termination, providing secure connections to the services.
4. Path-Based Routing: Allows for routing traffic to different services based on the request path.
5. Name-Based Virtual Hosting: Supports routing based on the hostname, enabling multiple applications to run on the same IP address.

FortiWeb Ingress Controller Overview

The FortiWeb Ingress Controller fulfills Kubernetes Ingress resources and allows you to manage FortiWeb objects from Kubernetes. It is deployed in a container within a pod in a Kubernetes cluster.

The major functionalities of FortiWeb Ingress Controller include:

1. List and Watch Ingress Resources: Manages Ingress-related resources such as Ingress, Service, Node, and Secret.
2. Convert Ingress Resources to FortiWeb Objects: Converts resources to FortiWeb objects, such as virtual servers, content routing, real server pools, and more.
3. Handle Events for Ingress Resources: Automatically implements corresponding actions on FortiWeb for Add/Update/Delete events.

Key Strengths of FortiWeb Ingress Controller

Integration of WAF and Ingress Controller Features

1. Web Application Firewall (WAF) Capabilities: Includes WAF capabilities, providing protection against common web vulnerabilities such as SQL injection, cross-site scripting (XSS), and other OWASP top 10 threats.
2. Comprehensive Security: Ensures that the traffic routed to the applications is secure, mitigating various web-based attacks at the ingress point itself.
3. Multi-Layered Protection: Uses multi-layered and correlated detection methods to defend applications from known vulnerabilities and zero-day threats.

Embedded Machine Learning (ML) Capabilities

1. Adaptive Security: The embedded ML capability allows FortiWeb to adapt and improve its threat detection over time. It can analyze traffic patterns and identify anomalies indicative of potential attacks.
2. Zero-Day Threat Protection: ML can detect and respond to zero-day threats by identifying suspicious behaviors that signify a new, previously unknown threat.
3. Real-Time Threat Mitigation: Offers real-time threat mitigation by continuously learning and adapting to new threat vectors, providing an extra layer of security for applications deployed in Kubernetes.

Additional Features of FortiWeb Ingress Controller

1. Health Check: Ensures that the services are running correctly and efficiently.
2. Traffic Log Management: Manages and logs traffic for monitoring and analysis.
3. FortiView: Provides visibility into the traffic and security events, facilitating the management of Kubernetes ingress resources.

Importance of ML in Addressing Zero-Day Threats

• Proactive Defense: ML-based systems can predict and identify new attack patterns that have not yet been cataloged in traditional databases.
• Dynamic Adaptation: Unlike static rule-based systems, ML systems evolve with the data they process, enhancing their capability to spot and mitigate new types of attacks.
• Behavioral Analysis: ML algorithms can analyze the behavior of users and traffic patterns to detect anomalies that could signify a zero-day threat.

In summary, the FortiWeb Ingress Controller’s integration of WAF and ingress features, coupled with its embedded ML capabilities, provides a robust solution for securing Kubernetes applications. The ML capabilities are particularly crucial for detecting and mitigating zero-day threats, ensuring that applications remain protected against evolving security challenges.

Deployment Models of Ingress Controllers

Most Ingress Controllers, such as NGINX or Kong, are deployed inside Kubernetes as a Kubernetes Deployment. They run in containers within the cluster, managing ingress traffic internally.

In contrast, the FortiWeb Ingress Controller operates differently. It is backed by a FortiWeb VM or physical appliance, with a light agent deployed in Kubernetes. This agent uses APIs to communicate with the external FortiWeb VM or appliance.

Benefits of FortiWeb Ingress Controller

Benefits

1. Enhanced Security:

   • Dedicated Hardware: Using a physical appliance or VM provides dedicated resources for security operations, potentially enhancing performance and security.
   • Advanced WAF Features: FortiWeb appliances come with sophisticated Web Application Firewall capabilities that might surpass those available in container-based solutions.

2. Resource Efficiency:

   • Offloading Workload: By offloading security processing to an external appliance, it reduces the resource consumption within the Kubernetes cluster, freeing up resources for other workloads.

3. Scalability:

   • Independent Scaling: The FortiWeb appliance can be scaled independently of the Kubernetes cluster. This allows for flexible resource allocation based on traffic and security needs without affecting the cluster’s performance.

Install and Setup FortiWeb Ingress Controller

Network Diagram

In this chapter, we are going to create a lab setup as illustrated in the network diagram below.

FortiWeb can be configured with two ports: port1 for incoming traffic and port2 for proxy traffic to the backend application. This is called the twoarms mode here.

FortiWeb TwoLegs Mode

FortiWeb can also be configured with a single port, where port1 handles both incoming traffic and proxy traffic to the backend application. This is called the one-arm mode.

FortiWeb OneArm Mode

1. Prepare Environment Variables

read -p "Enter deploy mode (twoarms/onearm) [onearm]: " fortiwebdeploymode
fortiwebdeploymode=${fortiwebdeploymode:-onearm}
echo $fortiwebdeploymode 
if [ "$fortiwebdeploymode" == "twoarms" ]; then
    secondaryIp="10.0.2.100"
else
    secondaryIp="10.0.1.100"
fi
owner="tecworkshop"
currentUser=$(az account show --query user.name -o tsv)
resourceGroupName=$(az group list --query "[?contains(name, '$(whoami)') && contains(name, 'workshop')].name" -o tsv)
#resourceGroupName=$(az group list --query "[?tags.UserPrincipalName=='$currentUser'].name" -o tsv)
if [ -z "$resourceGroupName" ]; then
    resourceGroupName=$owner-$(whoami)-"fortiweb-"$location-$(date -I)
    az group create --name $resourceGroupName --tags UserPrincipalName=$currentUser --location $location
    resourceGroupName=$resourceGroupName
fi
location=$(az group show --name $resourceGroupName --query location -o tsv)
echo "Using resource group $resourceGroupName in location $location"

cat << EOF | tee > $HOME/variable.sh
#!/bin/bash -x
vnetName="fortiweb-VNET"
aksVnetName="AKS-VNET"
imageName="fortinet:fortinet_fortiweb-vm_v5:fortinet_fw-vm:latest"
fortiwebUsername="azureuser"
fortiwebPassword='Welcome.123456!'
fortiwebvmdnslabel="$(whoami)fortiwebvm7"
aksClusterName=$(whoami)-aks-cluster
rsakeyname="id_rsa_tecworkshop"
vm_name="$(whoami)fortiwebvm7.${location}.cloudapp.azure.com"
fortiwebvmdnslabelport2="$(whoami)px2"
svcdnsname="$(whoami)px2.${location}.cloudapp.azure.com"
remoteResourceGroup="MC"_${resourceGroupName}_$(whoami)-aks-cluster_${location} 
nicName1="NIC1"
nicName2="NIC2"
alias k=kubectl
EOF
echo fortiwebdeploymode=$fortiwebdeploymode >> $HOME/variable.sh
echo secondaryIp=$secondaryIp >> $HOME/variable.sh
echo location=$location >> $HOME/variable.sh
echo owner=$owner >> $HOME/variable.sh
echo resourceGroupName=$resourceGroupName >> $HOME/variable.sh
chmod +x $HOME/variable.sh
line='if [ -f "$HOME/variable.sh" ]; then source $HOME/variable.sh ; fi'
grep -qxF "$line" ~/.bashrc || echo "$line" >> ~/.bashrc
source $HOME/variable.sh
$HOME/variable.sh
if [ -f $HOME/.ssh/known_hosts ]; then 
grep -qxF "$vm_name" "$HOME/.ssh/known_hosts"  && ssh-keygen -R "$vm_name"
fi

2. Create Kubernetes Cluster

We can use either managed K8s like AKS, EKS or self-managed k8s like kubeadm etc., in this workshop, let’s use AKS.

We will create aks VNET and fortiweb VNET in same resourceGroup, in reality, you can also create them in different resourceGroup.

az network vnet create -g $resourceGroupName  --name  $aksVnetName --location $location  --subnet-name aksSubnet --subnet-prefix 10.224.0.0/24 --address-prefix 10.224.0.0/16

aksSubnetId=$(az network vnet subnet show \
  --resource-group $resourceGroupName \
  --vnet-name $aksVnetName \
  --name aksSubnet \
  --query id -o tsv)
echo $aksSubnetId

[ ! -f ~/.ssh/$rsakeyname ] && ssh-keygen -t rsa -b 4096 -q -N "" -f ~/.ssh/$rsakeyname

az aks create \
    --name ${aksClusterName} \
    --node-count 1 \
    --vm-set-type VirtualMachineScaleSets \
    --network-plugin azure \
    --location $location \
    --service-cidr  10.96.0.0/16 \
    --dns-service-ip 10.96.0.10 \
    --nodepool-name worker \
    --resource-group $resourceGroupName \
    --kubernetes-version 1.28.9 \
    --vnet-subnet-id $aksSubnetId \
    --only-show-errors \
    --ssh-key-value ~/.ssh/${rsakeyname}.pub
az aks get-credentials -g  $resourceGroupName -n ${aksClusterName} --overwrite-existing

kubectl get node  -o wide

NAME                             STATUS   ROLES   AGE     VERSION   INTERNAL-IP   EXTERNAL-IP   OS-IMAGE             KERNEL-VERSION      CONTAINER-RUNTIME
aks-worker-12061195-vmss000000   Ready    agent   8m51s   v1.28.9   10.224.0.4    <none>        Ubuntu 22.04.4 LTS   5.15.0-1064-azure   containerd://1.7.15-1

az network vnet list -g $resourceGroupName -o table

k8s51 [ ~ ]$ az network vnet list -g $resourceGroupName -o table
Name      ResourceGroup          Location    NumSubnets    Prefixes       DnsServers    DDOSProtection    VMProtection
--------  ---------------------  ----------  ------------  -------------  ------------  ----------------  --------------
AKS-VNET  k8s51-k8s101-workshop  eastus      1             10.224.0.0/16                False

3. Prepare to deploy fortiweb VM in dedicated VNET

In this workshop, We are going to deploy fortiweb VM in it’s own VNET, fortiweb will use twoarms or onearm deployment model, below lists the components going to be deployed

• VNET : 10.0.0.0/16
• Subnet1: 10.0.1.0/24
• Subnet2: 10.0.2.0/24 when fortiweb in twoarms mode
• NSG : allow all traffic
• NIC1 with Public IP for SSH access and Management, in Subnet1
• NIC2 for internal traffic, in Subnet2, when fortiweb in twoarms mode
• VM with Extra DISK for log

az network vnet create \
  --resource-group $resourceGroupName \
  --name $vnetName \
  --location $location \
  --address-prefix 10.0.0.0/16 \
  --subnet-name ExternalSubnet \
  --subnet-prefix 10.0.1.0/24

if [ "$fortiwebdeploymode" == "twoarms" ]; then 
az network vnet subnet create \
  --resource-group $resourceGroupName \
  --vnet-name $vnetName \
  --name InternalSubnet \
  --address-prefix 10.0.2.0/24
fi

az network nsg create \
  --resource-group $resourceGroupName \
  --location $location \
  --name MyNSG

az network nsg rule create \
  --resource-group $resourceGroupName \
  --nsg-name MyNSG \
  --name AllowAll \
  --protocol '*' \
  --direction Inbound \
  --priority 1000 \
  --source-address-prefix '*' \
  --source-port-range '*' \
  --destination-address-prefix '*' \
  --destination-port-range '*'

az network public-ip create \
  --resource-group $resourceGroupName \
  --location $location \
  --name FWBPublicIP \
  --allocation-method Static \
  --sku Standard \
  --dns-name $fortiwebvmdnslabel \
  --only-show-errors 

az network nic create \
  --resource-group $resourceGroupName \
  --location $location \
  --name NIC1 \
  --vnet-name $vnetName \
  --subnet ExternalSubnet \
  --network-security-group MyNSG \
  --public-ip-address FWBPublicIP

az network nic update \
    --resource-group $resourceGroupName \
    --name NIC1 \
    --ip-forwarding true

if [ "$fortiwebdeploymode" == "twoarms" ]; then 
az network nic create \
  --resource-group $resourceGroupName \
  --location $location \
  --name NIC2 \
  --vnet-name $vnetName \
  --subnet InternalSubnet \
  --network-security-group MyNSG

az network nic update \
    --resource-group $resourceGroupName \
    --name NIC2 \
    --ip-forwarding true
fi 

4. Deploy fortiweb VM

if [ "$fortiwebdeploymode" == "twoarms" ]; then 
nics="NIC1 NIC2" 
else
nics="NIC1"
fi

az vm create \
  --resource-group $resourceGroupName \
  --name MyfortiwebVM \
  --size Standard_F2s \
  --image $imageName \
  --admin-username $fortiwebUsername \
  --admin-password $fortiwebPassword \
  --nics $nics \
  --location $location \
  --public-ip-address-dns-name $fortiwebvmdnslabel \
  --data-disk-sizes-gb 30 \
  --ssh-key-values @~/.ssh/${rsakeyname}.pub \
  --only-show-errors

{
  "fqdns": "k8s52fortiwebvm7.eastus.cloudapp.azure.com",
  "id": "/subscriptions/02b50049-c444-416f-a126-3e4c815501ac/resourceGroups/k8s52-k8s101-workshop/providers/Microsoft.Compute/virtualMachines/MyfortiwebVM",
  "location": "eastus",
  "macAddress": "60-45-BD-D6-B4-8C",
  "powerState": "VM running",
  "privateIpAddress": "10.0.1.4",
  "publicIpAddress": "52.170.217.44",
  "resourceGroup": "k8s52-k8s101-workshop",
  "zones": ""
}

{
  "fqdns": "k8s51fortiwebvm7.eastus.cloudapp.azure.com",
  "id": "/subscriptions/02b50049-c444-416f-a126-3e4c815501ac/resourceGroups/k8s51-k8s101-workshop/providers/Microsoft.Compute/virtualMachines/MyfortiwebVM",
  "location": "eastus",
  "macAddress": "60-45-BD-D8-14-AF,60-45-BD-D8-1D-FE",
  "powerState": "VM running",
  "privateIpAddress": "10.0.1.4,10.0.2.4",
  "publicIpAddress": "13.90.210.29",
  "resourceGroup": "k8s51-k8s101-workshop",
  "zones": ""
}

az resource list -g $resourceGroupName -o table

k8s51 [ ~ ]$ az resource list -g $resourceGroupName -o table
Name                                                    ResourceGroup          Location    Type                                        Status
------------------------------------------------------  ---------------------  ----------  ------------------------------------------  --------
AKS-VNET                                                k8s51-k8s101-workshop  eastus      Microsoft.Network/virtualNetworks
k8s51-aks-cluster                                       k8s51-k8s101-workshop  eastus      Microsoft.ContainerService/managedClusters
FortiWeb-VNET                                           k8s51-k8s101-workshop  eastus      Microsoft.Network/virtualNetworks
MyNSG                                                   k8s51-k8s101-workshop  eastus      Microsoft.Network/networkSecurityGroups
FWBPublicIP                                             k8s51-k8s101-workshop  eastus      Microsoft.Network/publicIPAddresses
NIC1                                                    k8s51-k8s101-workshop  eastus      Microsoft.Network/networkInterfaces
NIC2                                                    k8s51-k8s101-workshop  eastus      Microsoft.Network/networkInterfaces
MyfortiwebVM                                            k8s51-k8s101-workshop  eastus      Microsoft.Compute/virtualMachines
MyfortiwebVM_disk2_1a5d56afec9745dba51cfed47fd133dc     K8S51-K8S101-WORKSHOP  eastus      Microsoft.Compute/disks
MyfortiwebVM_OsDisk_1_6259c4a932fe4cfd866015e1fb611558  K8S51-K8S101-WORKSHOP  eastus      Microsoft.Compute/disks

ssh -o "StrictHostKeyChecking=no" azureuser@$vm_name -i $HOME/.ssh/$rsakeyname 

ssh -o "StrictHostKeyChecking=no" azureuser@$vm_name -i $HOME/.ssh/$rsakeyname "get system status"

5. Create VNET Peering

Because AKS and fortiweb are in different VNETs, they are isolated from each other. We are going to use VNET Peering to connect fortiweb VM with AKS workernode. To do that, we need to get both vnetIds to create peering.

localPeeringName="fortiwebToAksPeering"
remotePeeringName="AksTofortiwebPeering"

localVnetId=$(az network vnet show --resource-group $resourceGroupName --name $vnetName --query "id" -o tsv)

remoteVnetId=$(az network vnet show  --resource-group $resourceGroupName --name $aksVnetName  --query "id" -o tsv)
echo $remoteVnetId

az network vnet peering create \
  --name $localPeeringName \
  --resource-group $resourceGroupName \
  --vnet-name $vnetName \
  --remote-vnet $remoteVnetId \
  --allow-vnet-access

az network vnet peering create \
  --name $remotePeeringName \
  --resource-group $resourceGroupName \
  --vnet-name $aksVnetName \
  --remote-vnet $localVnetId \
  --allow-vnet-access

az network vnet peering list -g $resourceGroupName --vnet-name AKS-VNET -o table
az network vnet peering list -g $resourceGroupName --vnet-name fortiweb-VNET -o table

AllowForwardedTraffic    AllowGatewayTransit    AllowVirtualNetworkAccess    DoNotVerifyRemoteGateways    Name                  PeeringState    PeeringSyncLevel    ProvisioningState    ResourceGroup          ResourceGuid                          UseRemoteGateways
-----------------------  ---------------------  ---------------------------  ---------------------------  --------------------  --------------  ------------------  -------------------  ---------------------  ------------------------------------  -------------------
False                    False                  True                         False                        AksTofortiwebPeering  Connected       FullyInSync         Succeeded            k8s51-k8s101-workshop  e867030a-0101-00b2-19a0-fba24c2151dd  False
AllowForwardedTraffic    AllowGatewayTransit    AllowVirtualNetworkAccess    DoNotVerifyRemoteGateways    Name                  PeeringState    PeeringSyncLevel    ProvisioningState    ResourceGroup          ResourceGuid                          UseRemoteGateways
-----------------------  ---------------------  ---------------------------  ---------------------------  --------------------  --------------  ------------------  -------------------  ---------------------  ------------------------------------  -------------------
False                    False                  True                         False                        fortiwebToAksPeering  Connected       FullyInSync         Succeeded            k8s51-k8s101-workshop  e867030a-0101-00b2-19a0-fba24c2151dd  False

6. Verify the connectivity between fortiweb VM and AKS

nodeIp=$(kubectl get nodes -o jsonpath='{range .items[*]}{.status.addresses[?(@.type=="InternalIP")].address}{"\n"}{end}')
echo $nodeIp

ssh -o "StrictHostKeyChecking=no" azureuser@$vm_name -i ~/.ssh/$rsakeyname execute ping $nodeIp

MyfortiwebVM # PING 10.224.0.4 (10.224.0.4): 56 data bytes
64 bytes from 10.224.0.4: icmp_seq=1 ttl=64 time=2.5 ms
64 bytes from 10.224.0.4: icmp_seq=2 ttl=64 time=1.1 ms
64 bytes from 10.224.0.4: icmp_seq=3 ttl=64 time=1.2 ms
64 bytes from 10.224.0.4: icmp_seq=4 ttl=64 time=1.2 ms
64 bytes from 10.224.0.4: icmp_seq=5 ttl=64 time=15.0 ms

7. Config fortiweb VM

FortiWeb requires some basic configuration to work with ingress Controller config list:

1. enable HTTPS API access on TCP port 443
2. enable traffic log
3. config static route

• static route to AKS vnet subnet via Port1
• default route to internet via Port2 when use fortiweb in twoarms mode
• static route to your client IP (your azure shell) via Port1
  • This ensures your client session (your azure shell) can SSH into fortiweb via Port1 public ip.

##get your azure shell client ip
myclientip=$(curl -s https://api.ipify.org)
echo $myclientip 

cat << EOF | tee > basiconfig.txt
config system global
  set admin-sport 443
end
config log traffic-log
  set status enable
end
EOF
cat << EOF | tee > interfaceport2config.txt
config system interface
  edit "port2"
    set type physical
    set allowaccess ping ssh snmp http https FWB-manager 
    set mode dhcp
  next
end
EOF

cat << EOF | tee > staticrouteconfigtwoarms.txt
config router static
  edit 10
    set dst 10.224.0.0/16
    set gateway 10.0.1.1
    set device port1
  next
  edit 2000
    set dst 0.0.0.0/0
    set gateway 10.0.2.1
    set device port2
  next
  edit 1000
    set dst $myclientip
    set gateway 10.0.1.1
    set device port1
  next
end
EOF

cat << EOF | tee > staticrouteconfigonearm.txt
config router static
  edit 10
    set dst 10.224.0.0/16
    set gateway 10.0.1.1
    set device port1
  next
EOF

ssh -o "StrictHostKeyChecking=no" azureuser@$vm_name  -i  ~/.ssh/$rsakeyname < basiconfig.txt

if [ "$fortiwebdeploymode" == "twoarms" ]; then 
ssh -o "StrictHostKeyChecking=no" azureuser@$vm_name  -i  ~/.ssh/$rsakeyname < interfaceport2config.txt
ssh -o "StrictHostKeyChecking=no" azureuser@$vm_name  -i  ~/.ssh/$rsakeyname < staticrouteconfigtwoarms.txt
else 
ssh -o "StrictHostKeyChecking=no" azureuser@$vm_name  -i  ~/.ssh/$rsakeyname < staticrouteconfigonearm.txt
fi

ssh -o "StrictHostKeyChecking=no" azureuser@$vm_name  -i  ~/.ssh/$rsakeyname show router static

8. SSH into fortiweb via internal ip

You may lose connectivity to fortiweb Public IP via SSH if your client ip subnet is not in fortiweb static route config. In this case, you can use fortiweb internal IP for ssh. We can create an ssh client pod to connect to fortiweb via internal IP.

cat << EOF | tee sshclient.yaml 
apiVersion: v1
kind: Pod
metadata:
  name: ssh-jump-host
  labels:
    app: ssh-jump-host
spec:
  containers:
  - name: ssh-client
    image: alpine
    command: ["/bin/sh"]
    args: ["-c", "apk add --no-cache openssh && apk add --no-cache curl && tail -f /dev/null"]
    stdin: true
    tty: true
EOF

kubectl apply -f sshclient.yaml

nic1privateip=$(az network nic show --name NIC1 -g $resourceGroupName  --query "ipConfigurations[0].privateIPAddress" --output tsv)
echo $nic1privateip
echo username $fortiwebUsername
echo password $fortiwebPassword
kubectl exec -it po/ssh-jump-host -- ssh $fortiwebUsername@$nic1privateip

9. Use Helm to deploy fortiweb Ingress controller

• What is Helm

Helm is a package manager for Kubernetes that simplifies the deployment and management of applications within Kubernetes clusters. It uses charts, which are pre-configured packages of Kubernetes resources. Helm also uses Helm repositories, which are collections of charts that can be shared and accessed by others, facilitating the distribution and collaboration of Kubernetes applications. If you use the Azure Cloud Shell, the Helm CLI (Helm v3.6.3 or later ) is already installed. For installation instructions on your local platform, see Installing Helm https://helm.sh/docs/intro/install/

fortiwebingresscontrollernamespace="fortiwebingress"
releasename="fortiweb-ingress-controller/fwb-k8s-ctrl"

helm repo add fortiweb-ingress-controller https://fortinet.github.io/fortiweb-ingress/

helm repo update

kubectl create namespace $fortiwebingresscontrollernamespace

helm install first-release $releasename --namespace $fortiwebingresscontrollernamespace

NAME: first-release
LAST DEPLOYED: Tue Jun 11 03:19:14 2024
NAMESPACE: fortiwebingress
STATUS: deployed
REVISION: 1
TEST SUITE: None

helm get manifest first-release -n $fortiwebingresscontrollernamespace 

kubectl rollout status deployment first-release-fwb-k8s-ctrl -n fortiwebingress

kubectl logs -n 50 -l app.kubernetes.io/name=fwb-k8s-ctrl -n $fortiwebingresscontrollernamespace

Stopping fortiweb ingress controller
Starting fortiweb ingress controller
time="2024-06-11T03:19:34Z" level=info msg="==Starting fortiweb Ingress controller"

10. Deploy Backend Application in AKS

We will deploy two service and expose with ClusterIP SVC , service1 and service2

imageRepo="public.ecr.aws/t8s9q7q9/andy2024public"
cat << EOF | tee > service1.yaml
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: sise
spec:
  replicas: 1
  selector:
    matchLabels:
      app: sise
  template:
    metadata:
      labels:
        app: sise
    spec:
      containers:
      - name: sise
        image: $imageRepo:demogeminiclient0.5.0
        imagePullPolicy: Always
        env: 
          - name: PORT
            value: "9876"
          - name: GEMINI_API_KEY
            value: ""
---
kind: Service
apiVersion: v1
metadata:
  name: service1
  annotations:
    health-check-ctrl: HLTHCK_ICMP
    lb-algo: round-robin
spec:
  type: NodePort
  ports:
  - port: 1241
    protocol: TCP
    targetPort: 9876
  selector:
    app: sise
  sessionAffinity: None

EOF
kubectl apply -f service1.yaml
kubectl rollout status deployment sise

imageRepo="public.ecr.aws/t8s9q7q9/andy2024public"
cat << EOF | tee > service2.yaml
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: goweb
spec:
  replicas: 1
  selector:
    matchLabels:
      app: goweb
  template:
    metadata:
      labels:
        app: goweb
    spec:
      containers:
      - name: goweb
        image: $imageRepo:demogeminiclient0.5.0
        imagePullPolicy: Always
        env: 
          - name: PORT
            value: "9876"
---
kind: Service
apiVersion: v1
metadata:
  name: service2
  annotations:
    health-check-ctrl: HLTHCK_ICMP
    lb-algo: round-robin
spec:
  type: NodePort
  ports:
  - port: 1242
    protocol: TCP
    targetPort: 9876
  selector:
    app: goweb
  sessionAffinity: None
EOF
kubectl apply -f service2.yaml
kubectl rollout status deployment goweb

kubectl get ep service1
kubectl get ep service2

NAME       ENDPOINTS          AGE
service1   10.224.0.22:9876   14s
NAME       ENDPOINTS          AGE
service2   10.224.0.19:9876   6s

11. Create ingress rule with yaml file

FortiWeb ingress controller is the default ingress controller, it will read and parse the ingress rule. the ingress controller will also read annotation from yaml file for some configuration parameters like fortiweb login ip and secrets etc., We will tell fortiweb ingress controller use fortiweb port1 ip for API access, and create VIP on fortiweb Port2, the VIP address is on same subnet with Port2 with last octet set to .100.

Use the script below to get fortiweb Port1 and Port2 IP address , then create yaml file with these IP address

output=$(ssh -o "StrictHostKeyChecking=no" azureuser@$vm_name -i ~/.ssh/$rsakeyname 'get system interface')
port1ip=$(echo "$output" | grep -A 7 "== \[ port1 \]" | grep "ip:" | awk '{print $2}' | cut -d'/' -f1)
if [ "$fortiwebdeploymode" == "twoarms" ]; then
port2ip=$(echo "$output" | grep -A 7 "== \[ port2 \]" | grep "ip:" | awk '{print $2}' | cut -d'/' -f1)
echo port2ip=$port2ip
vip=$(echo "$port2ip" | cut -d'.' -f1-3).100
else 
vip=$(echo "$port1ip" | cut -d'.' -f1-3).100
fi
echo port1ip=$port1ip
echo vip=$vip

kubectl create secret generic fwb-login1 --from-literal=username=$fortiwebUsername --from-literal=password=$fortiwebPassword

if [ "$fortiwebdeploymode" == "twoarms" ]; then
vipport="port2"
else
vipport="port1"
fi
cat << EOF | tee > 04_minimal-ingress.yaml 
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: m
  annotations: {
    "fortiweb-ip" : $port1ip,    
    "fortiweb-login" : "fwb-login1",  
    "fortiweb-ctrl-log" : "enable",
    "virtual-server-ip" : $vip,
    "virtual-server-addr-type" : "ipv4",
    "virtual-server-interface" :$vipport, 
    "server-policy-web-protection-profile" : "Inline Standard Protection",
    "server-policy-https-service" : "HTTPS",
    "server-policy-http-service" : "HTTP",
    "server-policy-syn-cookie" : "enable",
    "server-policy-http-to-https" : "disable"
  }
spec:
  ingressClassName: fwb-ingress-controller
  rules:
  - host: $fortiwebvmdnslabelport2
    http:
      paths:
      - path: /generate
        pathType: Prefix
        backend:
          service:
            name: service1
            port:
              number: 1241
      - path: /info
        pathType: Prefix
        backend:
          service:
            name: service2
            port:
              number: 1242
EOF

kubectl logs -f  -l app.kubernetes.io/name=fwb-k8s-ctrl -n fortiwebingress &  

kubectl apply -f 04_minimal-ingress.yaml

12. fortiweb Configuration

You will see now fortiweb has configured a few thingss.

 ssh -o "StrictHostKeyChecking=no" azureuser@$vm_name -i ~/.ssh/$rsakeyname 'get system vip'

ssh -o "StrictHostKeyChecking=no" azureuser@$vm_name -i ~/.ssh/$rsakeyname show server-policy policy 

ssh -o "StrictHostKeyChecking=no" azureuser@$vm_name -i ~/.ssh/$rsakeyname  show server-policy vserver 

ssh -o "StrictHostKeyChecking=no" azureuser@$vm_name -i ~/.ssh/$rsakeyname  show server-policy server-pool

ssh -o "StrictHostKeyChecking=no" azureuser@$vm_name -i ~/.ssh/$rsakeyname  show server-policy http-content-routing-policy

13. Verify ingress rule

kubectl get ingress

cat << EOF | tee > clientpod.yaml
apiVersion: v1
kind: Pod
metadata:
  name: clientpod
  labels: 
    app: clientpod
spec:
  containers:
  - name: clientpod
    image: praqma/network-multitool
EOF
kubectl apply -f clientpod.yaml

nodePort=$(kubectl get svc service1 -o jsonpath='{.spec.ports[0].nodePort}')
kubectl exec -it po/clientpod -- curl  http://$nodeIp:$nodePort/info 

ssh -o "StrictHostKeyChecking=no" azureuser@$vm_name -i $HOME/.ssh/$rsakeyname execute curl http://$nodeIp:$nodePort/info

MyfortiwebVM #   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0*   Trying 10.224.0.10:30890...
* Connected to 10.224.0.10 (10.224.0.10) port 30890
> GET /info HTTP/1.1
> Host: 10.224.0.10:30890
> User-Agent: curl/8.4.0
> Accept: */*
> 
< HTTP/1.1 200 OK
< Content-Type: application/json
< Date: Tue, 25 Jun 2024 01:59:56 GMT
< Content-Length: 20
< 
{ [20 bytes data]
100    20  100    20    0     0HTTP/1.1 200 OK
Content-Type: application/json
Date: Tue, 25 Jun 2024 01:59:56 GMT
Content-Length: 20

{"version":"0.6.0"}
   5333      0 --:--:-- --:--:-- --:--:--  6666
* Connection #0 to host 10.224.0.10 left intact

14. Create secondary ip and associate with public ip

This is to create an IP to use as VIP on fortiweb and associate with a public ip for external access, when run fortiweb with twoarms mode the secondary ip is on NIC2 , when in onearm mode, the secondary ip is on NIC1.

if [ "$fortiwebdeploymode" == "twoarms" ]; then
vipnicname="NIC2"
else        
vipnicname="NIC1"  
fi  
az network public-ip create \
  --resource-group $resourceGroupName \
  --name FWBPublicIPPort2 \
  --allocation-method Static \
  --sku Standard \
  --dns-name $fortiwebvmdnslabelport2


# Add a secondary IP configuration to NIC2
az network nic ip-config create \
  --resource-group $resourceGroupName \
  --nic-name $vipnicname \
  --name ipconfigSecondary \
  --private-ip-address $secondaryIp \
  --public-ip-address FWBPublicIPPort2

az network nic show \
  --resource-group $resourceGroupName \
  --name $vipnicname \
  --query "ipConfigurations[]" \
  --output table

Name               Primary    PrivateIPAddress    PrivateIPAddressVersion    PrivateIPAllocationMethod    ProvisioningState    ResourceGroup
-----------------  ---------  ------------------  -------------------------  ---------------------------  -------------------  ---------------------
ipconfig1          True       10.0.2.4            IPv4                       Dynamic                      Succeeded            k8s51-k8s101-workshop
ipconfigSecondary  False      10.0.2.100          IPv4                       Static                       Succeeded            k8s51-k8s101-workshop

Name               Primary    PrivateIPAddress    PrivateIPAddressVersion    PrivateIPAllocationMethod    ProvisioningState    ResourceGroup
-----------------  ---------  ------------------  -------------------------  ---------------------------  -------------------  ---------------------
ipconfig1          True       10.0.1.4            IPv4                       Dynamic                      Succeeded            k8s51-k8s101-workshop
ipconfigSecondary  False      10.0.1.100          IPv4                       Static                       Succeeded            k8s51-k8s101-workshop

Verify connectivity to fortiweb VIP FortiWeb has VIP configured which it’s an alias of NIC2 interface(in twoarms mode) or NIC1 (in onearm mode). from client pod, you shall able to ping it.

kubectl exec -it po/clientpod -- ping -c 5 $secondaryIp

Reach ingress rule via fortiweb reverse proxy on VIP

Because fortiweb has configured with reverseProxy on VIP with ingress rule. client pod shall able to access url via fortiweb.

We have add “Host: $svcdnsname” in HTTP request Host header, as this is required in the ingress rule definition. the target application is gemini AI client. so we can send request data with your “prompt”.

kubectl exec -it po/clientpod -- curl -v -H "Host: $svcdnsname" http://$secondaryIp:80/generate  -H "Content-Type: application/json" -d '{"prompt": "hi"}' | grep "HTTP/1.1 200 OK"

you shall get the response from backend server like this , which indicate you do not have Token for use gemini yet.

Access ingress service via external public ip or dns name

kubectl exec -it po/clientpod -- curl http://$svcdnsname/info 

15. Clean up:

• delete all resource

if you want startover again, you can delete all resource then redo the installation

resources=$(az resource list -g $resourceGroupName --query "[].{name:name, type:type}" -o tsv)
az resource list -g $resourceGroupName -o table
echo delete aks cluster
az aks delete --name $aksClusterName -g $resourceGroupName 
echo delete fortiweb vm 
az vm delete --name MyfortiwebVM -g $resourceGroupName
echo delete nic 
az network nic delete --name NIC1 -g $resourceGroupName 
az network nic delete --name NIC2 -g $resourceGroupName 

echo delete public ip 
az network public-ip delete --name FWBPublicIP -g $resourceGroupName
az network public-ip delete --name FWBPublicIPPort2 -g $resourceGroupName
echo delete fortiwebvm disk
disks=$(az disk list -g $resourceGroupName --query "[].name" -o tsv)
for disk in $disks; do
az disk delete --name $disk --resource-group $resourceGroupName 
done
echo delete NSG
az network nsg delete --name MyNSG --resource-group $resourceGroupName
echo delete vnet
az network vnet delete --name $vnetName -g $resourceGroupName
az network vnet delete --name $aksVnetName -g $resourceGroupName
az resource list  -g $resourceGroupName -o table 
rm ~/.kube/config
ssh-keygen -R $vm_name

routing and load balancing

Basic configuration settings for routing and load balancing

FortiWeb is Fortinet’s web application firewall (WAF) that provides protection against web-based threats and allows for the management of web traffic. Here’s a basic overview on how to configure routing and load balancing on FortiWeb:

Routing Configuration

Static routes direct traffic exiting the FortiWeb appliance based upon the packet’s destination—you can specify through which network interface a packet leaves and the IP address of a next-hop router that is reachable from that network interface.

Routers are aware of which IP addresses are reachable through various network pathways and can forward those packets along pathways capable of reaching the packets’ ultimate destinations. Your FortiWeb itself does not need to know the full route, as long as the routers can pass along the packet.

You must configure FortiWeb with at least one static route that points to a router, often a router that is the gateway to the Internet or intrinsic router of Azure if its public cloud hosted VM. You may need to configure multiple static routes if you have multiple gateway routers (e.g. each of which should receive packets destined for a different subset of IP addresses), redundant routers (e.g. redundant Internet/ISP links), or other special routing cases.

However, often you will only need to configure one route: a default route.

For example, if a web server is directly attached to one physical port on the FortiWeb, but all other destinations, such as connecting clients, are located in different VNETS in Azure, you might need to add a route to the destination vnet with the gateway IP of Port you would like to use. In this lab’s case it will be the Azure intrinsic gateway IP of Port1.

Loadbalancing:

FortiWeb supports multiple load balancing algortihms.

• Round Robin: Distributes new TCP connections to the next pool member, regardless of weight, response time, traffic load, or number of existing connections. FortiWeb avoids unresponsive servers.

  Suppose you have three servers in your pool: Server1, Server2, and Server3. New TCP connections are distributed in the following order: Server1, then Server2, then Server3, and it repeats in this cycle.

• Weighted Round Robin: Distributes new TCP connections using the round-robin method, except that members with a higher weight value receive a larger percentage of connections.

  If Server1 has a weight of 3, Server2 a weight of 1, and Server3 a weight of 2, the distribution of connections might look like: Server1, Server1, Server1, Server2, Server3, Server3, and then it repeats. Server1 gets more connections due to its higher weight.

• Least Connection: Distributes new TCP connections to the member with the fewest number of existing, fully-formed TCP connections. If there are multiple servers with the same least number of connections, FortiWeb will take turns and avoid always selecting the same member to distribute new connections.

  If Server1 has 10 connections, Server2 has 5 connections, and Server3 has 5 connections, the next new connection will be sent to either Server2 or Server3, whichever has fewer active connections.

• URI Hash: Distributes new TCP connections using a hash algorithm based on the URI found in the HTTP header, excluding hostname.

  A hash function is applied to the URI in the HTTP header (excluding the hostname). Suppose the URI “/api/data” consistently hashes to Server1, then all requests to “/api/data” will always be directed to Server1.

• Full URI Hash: Distributes new TCP connections using a hash algorithm based on the full URI string found in the HTTP header. The full URI string includes the hostname and path.

  Similar to URI Hash, but the hash function is applied to the entire URI, including the hostname. For instance, requests to “http://example.com/api/data" might hash to Server2.

• Host Hash: Distributes new TCP connections using a hash algorithm based on the hostname in the HTTP Request header Host field.

  This method applies a hash to the hostname found in the HTTP request’s Host field. If requests come with “host1.example.com”, and the hash points to Server3, all requests to “host1.example.com” will be directed to Server3.

• Host Domain Hash: Distributes new TCP connections using a hash algorithm based on the domain name in the HTTP Request header Host field.

  A hash function is applied to the domain name in the Host field. For example, requests to any subdomain of “example.com” might consistently hash to Server1 based on the domain name.

• Source IP Hash: Distributes new TCP connections using a hash algorithm based on the source IP address of the request.

  A hash function is applied to the source IP address of incoming requests. If the IP address 192.168.1.100 hashes to Server2, all requests from this IP will be directed to Server2.

• Least Response Time: Distributes incoming traffic to the back-end servers by multiplying average response time by the number of concurrent connections. Servers with the lowest value will get the traffic. In this way the client can connect to the most efficient back-end server.

  Suppose the average response times are 20 ms for Server1, 30 ms for Server2, and 15 ms for Server3, with all having similar connection numbers. The next connection will be directed to Server3 due to its lowest response time.

• Probabilistic Weighted Least Response Time: For the Least Response Time, in extreme cases there might be a server consistently has relatively low response time compared to others, which causes most of traffic to be distributed to one server. As a solution to this case, Probabilistic Weighted Least Response Time distributes traffic based on least response time as well as probabilities. The least response time server is most likely to receive traffic, while the rest servers still have chance to process some of the traffic.

  Using the previous example, if Server3 begins to receive a disproportionate amount of traffic, reducing its performance, this method might adjust probabilities such that Server1 and Server2 start to receive more connections to balance the load, even though Server3 still has the best average response time.

K8s ingress types

Kubernetes Ingress is a vital component for managing access to applications running within a Kubernetes cluster from outside the cluster. It provides routing rules to manage external users’ access to the services inside the cluster. Here’s a breakdown of different types of Kubernetes Ingress configurations:

1. Minimal Ingress

Minimal Ingress is the most straightforward type of Ingress. It’s used primarily when you have a single service that needs to be exposed externally. The configuration directs all incoming traffic on the specified host to a single backend service.

Detailed Example:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: minimal-ingress
spec:
  defaultBackend:
    service:
      name: single-service
      port:
        number: 80
EOF

Explanation:

• This Ingress directs all traffic that does not match any other rule to the single-service at port 80.
• Useful for simple applications or initial development stages when complex routing rules are not needed.

2. Simple fanout:

A fanout configuration routes traffic to multiple services based on the URL path. for example, Used when hosting multiple services or APIs from the same IP address, directing users to different services based on the path.

Detailed Example:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: simple-fanout
spec:
  rules:
  - host: fanout.example.com
    http:
      paths:
      - path: /blog
        pathType: Prefix
        backend:
          service:
            name: blog-service
            port:
              number: 80
      - path: /shop
        pathType: Prefix
        backend:
          service:
            name: shop-service
            port:
              number: 80
EOF

Explanation:

• Traffic to fanout.example.com/blog is directed to blog-service.
• Traffic to fanout.example.com/shop is directed to shop-service.
• Each service handles different parts of the application, allowing for modular and scalable design.

3. Ingress with Default backend

This Ingress configuration includes both specific rules and a default backend to handle unmatched requests. for example, To manage traffic to specific services while ensuring that all other requests are caught by a default backend.

Detailed Example:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: ingress-with-default-backend
spec:
  rules:
  - host: specific.example.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: specific-service
            port:
              number: 80
  defaultBackend:
    service:
      name: default-service
      port:
        number: 80
EOF

4. TLS/SSL Termination

This type of Ingress handles encrypted traffic, decrypting requests before passing them on to the appropriate services.

Detailed Example:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: tls-ingress
spec:
  tls:
  - hosts:
    - secure.example.com
    secretName: example-tls
  rules:
  - host: secure.example.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: secure-service
            port:
              number: 80
EOF

Explanation:

• Traffic to secure.example.com is handled with TLS, using the certificates stored in the Kubernetes secret example-tls.

5. ingress wildcard host

This configuration uses a wildcard host to match requests to any subdomain of a specified domain.

Typical Use Case: Useful for organizations that have different environments for their app (like development, staging, and production) under different subdomains.

Detailed Example:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: wildcard-host
spec:
  rules:
  - host: "*.example.com"
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: wildcard-service
            port:
              number: 80
EOF

Explanation: Requests to any subdomain of example.com are routed to wildcard-service

6. default backend

This Ingress configuration specifies a default backend. It is used when none of the rules in an Ingress resource match the incoming request.

Typical Use Case: To catch all unmatched requests, providing a generic response, perhaps an error message or a redirect to the homepage.

Detailed Example:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: default-backend
spec:
  defaultBackend:
    service:
      name: default-service
      port:
        number: 80
EOF

Explanation: All traffic that does not fit other specified rules is directed to default-service on port 80.

Security and WAF policies

Setting up basic security features and WAF policies

FortiWeb is a web application firewall (WAF) from Fortinet that offers comprehensive security features designed to protect web applications from various attacks and vulnerabilities. Here are some of the basic security features provided by FortiWeb:

1. Web Application Firewall (WAF) FortiWeb provides strong protection against common web attacks and vulnerabilities such as SQL injection, cross-site scripting (XSS), and cross-site request forgery (CSRF), among others. It uses both signature-based and behavior-based detection methods to identify threats.

2. DDoS Protection FortiWeb offers Distributed Denial of Service (DDoS) attack protection to help safeguard your applications from volumetric attacks that aim to overwhelm the system with traffic, rendering the application non-responsive.

3. OWASP Top 10 Protection FortiWeb is designed to protect against the vulnerabilities listed in the OWASP Top 10, which are the most critical security risks to web applications. This includes injection flaws, broken authentication, sensitive data exposure, and more.

4. Bot Mitigation FortiWeb includes features to detect and block malicious bots that can perform credential stuffing, scraping, and other automated attacks that can undermine the security and performance of your applications.

5. SSL/TLS Encryption FortiWeb supports SSL/TLS encryption to secure data in transit between clients and servers. It also provides SSL offloading to help improve the performance of your web servers by handling the encryption and decryption processes.

6. API Protection With increasing use of APIs in modern applications, FortiWeb provides specialized protections for API-based services, including JSON and XML protection. It ensures that APIs are not exploited to gain unauthorized access or to compromise data.

7. Advanced Threat Protection FortiWeb integrates with FortiSandbox to offer advanced threat protection, allowing it to analyze files and web content in a safe and isolated environment to detect new malware and zero-day exploits.

8. Machine Learning Capabilities FortiWeb uses machine learning to dynamically and automatically identify normal behavior and detect anomalies. This helps in protecting against sophisticated threats and reduces false positives without extensive manual intervention.

9. Rate Limiting To prevent abuse and ensure service availability, FortiWeb allows administrators to set rate limits on incoming requests, thus protecting against brute force attacks and other abusive behaviors by limiting how often a user can repeat actions.

10. IP Reputation and Geo-IP Based Filtering FortiWeb leverages IP reputation services and Geo-IP based filtering to block traffic from known malicious sources or specific geographic regions, adding an additional layer of security.

These features collectively provide robust protection for web applications, ensuring they remain secure from a wide range of cyber threats while maintaining performance and accessibility. FortiWeb’s comprehensive security measures make it an effective solution for businesses looking to safeguard their online operations.

Lets create a protection profile to use for the TLS based ingress application.

1. Login to FortiWeb:

• run the following commands on azure shell:

echo $fortiwebUsername
echo $fortiwebPassword
echo $vm_name

• copy the VM name and use that on a browser to go to: https://vm_name

example:

https://srijapx2.westus.cloudapp.azure.com

• Login with username and password from above output.

2. On FortiWeb > Policy > Web protection profile > click on “inline extended protection” and click clone at the top and give it a name: ingress tls profile. This profile name will be used later in the Ingress definition file.

3. Open the cloned protection profile to see there are few modules enabled already. We will update these protection profiles as we perform attacks in the next chapter.

4. Now lets create the TLS based ingress.

5. To create TLS based ingress we need to first generate certificate.

location="eastus"
fortiwebvmdnslabel="$(whoami)fortiwebvm7"
echo $fortiwebvmdnslabel
vm_name="$fortiwebvmdnslabel.$location.cloudapp.azure.com"
fortiwebvmdnslabelport2="$(whoami)px2.$location.cloudapp.azure.com"
echo $fortiwebvmdnslabelport2
openssl req -new -newkey rsa:4096 -days 365 -nodes -x509 \
    -subj "/C=US/ST=California/L=Sunnyvale/O=GlobalSecurity/OU=Dev/CN=$fortiwebvmdnslabelport2" \
    -keyout cert.key  -out cert.crt

You should see cert.crt, cert.key created.

6. Lets create a TLS secret to use the above cert and key.

kubectl create secret tls tls-secret --cert=cert.crt --key=cert.key

7. We will tell FortiWeb ingress controller use FortiWeb port1 ip for API access, and create VIP on FortiWeb Port1 secondary IP, the VIP address is on same subnet with Port1 with last octet set to .100.

use below script to get FortiWeb Port1 and Port1 Secondary IP address , then create yaml file with these IP address

echo vm_name=$vm_name
rsakeyname="id_rsa_tecworkshop"
ssh-keygen -f "${HOME}/.ssh/known_hosts" -R "${vm_name}" 
output=$(ssh -o "StrictHostKeyChecking=no" azureuser@$vm_name -i ~/.ssh/$rsakeyname 'get system interface')
echo $output
if [ "$fortiwebdeploymode" == "twoarms" ]; then
    portName="port2"
else
    portName="port1"
fi


portip=$(echo "$output" | grep -A 7 "== \[ $portName \]" | grep "ip:" | awk '{print $2}' | cut -d'/' -f1)
echo $portip

portip_first3=$(echo "$portip" | cut -d'.' -f1-3)

8. To generate tlsingress.yaml file run the below code:

cat << EOF | tee > 08_tls-ingress.yaml 
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: m
  annotations: {
    "fortiweb-ip" : $portip,    
    "fortiweb-login" : "fwb-login1",  
    "fortiweb-ctrl-log" : "enable",
    "virtual-server-ip" : $portip_first3.100, 
    "virtual-server-addr-type" : "ipv4",
    "virtual-server-interface" : $portName,
    "server-policy-web-protection-profile" : "ingress tls profile",
    "server-policy-https-service" : "HTTPS",
    "server-policy-http-service" : "HTTP",
    "server-policy-syn-cookie" : "enable",
    "server-policy-http-to-https" : "disable"
  }
spec:
  ingressClassName: fwb-ingress-controller
  tls:
  - hosts: 
     - $fortiwebvmdnslabelport2
    secretName: tls-secret
  rules:
  - host: $fortiwebvmdnslabelport2
    http:
      paths:
      - path: /info
        pathType: Prefix
        backend:
          service:
            name: service1
            port:
              number: 1241
EOF

This is will generate 08_tls-ingress.yaml file

9. Now lets run kubectl apply -f 08_tls-ingress.yaml

output:

ingress.networking.k8s.io/m created
EOF

10. on FortiWeb, there will be now a Server policy with Certificate, Web protection profile we created in Step1.

11. to verify the certificate, please run the belwo ommand in Azure shell.

echo $fortiwebvmdnslabelport2

output:

srijapx2.eastus.cloudapp.azure.com/info

12. In the browser: https://FQDN/info

example: https://srijapx2.westus.cloudapp.azure.com/info

we should see the certifcate CN name match your DNS/FQDN.

Attack

Now lets create some attacks and defenses

cat << EOF | tee > juiceshopdeployment.yaml 
apiVersion: apps/v1
kind: Deployment
metadata:
  name: juiceshop-app
spec:
  selector: 
    matchLabels:
     app: juiceshop
  template:
    metadata:
      labels:
        app: juiceshop
    spec:
      containers:
      - name: juiceshop
        image: "bkimminich/juice-shop:snapshot"
EOF
kubectl apply -f juiceshopdeployment.yaml

cat << EOF | tee > juiceshopservice.yaml 
apiVersion: v1
kind: Service
metadata:
  name: juiceshop
  labels:
    app: juiceshop
spec:
  type: NodePort
  selector:
    app: juiceshop
  ports:
  - port: 80
    targetPort: 3000
    protocol: TCP
EOF
kubectl apply -f juiceshopservice.yaml

echo vm_name=$vm_name
rsakeyname="id_rsa_tecworkshop"
ssh-keygen -f "${HOME}/.ssh/known_hosts" -R "${vm_name}" 
output=$(ssh -o "StrictHostKeyChecking=no" azureuser@$vm_name -i ~/.ssh/$rsakeyname 'get system interface')
echo $output


if [ "$fortiwebdeploymode" == "twoarms" ]; then
    portName="port2" 
else
    portName="port1"
fi 
 
portip=$(echo "$output" | grep -A 7 "== \[ $portName \]" | grep "ip:" | awk '{print $2}' | cut -d'/' -f1)
echo $portip

portip_first3=$(echo "$portip" | cut -d'.' -f1-3)

cat << EOF | tee > 08_tls-ingress.yaml 
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: m
  annotations: {
    "fortiweb-ip" : $portip,    
    "fortiweb-login" : "fwb-login1",  
    "fortiweb-ctrl-log" : "enable",
    "virtual-server-ip" : $portip_first3.100, 
    "virtual-server-addr-type" : "ipv4",
    "virtual-server-interface" : $portName,
    "server-policy-web-protection-profile" : "ingress tls profile",
    "server-policy-https-service" : "HTTPS",
    "server-policy-http-service" : "HTTP",
    "server-policy-syn-cookie" : "enable",
    "server-policy-http-to-https" : "disable"
  }
spec:
  ingressClassName: fwb-ingress-controller
  tls:
  - hosts: 
     - $fortiwebvmdnslabelport2
    secretName: tls-secret
  rules:
  - host: $fortiwebvmdnslabelport2
    http:
      paths:
      - path: /info
        pathType: Prefix
        backend:
          service:
            name: service1
            port:
              number: 1241
      - path: /
        pathType: Prefix
        backend:
          service:
            name: juiceshop
            port:
              number: 80
 
EOF

ingress.networking.k8s.io/m configured.

srija [ ~/k8s-202-workshop/scripts/labEnv ]$ kubectl get ingress
NAME   CLASS                    HOSTS                                ADDRESS    PORTS     AGE
m      fwb-ingress-controller   srijapx2.westus.cloudapp.azure.com   10.0.1.5   80, 443   158m

Name:             m
Labels:           <none>
Namespace:        default
Address:          10.0.1.5
Ingress Class:    fwb-ingress-controller
Default backend:  <default>
TLS:
  hello-app-tls terminates srijapx2.westus.cloudapp.azure.com
Rules:
  Host                                Path  Backends
  ----                                ----  --------
  srijapx2.westus.cloudapp.azure.com  
                                      /       juiceshop:80 (10.224.0.8:3000)
                                      /info   service1:1241 (10.224.0.14:9876)
Annotations:                          FortiWeb-ctrl-log: disable
                                      FortiWeb-ip: 10.0.1.4
                                      FortiWeb-login: fwb-login1
                                      server-policy-http-service: HTTP
                                      server-policy-http-to-https: disable
                                      server-policy-https-service: HTTPS
                                      server-policy-syn-cookie: enable
                                      server-policy-web-protection-profile: Inline Standard Protection
                                      virtual-server-addr-type: ipv4
                                      virtual-server-interface: port1
                                      virtual-server-ip: 10.0.1.5
Events:                               <none>

Sql injection

The first attack will be a SQLi attack. Please use Chrome browser to best follow the insrtuctions.

FortiWeb ingress TLS protection profile already has Extended protection enabled for Signature based protection, so we can just continue to do an attack.

• Proceed with the Attack!

https://srijapx2.westus.cloudapp.azure.com/rest/products/search?q='''''''''''''UNION SELECT '2

"https://srijapx2.westus.cloudapp.azure.com/rest/products/search?q=<script>$('input').value="';DELETE FROM USERS;--";$('form').submit()</script>"

Q & A

1. What attack was triggered in step 6?
   Click for Answer…

   SQLi
   

2. How are signatures helping to block these OWASP TOP 10 attacks?
   Click for Answer…

   Signatures detect patterns of malicious SQL statements, command injection attempts, and other injection payloads.
   

   Example: Detecting ‘UNION SELECT’ or ’exec xp_’ in requests.

URL rewriting

Lets also explore URL rewriting with FortiWeb. FortiWeb supports several URL rewriting capabilities which can be very important and useful in production applications.

Key uses of URL rewriting on FortiWeb:

• SEO Optimization: Improves search engine rankings by transforming dynamic URLs into static, keyword-rich URLs that are easier for search engines to index.

• User-Friendly URLs: Creates readable and memorable URLs, enhancing the user experience and making it easier for users to navigate the website.

• Hiding Internal URL Structures: Conceals the internal structure and parameters of web applications, adding an extra layer of security and preventing exposure of sensitive details.

• Security Enhancement: Makes it harder for attackers to guess the structure of the web application, reducing the risk of attacks such as parameter tampering or direct access to sensitive endpoints.

• Redirection Management: Manages redirects efficiently, ensuring that users and search engines are directed to the correct pages even after the website structure changes or pages are moved.

• Consistent URL Structure: Ensures uniformity in URL formatting across the website, which aids in site maintenance and improves the overall user experience.

Attack1 - Rewriting Policy:

Lets create a rewriting policy to rewrite from Service1 to Juiceshop application.

Attack 1 - DOS protection/Rate limiting:

export fortiwebvmdnslabelport2=$fortiwebvmdnslabelport2
export location=$location
url="https://${fortiwebvmdnslabelport2}"
echo "Sending requests to: $url"

# Loop to run 100 GET requests
for i in {1..100}
do
    # Make the GET request and capture the output
    output=$(curl -s -k -w "\nHTTP Status: %{http_code}" "$url")
    
    # Check if curl command was successful
    if [ $? -eq 0 ]; then
        # Print the output
        echo "Request $i output:"
        echo "$output"
        echo "------------------------"
    else
        echo "Request $i failed"
    fi
    
    # Add a small delay between requests
    sleep 0.1
done

echo "All 100 requests completed."

Attack and traffic log

Attack logs and traffic logs are critical components of a comprehensive cybersecurity strategy. Here’s how they can be beneficial:

Attack Logs

• Incident Detection and Response: Attack logs provide detailed information about potential security incidents. By analyzing these logs, security teams can identify attack vectors, understand the methods used by attackers, and respond more effectively to mitigate threats.

• Forensic Analysis: In the event of a security breach, attack logs are invaluable for forensic purposes. They help trace the steps taken by the attacker, determine the scope of the breach, and identify compromised data.

• Compliance and Reporting: Many regulatory requirements mandate that organizations must monitor and log access to sensitive data. Attack logs help in demonstrating compliance with these regulations by providing an audit trail of security events.

• Trend Analysis and Threat Intelligence: Over time, attack logs can reveal patterns and trends in security threats. This information can be used to enhance threat intelligence, improve security postures, and proactively defend against future attacks.

Attack log in enabled by default on FortiWeb. To enable or diable attack log:

1. Navigate to Log Settings: Look for the “Log & Report” menu, then find the “log config” > “Other log settings”

2. To view attacks logs: In the “Log & Report” menu > Log Access > Attack

• Traffic Logs Network Monitoring and Management: Traffic logs record data about the flow of traffic through the network. This data is essential for network performance monitoring, troubleshooting network issues, and ensuring network resources are optimally utilized.

• Security Analysis: By examining traffic logs, organizations can detect anomalies in network traffic that may indicate malicious activity, such as unusual data flows or spikes in traffic to sensitive resources.

• Policy Enforcement: Traffic logs help verify that network usage policies and security controls are being followed. For instance, logs can show whether any prohibited services or applications are being accessed.

• Bandwidth Management: Traffic logs provide insights into bandwidth usage patterns. This information can be used to optimize bandwidth allocation, improve user experience, and reduce costs by identifying unnecessary or excessive usage.

• User and Entity Behavior Analytics (UEBA): By analyzing traffic logs, organizations can build profiles of normal user and system behavior. Deviations from these profiles may indicate potential security or operational issues.

Overall, both attack logs and traffic logs serve as essential tools for maintaining security, ensuring compliance, and optimizing network performance. They provide the visibility needed to manage, secure, and audit information systems effectively.

• To enable traffic log this has be done at policy level from the CLI.

1. Click on CLI of FortiWeb.

2. run the below commands in the CLI:

   config server-policy policy
   edit <policy to enable traffic log>
   set tlog enable
   next
   end

for example:

config server-policy policy
edit default_m
set tlog enable
next 
end

3. To view Traffic logs: In the “Log & Report” menu > Log Access > Traffic.

Troubleshooting

Diagnosing Network Connectivity Issues

One of your first tests when configuring a new policy should be to determine whether allowed traffic is flowing to your web servers.

1. Is there a server policy applied to the web server or servers FortiWeb was installed to protect? If it is operating in Reverse Proxy mode, FortiWeb will not allow any traffic to reach a protected web server unless there is a matching server policy that permits it.

2. If your network utilizes secure connections (HTTPS) and there is no traffic flow, is there a problem with your certificate?

3. If you run a test attack from a browser aimed at your website, does it show up in the attack log?

To verify, configure FortiWeb to detect the attack, then craft a proof-of-concept that will trigger the attack sensor.

For example, to see whether directory traversal attacks are being logged and/or blocked, you could use your web browser to go to: http://www.example.com/login?user=../../../../ Under normal circumstances, you should see a new attack log entry in the attack log console widget of the system dashboard.

4. use TCPDUMP equivalent command on FortiWeb diagnose network sniffer 4 0 -a

example: diagnose network sniffer any “port 8443” 4 0 -a

To ping a device from the FortiWeb CLI:

1. Log in to the CLI via either SSH, Telnet, or you can ping from the FortiWeb appliance in the CLI Console accessed from the web UI.

2. If you want to adjust the behavior of execute ping, first use the execute ping options command.

3. Enter the command:

execute ping <destination_ipv4> where <destination_ipv4> is the IP address of the device that you want to verify that the appliance can connect to, such as 192.168.1.5.

If the appliance can reach the host via ICMP, output similar to the following appears:

PING 192.0.2.96 (192.0.2.96): 56 data bytes
64 bytes from 192.0.2.96: icmp_seq=0 ttl=253 time=6.5 ms
64 bytes from 192.0.2.96: icmp_seq=1 ttl=253 time=7.4 ms
64 bytes from 192.0.2.96: icmp_seq=2 ttl=253 time=6.0 ms
64 bytes from 192.0.2.96: icmp_seq=3 ttl=253 time=5.5 ms
64 bytes from 192.0.2.96: icmp_seq=4 ttl=253 time=7.3 ms
--- 192.0.2.96 ping statistics ---
5 packets transmitted, 5 packets received, 0% packet loss
round-trip min/avg/max = 5.5/6.5/7.4 ms
EOF

If the appliance cannot reach the host via ICMP, output similar to the following appears:

PING 192.0.2.108 (192.0.2.108): 56 data bytes
Timeout ...
Timeout ...
Timeout ...
Timeout ...
Timeout ...
--- 192.0.2.108 ping statistics ---
5 packets transmitted, 0 packets received, 100% packet loss
EOF

“100% packet loss” and “Timeout” indicates that the host is not reachable.

For kubernetes ingress controller:

1. From the cluster check the ingress controller pod logs in namepsace to see if there are any errors.

kubectl get pods -n FortiWebingress

output:

NAME                                          READY   STATUS    RESTARTS   AGE
first-release-fwb-k8s-ctrl-59db65cddc-g4298   1/1     Running   0          2d15h
EOF

2. Kubectl logs first-release-fwb-k8s-ctrl-59db65cddc-g4298 -n FortiWebingress (Replace with your pod name and namespace)

output:

Do POST url https://10.0.1.4:443/api/v2.0/cmdb/server-policy/vserver/vip-list?mkey=default_m
default/m: Response status code: 500 URL https://10.0.1.4:443/api/v2.0/cmdb/server-policy/vserver/vip-list?mkey=default_m Resp {
        "results": {
                "errcode": -5,
                "message": "A duplicate entry has already existed."
        }
}
REQ BODY '{"data":{"status":"enable","use-interface-ip":"disable","vip":"default_m"}}'Do POST url https://10.0.1.4:443/api/v2.0/cmdb/server-policy/policy
default/m: Response status code: 500 URL https://10.0.1.4:443/api/v2.0/cmdb/server-policy/policy Resp {
        "results": {
                "errcode": -5,
                "message": "A duplicate entry has already existed."
        }
}
REQ BODY '{"data":{"case-sensitive":"disable","certificate":"","certificate-type":"disable","client-certificate-forwarding":"disable","client-certificate-forwarding-cert-header":"X-Client-Cert","
.
.
.
EOF

Finetuning

Optimizing the performance of a FortiWeb Web Application Firewall (WAF) is crucial for maintaining the security of your web applications without sacrificing user experience due to latency or downtime. Here are some best practices specifically tailored for tuning and optimizing FortiWeb:

1. Understand Traffic Patterns Analyze Traffic: Regularly analyze your web traffic patterns to understand peak usage times, common requests, and potential security threats. This will help you configure FortiWeb more effectively. Monitor Performance: Utilize FortiWeb’s built-in monitoring tools to keep an eye on the system’s health, resource usage, and performance metrics.

2. Optimize Security Policies Tailor Rules and Policies: Customize the security rules and policies to fit the specific needs and threats of your applications. Avoid using default policies which may not be optimized for your particular environment. Prioritize Rules: Arrange firewall rules by frequency and priority. Place the most commonly matched rules at the top to minimize the processing time.

3. Use Caching and Compression Enable Caching: Use FortiWeb’s caching capabilities to store static content, reducing the load on your servers and speeding up response times for end-users. Implement Compression: Enable HTTP compression to reduce the size of the data transmitted over the network, improving load times and reducing bandwidth usage.

4. SSL/TLS Optimization Offload SSL: Utilize SSL offloading to handle the SSL/TLS decryption process, relieving your backend servers from this computationally intensive task and improving overall response times. Manage Certificates: Keep your SSL/TLS certificates up to date and use strong cipher suites to ensure optimal security and performance.

5. Configure Connection Settings Connection Pooling: Use connection pooling to manage and reuse connections effectively, reducing the overhead associated with establishing new connections. Timeout Settings: Adjust timeout settings appropriately to avoid unnecessary resource utilization, ensuring that idle or stuck sessions are cleared promptly.

6. Scalability and High Availability Load Balancing: Deploy FortiWeb in a load-balanced environment to distribute traffic evenly across multiple instances, enhancing both performance and availability. High Availability Setup: Configure high availability (HA) for FortiWeb to ensure there is no single point of failure, improving the resilience of your web security infrastructure.

7. Regular Updates and Maintenance Firmware Updates: Regularly update the FortiWeb firmware to the latest version to benefit from performance improvements, new features, and security patches. Configuration Audits: Periodically review and audit the configuration settings to remove any obsolete or unnecessary rules that could degrade performance.

8. Advanced Features and Fine-Tuning Threat Intelligence Integration: Integrate with FortiGuard services for updated threat intelligence, which helps in proactive defense mechanisms by adjusting policies based on the latest threat landscape.

By following these best practices, you can ensure that your FortiWeb appliance is not only securing your web applications effectively but also operating at optimal efficiency and providing a seamless experience for your users.