Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add option for forced tunneling through TRE's Firewall #4238

Draft
wants to merge 5 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,7 @@ BUG FIXES:
* Fix VM actions where Workspace shared storage doesn't allow shared key access ([#4222](https://github.com/microsoft/AzureTRE/issues/4222))
* Fix public exposure in Guacamole service ([[#4199](https://github.com/microsoft/AzureTRE/issues/4199)])
* Fix Azure ML network tags to use name rather than ID ([[#4151](https://github.com/microsoft/AzureTRE/issues/4151)])
* Add option to force tunnel TRE's Firewall ([#4237](https://github.com/microsoft/AzureTRE/issues/4237))

COMPONENTS:

Expand Down
20 changes: 20 additions & 0 deletions docs/tre-admins/configure-firewall-force-tunneling.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
# Forced Tunneling to External Firewall in TRE

Forced tunneling ensures that all traffic from TRE is routed through a specific external firewall. This guarantees that all data passes through the firewall for inspection, control, or further processing before reaching its destination.

To setup forced tunneling to an external firewall, follow these steps:

## 1. Set the rp_bundle_values Parameter in the config.yaml file
Provide the external firewall's IP address:

```json
rp_bundle_values: '{"firewall_force_tunnel_ip":"10.0.0.4"}'
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a reason we are using rp_bundle_values rather than exposing it as a parameter and giving it its own variable in a similar way to firewal_sku?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can make that change, but from what I understand, rp_bundle_values are intended for passing parameters to the resource processor, which are used only by the bundles and not the processor itself. Should I change?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If configured as a parameter for the bundle, the IP can be passed as part of the deployment message.

Hmm, I can see firewall_sku was set on the RP as an env var. @tamirkamara think you worked on that, why was it done this way, rather than passed as a property in make firewall-install?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that was me :-) It may not have been the best way to pass it!

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, my view is that we should be passing both firewall sku and the tunnel IP as props, as part of firewall-install here:

AzureTRE/Makefile

Lines 304 to 313 in c79d9e1

deploy-shared-service:
$(call target_title, "Deploying ${DIR} shared service") \
&& . ${MAKEFILE_DIR}/devops/scripts/check_dependencies.sh porter,env \
&& ${MAKEFILE_DIR}/devops/scripts/ensure_cli_signed_in.sh $${TRE_URL} \
&& cd ${DIR} \
&& ${MAKEFILE_DIR}/devops/scripts/deploy_shared_service.sh $${PROPS}
firewall-install:
$(MAKE) bundle-build bundle-publish bundle-register deploy-shared-service \
DIR=${MAKEFILE_DIR}/templates/shared_services/firewall/ BUNDLE_TYPE=shared_service

Not sure how easy that is to do... The rp_bundle_values will work, but believe it was intended for setting something that should not be passed via the API.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How did you test it? I can't see them.

image

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I meant that if you set these values and run make tre-deploy from your local machine the RP will get the values and will be able to use them (same for the CI).
As for debug I don't think we tried. Are you sure you followed all steps to make these available after updating your config file? At any case, it works the 2 ways it counts :-)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They aren't output to private.env so it's not going to work unless that's updated somehow. I should have been searching for os.environ["RP_BUNDLE_custom_key_1"] , but it's still not there. Saying that, it's the same for Firewall SKU.

IMHO they should be properties on the bundle so they can be configured via the TRE API/UI, are visible via the UI, without destroying and recreating the resource processor.

If you want to get it in as is without the changes suggested, then do it and I can amend it next week with a further PR.

Copy link
Collaborator

@tamirkamara tamirkamara Jan 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But they are params in the bundle so maybe we can add them to the template schema to allow setting via the API. But TBH, passing it through the scripts isn't great and I don't think this is the right time to get into it.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are the one who added the code to allow the passing props into the shared services script ;-), all I showed above is where they need to be specified.

Lets start by adding them to the template schema.

```
This automatically creates a route table to direct TRE’s traffic to the specified IP.

## 2. Manually Connect TRE to Your Firewall
Configure connectivity between TRE’s VNet and your external firewall using one of the following methods:

1. **VNet Peering**: Peer the TRE VNet with your firewall’s VNet.
1. **ExpressRoute**: Use a private connection for firewalls located on-premises.
1. **Site-to-Site VPN**: Establish a VPN connection as an alternative.
1 change: 1 addition & 0 deletions mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -140,6 +140,7 @@ nav:
- Supported Clouds: tre-admins/supported-clouds.md
- Customer Managed Keys: tre-admins/customer-managed-keys.md
- Custom Domain Name: tre-admins/custom-domain.md
- Firewall Force Tunneling: tre-admins/configure-firewall-force-tunneling.md

- Development: # Docs related to the developing code for the AzureTRE
- Local Development: using-tre/local-development/local-development.md
Expand Down
8 changes: 7 additions & 1 deletion templates/shared_services/firewall/porter.yaml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
schemaVersion: 1.0.0
name: tre-shared-service-firewall
version: 1.2.8
version: 1.3.0
description: "An Azure TRE Firewall shared service"
dockerfile: Dockerfile.tmpl
registry: azuretre
Expand Down Expand Up @@ -54,6 +54,9 @@ parameters:
default: "graph.microsoft.com"
- name: arm_environment
type: string
- name: firewall_force_tunnel_ip
type: string
default: ""

mixins:
- terraform:
Expand All @@ -69,6 +72,7 @@ install:
api_driven_network_rule_collections_b64: ${ bundle.parameters.network_rule_collections }
firewall_sku: ${ bundle.parameters.firewall_sku }
microsoft_graph_fqdn: ${ bundle.parameters.microsoft_graph_fqdn }
firewall_force_tunnel_ip: ${ bundle.parameters.firewall_force_tunnel_ip }
backendConfig:
use_azuread_auth: "true"
use_oidc: "true"
Expand All @@ -87,6 +91,7 @@ upgrade:
api_driven_network_rule_collections_b64: ${ bundle.parameters.network_rule_collections }
firewall_sku: ${ bundle.parameters.firewall_sku }
microsoft_graph_fqdn: ${ bundle.parameters.microsoft_graph_fqdn }
firewall_force_tunnel_ip: ${ bundle.parameters.firewall_force_tunnel_ip }
backendConfig:
use_azuread_auth: "true"
use_oidc: "true"
Expand All @@ -105,6 +110,7 @@ uninstall:
api_driven_network_rule_collections_b64: ${ bundle.parameters.network_rule_collections }
firewall_sku: ${ bundle.parameters.firewall_sku }
microsoft_graph_fqdn: ${ bundle.parameters.microsoft_graph_fqdn }
firewall_force_tunnel_ip: ${ bundle.parameters.firewall_force_tunnel_ip }
backendConfig:
use_azuread_auth: "true"
use_oidc: "true"
Expand Down
4 changes: 2 additions & 2 deletions templates/shared_services/firewall/terraform/firewall.tf
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ moved {
}

resource "azurerm_public_ip" "fwmanagement" {
count = local.effective_firewall_sku == "Basic" ? 1 : 0
count = (var.firewall_force_tunnel_ip != "" || local.effective_firewall_sku == "Basic") ? 1 : 0
name = "pip-fw-management-${var.tre_id}"
resource_group_name = local.core_resource_group_name
location = data.azurerm_resource_group.rg.location
Expand All @@ -42,7 +42,7 @@ resource "azurerm_firewall" "fw" {
}

dynamic "management_ip_configuration" {
for_each = local.effective_firewall_sku == "Basic" ? [1] : []
for_each = (var.firewall_force_tunnel_ip != "" || local.effective_firewall_sku == "Basic") ? [1] : []
content {
name = "mgmtconfig"
subnet_id = data.azurerm_subnet.firewall_management.id
Expand Down
25 changes: 25 additions & 0 deletions templates/shared_services/firewall/terraform/routetable.tf
Original file line number Diff line number Diff line change
Expand Up @@ -87,3 +87,28 @@ resource "azurerm_subnet_route_table_association" "rt_airlock_events_subnet_asso
azurerm_firewall_policy_rule_collection_group.dynamic_application
]
}

resource "azurerm_route_table" "fw_tunnel_rt" {
count = var.firewall_force_tunnel_ip != "" ? 1 : 0
name = "rt-fw-tunnel-${var.tre_id}"
resource_group_name = local.core_resource_group_name
location = data.azurerm_resource_group.rg.location
bgp_route_propagation_enabled = true
tags = local.tre_shared_service_tags

lifecycle { ignore_changes = [tags] }

route {
name = "ForceTunnelRoute"
address_prefix = "0.0.0.0/0"
next_hop_type = "VirtualAppliance"
next_hop_in_ip_address = var.firewall_force_tunnel_ip
}
}

resource "azurerm_subnet_route_table_association" "rt_fw_tunnel_subnet_association" {
count = var.firewall_force_tunnel_ip != "" ? 1 : 0
subnet_id = data.azurerm_subnet.firewall.id
route_table_id = azurerm_route_table.fw_tunnel_rt[0].id
}

5 changes: 5 additions & 0 deletions templates/shared_services/firewall/terraform/variables.tf
Original file line number Diff line number Diff line change
Expand Up @@ -27,3 +27,8 @@ variable "firewall_sku" {
type = string
default = ""
}

variable "firewall_force_tunnel_ip" {
type = string
default = ""
}
Loading