GKE Module

Google Kubernetes Engine (GKE) is the most scalable and fully automated Kubernetes service. This module handles opinionated GKE cluster creation and configurations based on the Terraform module of GKE developed by Google.

This module deploy:

• Public/Private GKE standard with beta functionalities.

• Public/Private GCP Autopilot with beta functionalities.

• By default, the GCP Kubernetes cluster is autopilot and private. Otherwise, you set: private = false and/or var.autopilot = false.

Requirements

Name	Version
terraform	>=1.0
google	>= 4.75.0
null	>= 3.2.1

Providers

Name	Version
google	>= 4.75.0
null	>= 3.2.1

Modules

Name	Source	Version
autopilot	terraform-google-modules/kubernetes-engine/google//modules/beta-autopilot-public-cluster	36.1.0
gke	terraform-google-modules/kubernetes-engine/google//modules/beta-public-cluster	36.1.0
private_autopilot	terraform-google-modules/kubernetes-engine/google//modules/beta-autopilot-private-cluster	36.1.0
private_gke	terraform-google-modules/kubernetes-engine/google//modules/beta-private-cluster	36.1.0

Resources

Name	Type
google_kms_crypto_key_iam_member.kms	resource
null_resource.update_kubeconfig	resource
google_client_config.current	data source
google_compute_zones.available	data source
google_project.project	data source

Inputs

Name	Description	Type	Default	Required
add_cluster_firewall_rules	Create additional firewall rules.	bool	false	no
add_master_webhook_firewall_rules	Create master_webhook firewall rules for ports defined in firewall_inbound_ports.	bool	false	no
add_shadow_firewall_rules	Create GKE shadow firewall (the same as default firewall rules with firewall logs enabled).	bool	false	no
authenticator_security_group	The name of the RBAC security group for use with Google security groups in Kubernetes RBAC. Group name must be in format gke-security-groups@yourdomain.com.	string	null	no
autopilot	Create autopilot GKE cluster.	bool	true	no
cloudrun	(Beta) Enable CloudRun addon.	bool	false	no
cloudrun_load_balancer_type	(Beta) Configure the Cloud Run load balancer type. External by default. Set to LOAD_BALANCER_TYPE_INTERNAL to configure as an internal load balancer.	string	""	no
cluster_autoscaling	Cluster autoscaling configuration. See more details. For disk_tye see Persistent Disk types.	object({ enabled = bool autoscaling_profile = string auto_repair = bool auto_upgrade = bool disk_size = number disk_type = string gpu_resources = list(object({ resource_type = string minimum = number maximum = number })) min_cpu_cores = number max_cpu_cores = number min_memory_gb = number max_memory_gb = number })	{ “auto_repair”: true, “auto_upgrade”: true, “autoscaling_profile”: “BALANCED”, “disk_size”: 100, “disk_type”: “pd-standard”, “enabled”: false, “gpu_resources”: [], “max_cpu_cores”: 0, “max_memory_gb”: 0, “min_cpu_cores”: 0, “min_memory_gb”: 0 }	no
cluster_dns_domain	The suffix used for all cluster service records.	string	""	no
cluster_dns_provider	Which in-cluster DNS provider should be used. PROVIDER_UNSPECIFIED (default) or PLATFORM_DEFAULT or CLOUD_DNS.	string	"PROVIDER_UNSPECIFIED"	no
cluster_dns_scope	The scope of access to cluster DNS records. DNS_SCOPE_UNSPECIFIED (default) or CLUSTER_SCOPE or VPC_SCOPE.	string	"DNS_SCOPE_UNSPECIFIED"	no
cluster_ipv4_cidr	The IP address range of the Kubernetes pods in this cluster. Default is an automatically assigned CIDR.	string	null	no
cluster_resource_labels	The GCE resource labels (a map of key/value pairs) to be applied to the cluster.	map(string)	{}	no
cluster_telemetry_type	Available options include ENABLED, DISABLED, and SYSTEM_ONLY.	string	null	no
config_connector	(Beta) Whether ConfigConnector is enabled for this cluster.	bool	false	no
configure_ip_masq	Enables the installation of ip masquerading, which is usually no longer required when using aliasied IP addresses. IP masquerading uses a kubectl call, so when you have a private cluster, you will need access to the API server.	bool	false	no
database_encryption	Application-layer Secrets Encryption settings. Valid values of state are: “ENCRYPTED”; “DECRYPTED”.	list(object({ state = string key_name = string }))	[ { “key_name”: “”, “state”: “DECRYPTED” } ]	no
datapath_provider	The desired datapath provider for this cluster. By default, DATAPATH_PROVIDER_UNSPECIFIED enables the IPTables-based kube-proxy implementation. ADVANCED_DATAPATH enables Dataplane-V2 feature.	string	"DATAPATH_PROVIDER_UNSPECIFIED"	no
default_max_pods_per_node	The maximum number of pods to schedule per node. Note: For GKE versions earlier than 1.23.5-gke.1300, the limit is 110 Pods, otherwise the limit is 256 Pods.	number	110	no
deploy_using_private_endpoint	(Beta) A toggle for Terraform and kubectl to connect to the master’s internal IP address during deployment. Used when private set to true.	bool	true	no
description	The description of the GKE cluster.	string	""	no
disable_default_snat	Whether to disable the default SNAT to support the private use of public IP addresses	bool	false	no
disable_legacy_metadata_endpoints	Disable the /0.1/ and /v1beta1/ metadata server endpoints on the node. Changing this value will cause all node pools to be recreated.	bool	true	no
dns_cache	The status of the NodeLocal DNSCache addon.	bool	null	no
enable_binary_authorization	Enable BinAuthZ Admission controller.	bool	false	no
enable_confidential_nodes	An optional flag to enable confidential node config.	bool	false	no
enable_cost_allocation	Enables Cost Allocation Feature and the cluster name and namespace of your GKE workloads appear in the labels field of the billing export to BigQuery.	bool	true	no
enable_identity_service	Enable the Identity Service component, which allows customers to use external identity providers with the K8S API.	bool	false	no
enable_intranode_visibility	Whether Intra-node visibility is enabled for this cluster. This makes same node pod to pod traffic visible for VPC network.	bool	false	no
enable_kubernetes_alpha	Whether to enable Kubernetes Alpha features for this cluster. Note that when this option is enabled, the cluster cannot be upgraded and will be automatically deleted after 30 days.	bool	false	no
enable_l4_ilb_subsetting	Enable L4 ILB Subsetting on the cluster. Used when beta set to true.	bool	false	no
enable_network_egress_export	Whether to enable network egress metering for this cluster. If enabled, a daemonset will be created in the cluster to meter network egress traffic.	bool	false	no
enable_pod_security_policy	enabled - Enable the PodSecurityPolicy controller for this cluster. If enabled, pods must be valid under a PodSecurityPolicy to be created. Pod Security Policy was removed from GKE clusters with version >= 1.25.0.Used when beta set to true.	bool	false	no
enable_resource_consumption_export	Whether to enable resource consumption metering on this cluster. When enabled, a table will be created in the resource export BigQuery dataset to store resource consumption data. The resulting table can be joined with the resource usage table or with BigQuery billing export.	bool	true	no
enable_shielded_nodes	Enable Shielded Nodes features on all nodes in this cluster.	bool	true	no
enable_tpu	Enable Cloud TPU resources in the cluster. WARNING: changing this after cluster creation is destructive! Used when beta set to true.	bool	false	no
enable_vertical_pod_autoscaling	Vertical Pod Autoscaling automatically adjusts the resources of pods controlled by it.	bool	false	no
filestore_csi_driver	The status of the Filestore CSI driver addon, which allows the usage of filestore instance as volumes.	bool	false	no
firewall_inbound_ports	List of TCP ports for admission/webhook controllers. Either flag add_master_webhook_firewall_rules or add_cluster_firewall_rules (also adds egress rules) must be set to true for inbound-ports firewall rules to be applied.	list(string)	[ “8443”, “9443”, “15017” ]	no
firewall_priority	Priority rule for firewall rules.	number	1000	no
gateway_api_channel	The gateway api channel of this cluster. Accepted values are CHANNEL_STANDARD and CHANNEL_DISABLED.	string	null	no
gce_pd_csi_driver	Whether this cluster should enable the Google Compute Engine Persistent Disk Container Storage Interface (CSI) Driver.	bool	false	no
gke_backup_agent_config	Whether Backup for GKE agent is enabled for this cluster.	bool	false	no
grant_registry_access	Grants created cluster-specific service account storage.objectViewer and artifactregistry.reader roles.	bool	true	no
horizontal_pod_autoscaling	Enable horizontal pod autoscaling addon.	bool	false	no
http_load_balancing	Enable httpload balancer addon.	bool	true	no
identity_namespace	The workload pool to attach all Kubernetes service accounts to. (Default value of enabled automatically sets project-based pool [project_id].svc.id.goog)	string	"enabled"	no
initial_node_count	The number of nodes to create in this cluster’s default node pool.	number	0	no
ip_masq_link_local	Whether to masquerade traffic to the link-local prefix (169.254.0.0/16).	bool	false	no
ip_masq_resync_interval	The interval at which the agent attempts to sync its ConfigMap file from the disk.	string	"60s"	no
ip_range_pods	The name of the secondary subnet ip range to use for Kubernetes pods.	string	n/a	yes
ip_range_services	The name of the secondary subnet range to use for Kubernetes services.	string	n/a	yes
issue_client_certificate	Issues a client certificate to authenticate to the cluster endpoint. To maximize the security of your cluster, leave this option disabled. Client certificates don’t automatically rotate and aren’t easily revocable. WARNING: changing this after cluster creation is destructive!	bool	false	no
istio	(Beta) Enable Istio addon.	bool	false	no
istio_auth	(Beta) The authentication type between services in Istio.	string	"AUTH_MUTUAL_TLS"	no
kalm_config	(Beta) Whether KALM is enabled for this cluster.	bool	false	no
kubeconfig_path	Path to save the kubeconfig file.	string	null	no
kubernetes_version	The Kubernetes version of the masters. If set to ‘latest’ it will pull latest available version in the selected region.	string	"latest"	no
logging_enabled_components	List of services to monitor: SYSTEM_COMPONENTS, WORKLOADS. Empty list is default GKE configuration.	list(string)	[]	no
logging_service	The logging service that the cluster should write logs to. Available options include logging.googleapis.com, logging.googleapis.com/kubernetes (beta), and none	string	"logging.googleapis.com/kubernetes"	no
maintenance_end_time	Time window specified for recurring maintenance operations in RFC3339 format.	string	""	no
maintenance_exclusions	List of maintenance exclusions. A cluster can have up to three	list(object({ name = string start_time = string end_time = string exclusion_scope = string }))	[]	no
maintenance_recurrence	Frequency of the recurring maintenance window in RFC5545 format.	string	""	no
maintenance_start_time	Time window specified for daily or recurring maintenance operations in RFC3339 format.	string	"05:00"	no
master_authorized_networks	List of master authorized networks. If none are provided, disallow external access (except the cluster node IPs, which GKE automatically whitelists).	list(object({ cidr_block = string display_name = string }))	[]	no
master_global_access_enabled	Whether the cluster master is accessible globally (from any region) or only within the same region as the private endpoint. Used when private set to true.	bool	true	no
master_ipv4_cidr_block	(Beta) The IP range in CIDR notation to use for the hosted master network. Used when private set to true.	string	"10.0.0.0/28"	no
monitoring_enable_managed_prometheus	Configuration for Managed Service for Prometheus. Whether or not the managed collection is enabled.	bool	false	no
monitoring_enabled_components	List of services to monitor: SYSTEM_COMPONENTS, WORKLOADS (provider version >= 3.89.0). Empty list is default GKE configuration.	list(string)	[]	no
monitoring_service	The monitoring service that the cluster should write metrics to. Automatically send metrics from pods in the cluster to the Google Cloud Monitoring API. VM metrics will be collected by Google Compute Engine regardless of this setting Available options include monitoring.googleapis.com, monitoring.googleapis.com/kubernetes (beta) and none	string	"monitoring.googleapis.com/kubernetes"	no
name	The name of the GKE cluster.	string	n/a	yes
network	The VPC network to host the GKE cluster in.	string	n/a	yes
network_policy	Enable network policy addon.	bool	false	no
network_policy_provider	The network policy provider. See more about network policy.	string	"CALICO"	no
network_project_id	The project ID of the shared VPC’s host (for shared vpc support).	string	""	no
network_tags	(Optional, Beta) - List of network tags applied to auto-provisioned node pools.	list(string)	[]	no
node_metadata	Specifies how node metadata is exposed to the workload running on the node	string	"GKE_METADATA"	no
node_pools	List of maps containing node pools.	list(map(any))	[ { “name”: “default” } ]	no
node_pools_labels	Map of maps containing node labels by node-pool name.	map(map(string))	{ “all”: {}, “default-node-pool”: {} }	no
node_pools_linux_node_configs_sysctls	Map of maps containing linux node config sysctls by node-pool name.	map(map(string))	{ “all”: {}, “default-node-pool”: {} }	no
node_pools_metadata	Map of maps containing node metadata by node-pool name	map(map(string))	{ “all”: {}, “default-node-pool”: {} }	no
node_pools_oauth_scopes	Map of lists containing node oauth scopes by node-pool name.	map(list(string))	{ “all”: [ “https://www.googleapis.com/auth/cloud-platform” ], “default-node-pool”: [] }	no
node_pools_resource_labels	Map of maps containing resource labels by node-pool name.	map(map(string))	{ “all”: {}, “default-node-pool”: {} }	no
node_pools_tags	Map of lists containing node network tags by node-pool name.	map(list(string))	{ “all”: [], “default-node-pool”: [] }	no
node_pools_taints	Map of lists containing node taints by node-pool name.	map(list(object({ key = string value = string effect = string })))	{ “all”: [], “default-node-pool”: [] }	no
non_masquerade_cidrs	List of strings in CIDR notation that specify the IP address ranges that do not use IP masquerading.	list(string)	[ “10.0.0.0/8”, “172.16.0.0/12”, “192.168.0.0/16” ]	no
notification_config_topic	The desired Pub/Sub topic to which notifications will be sent by GKE. Format is projects/{project}/topics/{topic}.	string	""	no
private	Create a private GKE cluster.	bool	true	no
region	The region to host the cluster in (optional if zonal cluster / required if regional)	string	null	no
regional	Whether is a regional cluster (zonal cluster if set false. WARNING: changing this after cluster creation is destructive!)	bool	true	no
registry_project_ids	Projects holding Google Container Registries. If empty, we use the cluster project. If a service account is created and the grant_registry_access variable is set to true, the storage.objectViewer and artifactregsitry.reader roles are assigned on these projects.	list(string)	[]	no
release_channel	The release channel of this cluster. This allows you to opt for the alpha releases as part of the RAPID option, REGULAR for standard release needs and STABLE when the tried-and-tested version becomes available.	string	"REGULAR"	no
remove_default_node_pool	Remove default node pool while setting up the cluster.	bool	true	no
resource_usage_export_dataset_id	The ID of a BigQuery Dataset for using BigQuery as the destination of resource usage export.	string	""	no
sandbox_enabled	(Beta) Enable GKE Sandbox (Do not forget to set image_type = COS_CONTAINERD to use it).	bool	false	no
service_account	The service account to run nodes as if not overridden in node_pools. The create_service_account variable default value (true) will cause a cluster-specific service account to be created. This service account should already exists and it will be used by the node pools. If you wish to only override the service account name, you can use service_account_name variable.	string	""	no
service_account_name	The name of the service account that will be created if create_service_account is true. If you wish to use an existing service account, use service_account variable.	string	""	no
service_external_ips	Whether external ips specified by a service will be allowed in this cluster.	bool	false	no
shadow_firewall_rules_log_config	The log_config for shadow firewall rules. You can set this variable to null to disable logging.	object({ metadata = string })	{ “metadata”: “INCLUDE_ALL_METADATA” }	no
shadow_firewall_rules_priority	The firewall priority of GKE shadow firewall rules. The priority should be less than default firewall, which is 1000.	number	999	no
stub_domains	Map of stub domains and their resolvers to forward DNS queries for a certain domain to an external DNS server. Not used for Autopilot GKE.	map(list(string))	{}	no
subnetwork	The subnetwork to host the GKE cluster in.	string	n/a	yes
subnetwork_cidr	CIDR of the subnetwork of nodes in GKE cluster.	string	n/a	yes
timeouts	Timeout for cluster operations.	map(string)	{}	no
upstream_nameservers	If specified, the values replace the nameservers taken by default from the node’s /etc/resolv.conf. Not used for Autopilot GKE.	list(string)	[]	no
windows_node_pools	List of maps containing Windows node pools.	list(map(string))	[]	no
workload_config_audit_mode	(beta) Worload config audit mode.	string	"DISABLED"	no
workload_vulnerability_mode	(beta) Vulnerability mode.	string	""	no
zones	The zones to host the cluster in (optional if regional cluster / required if zonal)	list(string)	[]	no

Outputs

Name	Description
autopilot_enabled	Autopilot GKE cluster.
ca_certificate	Cluster ca certificate (base64 encoded).
cloudrun_enabled	Whether CloudRun enabled.
cluster_id	GKE cluster ID.
dns_cache_enabled	Whether DNS Cache enabled
endpoint	Cluster endpoint.
gateway_api_channel	The gateway api channel of this cluster.
horizontal_pod_autoscaling_enabled	Whether horizontal pod autoscaling enabled.
http_load_balancing_enabled	Whether http load balancing enabled.
identity_namespace	Workload Identity pool.
identity_service_enabled	Whether Identity Service is enabled.
instance_group_urls	List of GKE generated instance groups.
intranode_visibility_enabled	Whether intra-node visibility is enabled.
istio_enabled	Whether Istio is enabled.
kubeconfig_path	Path where the kubeconfig file is saved.
location	Cluster location (region if regional cluster, zone if zonal cluster).
logging_service	Logging service used.
master_authorized_networks_config	Networks from which access to master is permitted.
master_ipv4_cidr_block	The IP range in CIDR notation used for the hosted master network.
master_version	Current master kubernetes version.
min_master_version	Minimum master kubernetes version.
monitoring_service	Monitoring service used.
name	GKE cluster name.
network_policy_enabled	Whether network policy enabled.
node_pools_names	List of node pools names.
node_pools_versions	Node pool versions by node pool name.
peering_name	The name of the peering between this cluster and the Google owned VPC.
pod_security_policy_enabled	Whether pod security policy is enabled.
private_enabled	Private GKE cluster.
region	Cluster region.
release_channel	The release channel of this cluster.
service_account	The service account to default running nodes as if not overridden in node_pools.
tpu_ipv4_cidr_block	The IP range in CIDR notation used for the TPUs
type	GKE cluster type (regional / zonal).
vertical_pod_autoscaling_enabled	Whether vertical pod autoscaling enabled.
zones	List of zones in which the cluster resides

Examples

• Complete Autopilot GKE
  • Requirements
  • Providers
  • Modules
  • Resources
  • Inputs
  • Outputs
• Complete Standard GKE
  • Requirements
  • Providers
  • Modules
  • Resources
  • Inputs
  • Outputs
• Simple GKE
  • Requirements
  • Providers
  • Modules
  • Resources
  • Inputs
  • Outputs