1 - Feature Gates

This page contains an overview of the various feature gates an administrator can specify on different Kubernetes components.

See feature stages for an explanation of the stages for a feature.

Overview

Feature gates are a set of key=value pairs that describe Kubernetes features. You can turn these features on or off using the --feature-gates command line flag on each Kubernetes component.

Each Kubernetes component lets you enable or disable a set of feature gates that are relevant to that component. Use -h flag to see a full set of feature gates for all components. To set feature gates for a component, such as kubelet, use the --feature-gates flag assigned to a list of feature pairs:

--feature-gates=...,GracefulNodeShutdown=true

The following tables are a summary of the feature gates that you can set on different Kubernetes components.

  • The "Since" column contains the Kubernetes release when a feature is introduced or its release stage is changed.
  • The "Until" column, if not empty, contains the last Kubernetes release in which you can still use a feature gate.
  • If a feature is in the Alpha or Beta state, you can find the feature listed in the Alpha/Beta feature gate table.
  • If a feature is stable you can find all stages for that feature listed in the Graduated/Deprecated feature gate table.
  • The Graduated/Deprecated feature gate table also lists deprecated and withdrawn features.

Feature gates for Alpha or Beta features

Feature gates for features in Alpha or Beta states
Feature Default Stage Since Until
AdmissionWebhookMatchConditions false Alpha 1.27 1.27
AdmissionWebhookMatchConditions true Beta 1.28
AggregatedDiscoveryEndpoint false Alpha 1.26 1.26
AggregatedDiscoveryEndpoint true Beta 1.27
AnyVolumeDataSource false Alpha 1.18 1.23
AnyVolumeDataSource true Beta 1.24
APIResponseCompression false Alpha 1.7 1.15
APIResponseCompression true Beta 1.16
APIServerIdentity false Alpha 1.20 1.25
APIServerIdentity true Beta 1.26
APIServerTracing false Alpha 1.22 1.26
APIServerTracing true Beta 1.27
AppArmor true Beta 1.4
CloudControllerManagerWebhook false Alpha 1.27
CloudDualStackNodeIPs false Alpha 1.27 1.28
CloudDualStackNodeIPs true Beta 1.29
ClusterTrustBundle false Alpha 1.27
ClusterTrustBundleProjection false Alpha 1.29
ComponentSLIs false Alpha 1.26 1.26
ComponentSLIs true Beta 1.27
ConsistentListFromCache false Alpha 1.28
ContainerCheckpoint false Alpha 1.25
ContextualLogging false Alpha 1.24
CPUManagerPolicyAlphaOptions false Alpha 1.23
CPUManagerPolicyBetaOptions true Beta 1.23
CPUManagerPolicyOptions false Alpha 1.22 1.22
CPUManagerPolicyOptions true Beta 1.23
CRDValidationRatcheting false Alpha 1.28
CronJobsScheduledAnnotation true Beta 1.28
CrossNamespaceVolumeDataSource false Alpha 1.26
CSIMigrationPortworx false Alpha 1.23 1.24
CSIMigrationPortworx false Beta 1.25
CSIVolumeHealth false Alpha 1.21
CustomCPUCFSQuotaPeriod false Alpha 1.12
DevicePluginCDIDevices false Alpha 1.28 1.28
DevicePluginCDIDevices true Beta 1.29
DisableCloudProviders false Alpha 1.22 1.28
DisableCloudProviders true Beta 1.29
DisableKubeletCloudCredentialProviders false Alpha 1.23 1.28
DisableKubeletCloudCredentialProviders true Beta 1.29
DisableNodeKubeProxyVersion false Alpha 1.29
DynamicResourceAllocation false Alpha 1.26
ElasticIndexedJob true Beta 1.27
EventedPLEG false Alpha 1.26 1.26
EventedPLEG false Beta 1.27
GracefulNodeShutdown false Alpha 1.20 1.20
GracefulNodeShutdown true Beta 1.21
GracefulNodeShutdownBasedOnPodPriority false Alpha 1.23 1.23
GracefulNodeShutdownBasedOnPodPriority true Beta 1.24
HonorPVReclaimPolicy false Alpha 1.23
HPAContainerMetrics false Alpha 1.20 1.26
HPAContainerMetrics true Beta 1.27
HPAScaleToZero false Alpha 1.16
ImageMaximumGCAge false Alpha 1.29
InPlacePodVerticalScaling false Alpha 1.27
InTreePluginAWSUnregister false Alpha 1.21
InTreePluginAzureDiskUnregister false Alpha 1.21
InTreePluginAzureFileUnregister false Alpha 1.21
InTreePluginGCEUnregister false Alpha 1.21
InTreePluginOpenStackUnregister false Alpha 1.21
InTreePluginPortworxUnregister false Alpha 1.23
InTreePluginvSphereUnregister false Alpha 1.21
JobBackoffLimitPerIndex false Alpha 1.28 1.28
JobBackoffLimitPerIndex true Beta 1.29
JobPodFailurePolicy false Alpha 1.25 1.25
JobPodFailurePolicy true Beta 1.26
JobPodReplacementPolicy false Alpha 1.28 1.28
JobPodReplacementPolicy true Beta 1.29
KubeletCgroupDriverFromCRI false Alpha 1.28
KubeletInUserNamespace false Alpha 1.22
KubeletPodResourcesDynamicResources false Alpha 1.27
KubeletPodResourcesGet false Alpha 1.27
KubeletSeparateDiskGC false Alpha 1.29
KubeletTracing false Alpha 1.25 1.26
KubeletTracing true Beta 1.27
KubeProxyDrainingTerminatingNodes false Alpha 1.28
LegacyServiceAccountTokenCleanUp false Alpha 1.28 1.28
LegacyServiceAccountTokenCleanUp true Beta 1.29
LoadBalancerIPMode false Alpha 1.29
LocalStorageCapacityIsolationFSQuotaMonitoring false Alpha 1.15
LogarithmicScaleDown false Alpha 1.21 1.21
LogarithmicScaleDown true Beta 1.22
LoggingAlphaOptions false Alpha 1.24
LoggingBetaOptions true Beta 1.24
MatchLabelKeysInPodAffinity false Alpha 1.29
MatchLabelKeysInPodTopologySpread false Alpha 1.25 1.26
MatchLabelKeysInPodTopologySpread true Beta 1.27
MaxUnavailableStatefulSet false Alpha 1.24
MemoryManager false Alpha 1.21 1.21
MemoryManager true Beta 1.22
MemoryQoS false Alpha 1.22
MinDomainsInPodTopologySpread false Alpha 1.24 1.24
MinDomainsInPodTopologySpread false Beta 1.25 1.26
MinDomainsInPodTopologySpread true Beta 1.27
MultiCIDRServiceAllocator false Alpha 1.27
NewVolumeManagerReconstruction false Beta 1.27 1.27
NewVolumeManagerReconstruction true Beta 1.28
NFTablesProxyMode false Alpha 1.29
NodeInclusionPolicyInPodTopologySpread false Alpha 1.25 1.25
NodeInclusionPolicyInPodTopologySpread true Beta 1.26
NodeLogQuery false Alpha 1.27
NodeSwap false Alpha 1.22 1.27
NodeSwap false Beta 1.28
OpenAPIEnums false Alpha 1.23 1.23
OpenAPIEnums true Beta 1.24
PDBUnhealthyPodEvictionPolicy false Alpha 1.26 1.26
PDBUnhealthyPodEvictionPolicy true Beta 1.27
PersistentVolumeLastPhaseTransitionTime false Alpha 1.28 1.28
PersistentVolumeLastPhaseTransitionTime true Beta 1.29
PodAndContainerStatsFromCRI false Alpha 1.23
PodDeletionCost false Alpha 1.21 1.21
PodDeletionCost true Beta 1.22
PodDisruptionConditions false Alpha 1.25 1.25
PodDisruptionConditions true Beta 1.26
PodHostIPs false Alpha 1.28 1.28
PodHostIPs true Beta 1.29
PodIndexLabel true Beta 1.28
PodLifecycleSleepAction false Alpha 1.29
PodReadyToStartContainersCondition false Alpha 1.28 1.28
PodReadyToStartContainersCondition true Beta 1.29
PodSchedulingReadiness false Alpha 1.26 1.26
PodSchedulingReadiness true Beta 1.27
ProcMountType false Alpha 1.12
QOSReserved false Alpha 1.11
RecoverVolumeExpansionFailure false Alpha 1.23
RotateKubeletServerCertificate false Alpha 1.7 1.11
RotateKubeletServerCertificate true Beta 1.12
RuntimeClassInImageCriApi false Alpha 1.29
SchedulerQueueingHints true Beta 1.28 1.28
SchedulerQueueingHints false Beta 1.29
SecurityContextDeny false Alpha 1.27
SELinuxMountReadWriteOncePod false Alpha 1.25 1.26
SELinuxMountReadWriteOncePod false Beta 1.27 1.27
SELinuxMountReadWriteOncePod true Beta 1.28
SeparateTaintEvictionController true Beta 1.29
ServiceAccountTokenJTI false Alpha 1.29
ServiceAccountTokenNodeBinding false Alpha 1.29
ServiceAccountTokenNodeBindingValidation false Alpha 1.29
ServiceAccountTokenPodNodeInfo false Alpha 1.29
SidecarContainers false Alpha 1.28 1.28
SidecarContainers true Beta 1.29
SizeMemoryBackedVolumes false Alpha 1.20 1.21
SizeMemoryBackedVolumes true Beta 1.22
StableLoadBalancerNodeSet true Beta 1.27
StatefulSetAutoDeletePVC false Alpha 1.23 1.26
StatefulSetAutoDeletePVC false Beta 1.27
StatefulSetStartOrdinal false Alpha 1.26 1.26
StatefulSetStartOrdinal true Beta 1.27
StorageVersionAPI false Alpha 1.20
StorageVersionHash false Alpha 1.14 1.14
StorageVersionHash true Beta 1.15
StructuredAuthenticationConfiguration false Alpha 1.29
StructuredAuthorizationConfiguration false Alpha 1.29
TopologyAwareHints false Alpha 1.21 1.22
TopologyAwareHints false Beta 1.23 1.23
TopologyAwareHints true Beta 1.24
TopologyManagerPolicyAlphaOptions false Alpha 1.26
TopologyManagerPolicyBetaOptions false Beta 1.26 1.27
TopologyManagerPolicyBetaOptions true Beta 1.28
TopologyManagerPolicyOptions false Alpha 1.26 1.27
TopologyManagerPolicyOptions true Beta 1.28
TranslateStreamCloseWebsocketRequests false Alpha 1.29
UnauthenticatedHTTP2DOSMitigation false Beta 1.28 1.28
UnauthenticatedHTTP2DOSMitigation true Beta 1.29
UnknownVersionInteroperabilityProxy false Alpha 1.28
UserNamespacesPodSecurityStandards false Alpha 1.29
UserNamespacesSupport false Alpha 1.28
ValidatingAdmissionPolicy false Alpha 1.26 1.27
ValidatingAdmissionPolicy false Beta 1.28
VolumeAttributesClass false Alpha 1.29
VolumeCapacityPriority false Alpha 1.21
WatchList false Alpha 1.27
WindowsHostNetwork true Alpha 1.26
WinDSR false Alpha 1.14
WinOverlay false Alpha 1.14 1.19
WinOverlay true Beta 1.20
ZeroLimitedNominalConcurrencyShares false Beta 1.29

Feature gates for graduated or deprecated features

Feature Gates for Graduated or Deprecated Features
Feature Default Stage Since Until
AllowServiceLBStatusOnNonLB false Deprecated 1.29
APIListChunking false Alpha 1.8 1.8
APIListChunking true Beta 1.9 1.28
APIListChunking true GA 1.29
APIPriorityAndFairness false Alpha 1.18 1.19
APIPriorityAndFairness true Beta 1.20 1.28
APIPriorityAndFairness true GA 1.29
APISelfSubjectReview false Alpha 1.26 1.26
APISelfSubjectReview true Beta 1.27 1.27
APISelfSubjectReview true GA 1.28
ConsistentHTTPGetHandlers true GA 1.25
CPUManager false Alpha 1.8 1.9
CPUManager true Beta 1.10 1.25
CPUManager true GA 1.26
CSIMigrationAzureFile false Alpha 1.15 1.20
CSIMigrationAzureFile false Beta 1.21 1.23
CSIMigrationAzureFile true Beta 1.24 1.25
CSIMigrationAzureFile true GA 1.26
CSIMigrationRBD false Alpha 1.23 1.27
CSIMigrationRBD false Deprecated 1.28
CSINodeExpandSecret false Alpha 1.25 1.26
CSINodeExpandSecret true Beta 1.27 1.28
CSINodeExpandSecret true GA 1.29
CustomResourceValidationExpressions false Alpha 1.23 1.24
CustomResourceValidationExpressions true Beta 1.25 1.28
CustomResourceValidationExpressions true GA 1.29
DefaultHostNetworkHostPortsInPodTemplates false Deprecated 1.28
EfficientWatchResumption false Alpha 1.20 1.20
EfficientWatchResumption true Beta 1.21 1.23
EfficientWatchResumption true GA 1.24
ExecProbeTimeout true GA 1.20
ExpandedDNSConfig false Alpha 1.22 1.25
ExpandedDNSConfig true Beta 1.26 1.27
ExpandedDNSConfig true GA 1.28
ExperimentalHostUserNamespaceDefaulting false Beta 1.5 1.27
ExperimentalHostUserNamespaceDefaulting false Deprecated 1.28
InTreePluginRBDUnregister false Alpha 1.23 1.27
InTreePluginRBDUnregister false Deprecated 1.28
IPTablesOwnershipCleanup false Alpha 1.25 1.26
IPTablesOwnershipCleanup true Beta 1.27 1.27
IPTablesOwnershipCleanup true GA 1.28
JobReadyPods false Alpha 1.23 1.23
JobReadyPods true Beta 1.24 1.28
JobReadyPods true GA 1.29
KMSv1 true Deprecated 1.28 1.28
KMSv1 false Deprecated 1.29
KMSv2 false Alpha 1.25 1.26
KMSv2 true Beta 1.27 1.28
KMSv2 true GA 1.29
KMSv2KDF false Beta 1.28 1.28
KMSv2KDF true GA 1.29
KubeletPodResources false Alpha 1.13 1.14
KubeletPodResources true Beta 1.15 1.27
KubeletPodResources true GA 1.28
KubeletPodResourcesGetAllocatable false Alpha 1.21 1.22
KubeletPodResourcesGetAllocatable true Beta 1.23 1.27
KubeletPodResourcesGetAllocatable true GA 1.28
LegacyServiceAccountTokenTracking false Alpha 1.26 1.26
LegacyServiceAccountTokenTracking true Beta 1.27 1.27
LegacyServiceAccountTokenTracking true GA 1.28
MinimizeIPTablesRestore false Alpha 1.26 1.26
MinimizeIPTablesRestore true Beta 1.27 1.27
MinimizeIPTablesRestore true GA 1.28
NodeOutOfServiceVolumeDetach false Alpha 1.24 1.25
NodeOutOfServiceVolumeDetach true Beta 1.26 1.27
NodeOutOfServiceVolumeDetach true GA 1.28
ProxyTerminatingEndpoints false Alpha 1.22 1.25
ProxyTerminatingEndpoints true Beta 1.26 1.27
ProxyTerminatingEndpoints true GA 1.28
ReadWriteOncePod false Alpha 1.22 1.26
ReadWriteOncePod true Beta 1.27 1.28
ReadWriteOncePod true GA 1.29
RemainingItemCount false Alpha 1.15 1.15
RemainingItemCount true Beta 1.16 1.28
RemainingItemCount true GA 1.29
RemoveSelfLink false Alpha 1.16 1.19
RemoveSelfLink true Beta 1.20 1.23
RemoveSelfLink true GA 1.24
ServerSideApply false Alpha 1.14 1.15
ServerSideApply true Beta 1.16 1.21
ServerSideApply true GA 1.22
ServerSideFieldValidation false Alpha 1.23 1.24
ServerSideFieldValidation true Beta 1.25 1.26
ServerSideFieldValidation true GA 1.27
ServiceNodePortStaticSubrange false Alpha 1.27 1.27
ServiceNodePortStaticSubrange true Beta 1.28 1.28
ServiceNodePortStaticSubrange true GA 1.29
SkipReadOnlyValidationGCE false Alpha 1.28 1.28
SkipReadOnlyValidationGCE true Deprecated 1.29
WatchBookmark false Alpha 1.15 1.15
WatchBookmark true Beta 1.16 1.16
WatchBookmark true GA 1.17

Using a feature

Feature stages

A feature can be in Alpha, Beta or GA stage. An Alpha feature means:

  • Disabled by default.
  • Might be buggy. Enabling the feature may expose bugs.
  • Support for feature may be dropped at any time without notice.
  • The API may change in incompatible ways in a later software release without notice.
  • Recommended for use only in short-lived testing clusters, due to increased risk of bugs and lack of long-term support.

A Beta feature means:

  • Usually enabled by default. Beta API groups are disabled by default.
  • The feature is well tested. Enabling the feature is considered safe.
  • Support for the overall feature will not be dropped, though details may change.
  • The schema and/or semantics of objects may change in incompatible ways in a subsequent beta or stable release. When this happens, we will provide instructions for migrating to the next version. This may require deleting, editing, and re-creating API objects. The editing process may require some thought. This may require downtime for applications that rely on the feature.
  • Recommended for only non-business-critical uses because of potential for incompatible changes in subsequent releases. If you have multiple clusters that can be upgraded independently, you may be able to relax this restriction.

A General Availability (GA) feature is also referred to as a stable feature. It means:

  • The feature is always enabled; you cannot disable it.
  • The corresponding feature gate is no longer needed.
  • Stable versions of features will appear in released software for many subsequent versions.

List of feature gates

Each feature gate is designed for enabling/disabling a specific feature:

  • AdmissionWebhookMatchConditions: Enable match conditions on mutating & validating admission webhooks.

  • AllowServiceLBStatusOnNonLB: Enables .status.ingress.loadBalancer to be set on Services of types other than LoadBalancer.

  • AggregatedDiscoveryEndpoint: Enable a single HTTP endpoint /discovery/<version> which supports native HTTP caching with ETags containing all APIResources known to the API server.

  • AnyVolumeDataSource: Enable use of any custom resource as the DataSource of a PVC.

  • APIListChunking: Enable the API clients to retrieve (LIST or GET) resources from API server in chunks.

  • APIPriorityAndFairness: Enable managing request concurrency with prioritization and fairness at each server. (Renamed from RequestManagement)

  • APIResponseCompression: Compress the API responses for LIST or GET requests.

  • APISelfSubjectReview: Activate the SelfSubjectReview API which allows users to see the requesting subject's authentication information. See API access to authentication information for a client for more details.

  • APIServerIdentity: Assign each API server an ID in a cluster, using a Lease.

  • APIServerTracing: Add support for distributed tracing in the API server. See Traces for Kubernetes System Components for more details.

  • AppArmor: Enable use of AppArmor mandatory access control for Pods running on Linux nodes. See AppArmor Tutorial for more details.

  • CloudControllerManagerWebhook: Enable webhooks in cloud controller manager.

  • CloudDualStackNodeIPs: Enables dual-stack kubelet --node-ip with external cloud providers. See Configure IPv4/IPv6 dual-stack for more details.

  • ClusterTrustBundle: Enable ClusterTrustBundle objects and kubelet integration.

  • ClusterTrustBundleProjection: clusterTrustBundle projected volume sources.

  • ComponentSLIs: Enable the /metrics/slis endpoint on Kubernetes components like kubelet, kube-scheduler, kube-proxy, kube-controller-manager, cloud-controller-manager allowing you to scrape health check metrics.

  • ConsistentHTTPGetHandlers: Normalize HTTP get URL and Header passing for lifecycle handlers with probers.

  • ConsistentListFromCache: Allow the API server to serve consistent lists from cache.

  • ContainerCheckpoint: Enables the kubelet checkpoint API. See Kubelet Checkpoint API for more details.

  • ContextualLogging: When you enable this feature gate, Kubernetes components that support contextual logging add extra detail to log output.

  • CPUManager: Enable container level CPU affinity support, see CPU Management Policies.

  • CPUManagerPolicyAlphaOptions: This allows fine-tuning of CPUManager policies, experimental, Alpha-quality options This feature gate guards a group of CPUManager options whose quality level is alpha. This feature gate will never graduate to beta or stable.

  • CPUManagerPolicyBetaOptions: This allows fine-tuning of CPUManager policies, experimental, Beta-quality options This feature gate guards a group of CPUManager options whose quality level is beta. This feature gate will never graduate to stable.

  • CPUManagerPolicyOptions: Allow fine-tuning of CPUManager policies.

  • CRDValidationRatcheting: Enable updates to custom resources to contain violations of their OpenAPI schema if the offending portions of the resource update did not change. See Validation Ratcheting for more details.

  • CronJobsScheduledAnnotation: Set the scheduled job time as an annotation on Jobs that were created on behalf of a CronJob.

  • CronJobTimeZone: Allow the use of the timeZone optional field in CronJobs

  • CrossNamespaceVolumeDataSource: Enable the usage of cross namespace volume data source to allow you to specify a source namespace in the dataSourceRef field of a PersistentVolumeClaim.

  • CSIMigrationAzureFile: Enables shims and translation logic to route volume operations from the Azure-File in-tree plugin to AzureFile CSI plugin. Supports falling back to in-tree AzureFile plugin for mount operations to nodes that have the feature disabled or that do not have AzureFile CSI plugin installed and configured. Does not support falling back for provision operations, for those the CSI plugin must be installed and configured. Requires CSIMigration feature flag enabled.

  • CSIMigrationPortworx: Enables shims and translation logic to route volume operations from the Portworx in-tree plugin to Portworx CSI plugin. Requires Portworx CSI driver to be installed and configured in the cluster.

  • CSIMigrationRBD: Enables shims and translation logic to route volume operations from the RBD in-tree plugin to Ceph RBD CSI plugin. Requires CSIMigration and csiMigrationRBD feature flags enabled and Ceph CSI plugin installed and configured in the cluster. This flag has been deprecated in favor of the InTreePluginRBDUnregister feature flag which prevents the registration of in-tree RBD plugin.

  • CSINodeExpandSecret: Enable passing secret authentication data to a CSI driver for use during a NodeExpandVolume CSI operation.

  • CSIVolumeHealth: Enable support for CSI volume health monitoring on node.

  • CustomCPUCFSQuotaPeriod: Enable nodes to change cpuCFSQuotaPeriod in kubelet config.

  • CustomResourceValidationExpressions: Enable expression language validation in CRD which will validate customer resource based on validation rules written in the x-kubernetes-validations extension.

  • DefaultHostNetworkHostPortsInPodTemplates:

    Changes when the default value of PodSpec.containers[*].ports[*].hostPort is assigned. The default is to only set a default value in Pods.

    Enabling this means a default will be assigned even to embedded PodSpecs (e.g. in a Deployment), which is the historical default.

  • DevicePluginCDIDevices: Enable support to CDI device IDs in the Device Plugin API.

  • DisableCloudProviders: Disables any functionality in kube-apiserver, kube-controller-manager and kubelet related to the --cloud-provider component flag.

  • DisableKubeletCloudCredentialProviders: Disable the in-tree functionality in kubelet to authenticate to a cloud provider container registry for image pull credentials.

  • DisableNodeKubeProxyVersion: Disable setting the kubeProxyVersion field of the Node.

  • DynamicResourceAllocation: Enables support for resources with custom parameters and a lifecycle that is independent of a Pod.

  • EfficientWatchResumption: Allows for storage-originated bookmark (progress notify) events to be delivered to the users. This is only applied to watch operations.

  • ElasticIndexedJob: Enables Indexed Jobs to be scaled up or down by mutating both spec.completions and spec.parallelism together such that spec.completions == spec.parallelism. See docs on elastic Indexed Jobs for more details.

  • EventedPLEG: Enable support for the kubelet to receive container life cycle events from the container runtime via an extension to CRI. (PLEG is an abbreviation for “Pod lifecycle event generator”). For this feature to be useful, you also need to enable support for container lifecycle events in each container runtime running in your cluster. If the container runtime does not announce support for container lifecycle events then the kubelet automatically switches to the legacy generic PLEG mechanism, even if you have this feature gate enabled.

  • ExecProbeTimeout: Ensure kubelet respects exec probe timeouts. This feature gate exists in case any of your existing workloads depend on a now-corrected fault where Kubernetes ignored exec probe timeouts. See readiness probes.

  • ExpandedDNSConfig: Enable kubelet and kube-apiserver to allow more DNS search paths and longer list of DNS search paths. This feature requires container runtime support(Containerd: v1.5.6 or higher, CRI-O: v1.22 or higher). See Expanded DNS Configuration.

  • ExperimentalHostUserNamespaceDefaulting: Enabling the defaulting user namespace to host. This is for containers that are using other host namespaces, host mounts, or containers that are privileged or using specific non-namespaced capabilities (e.g. MKNODE, SYS_MODULE etc.). This should only be enabled if user namespace remapping is enabled in the Docker daemon.

  • GracefulNodeShutdown: Enables support for graceful shutdown in kubelet. During a system shutdown, kubelet will attempt to detect the shutdown event and gracefully terminate pods running on the node. See Graceful Node Shutdown for more details.

  • GracefulNodeShutdownBasedOnPodPriority: Enables the kubelet to check Pod priorities when shutting down a node gracefully.

  • GRPCContainerProbe: Enables the gRPC probe method for {Liveness,Readiness,Startup}Probe. See Configure Liveness, Readiness and Startup Probes.

  • HonorPVReclaimPolicy: Honor persistent volume reclaim policy when it is Delete irrespective of PV-PVC deletion ordering. For more details, check the PersistentVolume deletion protection finalizer documentation.

  • HPAContainerMetrics: Enable the HorizontalPodAutoscaler to scale based on metrics from individual containers in target pods.

  • HPAScaleToZero: Enables setting minReplicas to 0 for HorizontalPodAutoscaler resources when using custom or external metrics.

  • ImageMaximumGCAge: Enables the kubelet configuration field imageMaximumGCAge, allowing an administrator to specify the age after which an image will be garbage collected.

  • InPlacePodVerticalScaling: Enables in-place Pod vertical scaling.

  • InTreePluginAWSUnregister: Stops registering the aws-ebs in-tree plugin in kubelet and volume controllers.

  • InTreePluginAzureDiskUnregister: Stops registering the azuredisk in-tree plugin in kubelet and volume controllers.

  • InTreePluginAzureFileUnregister: Stops registering the azurefile in-tree plugin in kubelet and volume controllers.

  • InTreePluginGCEUnregister: Stops registering the gce-pd in-tree plugin in kubelet and volume controllers.

  • InTreePluginOpenStackUnregister: Stops registering the OpenStack cinder in-tree plugin in kubelet and volume controllers.

  • InTreePluginPortworxUnregister: Stops registering the Portworx in-tree plugin in kubelet and volume controllers.

  • InTreePluginRBDUnregister: Stops registering the RBD in-tree plugin in kubelet and volume controllers.

  • InTreePluginvSphereUnregister: Stops registering the vSphere in-tree plugin in kubelet and volume controllers.

  • IPTablesOwnershipCleanup: This causes kubelet to no longer create legacy iptables rules.

  • JobBackoffLimitPerIndex: Allows specifying the maximal number of pod retries per index in Indexed jobs.

  • JobMutableNodeSchedulingDirectives: Allows updating node scheduling directives in the pod template of Job.

  • JobPodFailurePolicy: Allow users to specify handling of pod failures based on container exit codes and pod conditions.

  • JobPodReplacementPolicy: Allows you to specify pod replacement for terminating pods in a Job

  • JobReadyPods: Enables tracking the number of Pods that have a Ready condition. The count of Ready pods is recorded in the status of a Job status.

  • KMSv1: Enables KMS v1 API for encryption at rest. See Using a KMS Provider for data encryption for more details.

  • KMSv2: Enables KMS v2 API for encryption at rest. See Using a KMS Provider for data encryption for more details.

  • KMSv2KDF: Enables KMS v2 to generate single use data encryption keys. See Using a KMS Provider for data encryption for more details. If the KMSv2 feature gate is not enabled in your cluster, the value of the KMSv2KDF feature gate has no effect.

  • KubeletCgroupDriverFromCRI: Enable detection of the kubelet cgroup driver configuration option from the CRI. You can use this feature gate on nodes with a kubelet that supports the feature gate and where there is a CRI container runtime that supports the RuntimeConfig CRI call. If both CRI and kubelet support this feature, the kubelet ignores the cgroupDriver configuration setting (or deprecated --cgroup-driver command line argument). If you enable this feature gate and the container runtime doesn't support it, the kubelet falls back to using the driver configured using the cgroupDriver configuration setting. See Configuring a cgroup driver for more details.

  • KubeletInUserNamespace: Enables support for running kubelet in a user namespace. See Running Kubernetes Node Components as a Non-root User.

  • KubeletPodResources: Enable the kubelet's pod resources gRPC endpoint. See Support Device Monitoring for more details.

  • KubeletPodResourcesDynamicResources: Extend the kubelet's pod resources gRPC endpoint to to include resources allocated in ResourceClaims via DynamicResourceAllocation API. See resource allocation reporting for more details. with informations about the allocatable resources, enabling clients to properly track the free compute resources on a node.

  • KubeletPodResourcesGet: Enable the Get gRPC endpoint on kubelet's for Pod resources. This API augments the resource allocation reporting.

  • KubeletPodResourcesGetAllocatable: Enable the kubelet's pod resources GetAllocatableResources functionality. This API augments the resource allocation reporting

  • KubeletSeparateDiskGC: Enable kubelet to garbage collect container images and containers even when those are on a separate filesystem.

  • KubeletTracing: Add support for distributed tracing in the kubelet. When enabled, kubelet CRI interface and authenticated http servers are instrumented to generate OpenTelemetry trace spans. See Traces for Kubernetes System Components for more details.

  • KubeProxyDrainingTerminatingNodes: Implement connection draining for terminating nodes for externalTrafficPolicy: Cluster services.

  • LegacyServiceAccountTokenCleanUp: Enable cleaning up Secret-based service account tokens when they are not used in a specified time (default to be one year).

  • LegacyServiceAccountTokenNoAutoGeneration: Stop auto-generation of Secret-based service account tokens.

  • LegacyServiceAccountTokenTracking: Track usage of Secret-based service account tokens.

  • LoadBalancerIPMode: Allows setting ipMode for Services where type is set to LoadBalancer. See Specifying IPMode of load balancer status for more information.

  • LocalStorageCapacityIsolationFSQuotaMonitoring: When LocalStorageCapacityIsolation is enabled for local ephemeral storage and the backing filesystem for emptyDir volumes supports project quotas and they are enabled, use project quotas to monitor emptyDir volume storage consumption rather than filesystem walk for better performance and accuracy.

  • LogarithmicScaleDown: Enable semi-random selection of pods to evict on controller scaledown based on logarithmic bucketing of pod timestamps.

  • LoggingAlphaOptions: Allow fine-tuing of experimental, alpha-quality logging options.

  • LoggingBetaOptions: Allow fine-tuing of experimental, beta-quality logging options.

  • MatchLabelKeysInPodAffinity: Enable the matchLabelKeys and mismatchLabelKeys field for pod (anti)affinity.

  • MatchLabelKeysInPodTopologySpread: Enable the matchLabelKeys field for Pod topology spread constraints.

  • MaxUnavailableStatefulSet: Enables setting the maxUnavailable field for the rolling update strategy of a StatefulSet. The field specifies the maximum number of Pods that can be unavailable during the update.

  • MemoryManager: Allows setting memory affinity for a container based on NUMA topology.

  • MemoryQoS: Enable memory protection and usage throttle on pod / container using cgroup v2 memory controller.

  • MinDomainsInPodTopologySpread: Enable minDomains in Pod topology spread constraints.

  • MinimizeIPTablesRestore: Enables new performance improvement logics in the kube-proxy iptables mode.

  • MultiCIDRServiceAllocator: Track IP address allocations for Service cluster IPs using IPAddress objects.

  • NewVolumeManagerReconstruction:

    Enables improved discovery of mounted volumes during kubelet startup. Since this code has been significantly refactored, we allow to opt-out in case kubelet gets stuck at the startup or is not unmounting volumes from terminated Pods. Note that this refactoring was behind SELinuxMountReadWriteOncePod alpha feature gate in Kubernetes 1.25.

    Before Kubernetes v1.25, the kubelet used different default behavior for discovering mounted volumes during the kubelet startup. If you disable this feature gate (it's enabled by default), you select the legacy discovery behavior.

    In Kubernetes v1.25 and v1.26, this behavior toggle was part of the SELinuxMountReadWriteOncePod feature gate.

  • NFTablesProxyMode: Allow running kube-proxy with in nftables mode.

  • NodeInclusionPolicyInPodTopologySpread: Enable using nodeAffinityPolicy and nodeTaintsPolicy in Pod topology spread constraints when calculating pod topology spread skew.

  • NodeLogQuery: Enables querying logs of node services using the /logs endpoint.

  • NodeOutOfServiceVolumeDetach: When a Node is marked out-of-service using the node.kubernetes.io/out-of-service taint, Pods on the node will be forcefully deleted if they can not tolerate this taint, and the volume detach operations for Pods terminating on the node will happen immediately. The deleted Pods can recover quickly on different nodes.

  • NodeSwap: Enable the kubelet to allocate swap memory for Kubernetes workloads on a node. Must be used with KubeletConfiguration.failSwapOn set to false. For more details, please see swap memory

  • OpenAPIEnums: Enables populating "enum" fields of OpenAPI schemas in the spec returned from the API server.

  • PDBUnhealthyPodEvictionPolicy: Enables the unhealthyPodEvictionPolicy field of a PodDisruptionBudget. This specifies when unhealthy pods should be considered for eviction. Please see Unhealthy Pod Eviction Policy for more details.

  • PersistentVolumeLastPhaseTransitionTime: Adds a new field to PersistentVolume which holds a timestamp of when the volume last transitioned its phase.

  • PodAndContainerStatsFromCRI: Configure the kubelet to gather container and pod stats from the CRI container runtime rather than gathering them from cAdvisor. As of 1.26, this also includes gathering metrics from CRI and emitting them over /metrics/cadvisor (rather than having cAdvisor emit them directly).

  • PodDeletionCost: Enable the Pod Deletion Cost feature which allows users to influence ReplicaSet downscaling order.

  • PodDisruptionConditions: Enables support for appending a dedicated pod condition indicating that the pod is being deleted due to a disruption.

  • PodHostIPs: Enable the status.hostIPs field for pods and the downward API. The field lets you expose host IP addresses to workloads.

  • PodIndexLabel: Enables the Job controller and StatefulSet controller to add the pod index as a label when creating new pods. See Job completion mode docs and StatefulSet pod index label docs for more details.

  • PodLifecycleSleepAction: Enables the sleep action in Container lifecycle hooks.

  • PodReadyToStartContainersCondition:

    Enable the kubelet to mark the PodReadyToStartContainers condition on pods.

    This feature gate was previously known as PodHasNetworkCondition, and the associated condition was named PodHasNetwork.

  • PodSchedulingReadiness: Enable setting schedulingGates field to control a Pod's scheduling readiness.

  • ProcMountType: Enables control over the type proc mounts for containers by setting the procMount field of a SecurityContext.

  • ProxyTerminatingEndpoints: Enable the kube-proxy to handle terminating endpoints when ExternalTrafficPolicy=Local.

  • QOSReserved: Allows resource reservations at the QoS level preventing pods at lower QoS levels from bursting into resources requested at higher QoS levels (memory only for now).

  • ReadWriteOncePod: Enables the usage of ReadWriteOncePod PersistentVolume access mode.

  • RecoverVolumeExpansionFailure: Enables users to edit their PVCs to smaller sizes so as they can recover from previously issued volume expansion failures. See Recovering from Failure when Expanding Volumes for more details.

  • RemainingItemCount: Allow the API servers to show a count of remaining items in the response to a chunking list request.

  • RemoveSelfLink: Sets the .metadata.selfLink field to blank (empty string) for all objects and collections. This field has been deprecated since the Kubernetes v1.16 release. When this feature is enabled, the .metadata.selfLink field remains part of the Kubernetes API, but is always unset.

  • RotateKubeletServerCertificate: Enable the rotation of the server TLS certificate on the kubelet. See kubelet configuration for more details.

  • RuntimeClassInImageCriApi: Enables images to be pulled based on the [runtime class] (/docs/concepts/containers/runtime-class/) of the pods that reference them.

  • SchedulerQueueingHints: Enables the scheduler's queueing hints enhancement, which benefits to reduce the useless requeueing. The scheduler retries scheduling pods if something changes in the cluster that could make the pod scheduled. Queueing hints are internal signals that allow the scheduler to filter the changes in the cluster that are relevant to the unscheduled pod, based on previous scheduling attempts.

  • SecurityContextDeny: This gate signals that the SecurityContextDeny admission controller is deprecated.

  • SELinuxMountReadWriteOncePod: Speeds up container startup by allowing kubelet to mount volumes for a Pod directly with the correct SELinux label instead of changing each file on the volumes recursively. The initial implementation focused on ReadWriteOncePod volumes.

  • SeparateTaintEvictionController: Enables running TaintEvictionController, that performs Taint-based Evictions, in a controller separated from NodeLifecycleController. When this feature is enabled, users can optionally disable Taint-based Eviction setting the --controllers=-taint-eviction-controller flag on the kube-controller-manager.

  • ServerSideApply: Enables the Sever Side Apply (SSA) feature on the API Server.

  • ServerSideFieldValidation: Enables server-side field validation. This means the validation of resource schema is performed at the API server side rather than the client side (for example, the kubectl create or kubectl apply command line).

  • ServiceAccountTokenJTI: Controls whether JTIs (UUIDs) are embedded into generated service account tokens, and whether these JTIs are recorded into the Kubernetes audit log for future requests made by these tokens.

  • ServiceAccountTokenNodeBinding: Controls whether the apiserver allows binding service account tokens to Node objects.

  • ServiceAccountTokenNodeBindingValidation: Controls whether the apiserver will validate a Node reference in service account tokens.

  • ServiceAccountTokenPodNodeInfo: Controls whether the apiserver embeds the node name and uid for the associated node when issuing service account tokens bound to Pod objects.

  • ServiceNodePortStaticSubrange: Enables the use of different port allocation strategies for NodePort Services. For more details, see reserve NodePort ranges to avoid collisions.

  • SidecarContainers: Allow setting the restartPolicy of an init container to Always so that the container becomes a sidecar container (restartable init containers). See Sidecar containers and restartPolicy for more details.

  • SizeMemoryBackedVolumes: Enable kubelets to determine the size limit for memory-backed volumes (mainly emptyDir volumes).

  • SkipReadOnlyValidationGCE: Skip validation for GCE, will enable in the next version.

  • StableLoadBalancerNodeSet: Enables less load balancer re-configurations by the service controller (KCCM) as an effect of changing node state.

  • StatefulSetAutoDeletePVC: Allows the use of the optional .spec.persistentVolumeClaimRetentionPolicy field, providing control over the deletion of PVCs in a StatefulSet's lifecycle. See PersistentVolumeClaim retention for more details.

  • StatefulSetStartOrdinal: Allow configuration of the start ordinal in a StatefulSet. See Start ordinal for more details.

  • StorageVersionAPI: Enable the storage version API.

  • StorageVersionHash: Allow API servers to expose the storage version hash in the discovery.

  • StructuredAuthenticationConfiguration: Enable structured authentication configuration for the API server.

  • StructuredAuthorizationConfiguration: Enable structured authorization configuration, so that cluster administrators can specify more than one authorization webhook in the API server handler chain.

  • TopologyAwareHints: Enables topology aware routing based on topology hints in EndpointSlices. See Topology Aware Hints for more details.

  • TopologyManager: Enable a mechanism to coordinate fine-grained hardware resource assignments for different components in Kubernetes. See Control Topology Management Policies on a node.

  • TopologyManagerPolicyAlphaOptions: Allow fine-tuning of topology manager policies, experimental, Alpha-quality options. This feature gate guards a group of topology manager options whose quality level is alpha. This feature gate will never graduate to beta or stable.

  • TopologyManagerPolicyBetaOptions: Allow fine-tuning of topology manager policies, experimental, Beta-quality options. This feature gate guards a group of topology manager options whose quality level is beta. This feature gate will never graduate to stable.

  • TopologyManagerPolicyOptions: Allow fine-tuning of topology manager policies,

  • TranslateStreamCloseWebsocketRequests: Allow WebSocket streaming of the remote command sub-protocol (exec, cp, attach) from clients requesting version 5 (v5) of the sub-protocol.

  • UnauthenticatedHTTP2DOSMitigation: Enables HTTP/2 Denial of Service (DoS) mitigations for unauthenticated clients. Kubernetes v1.28.0 through v1.28.2 do not include this feature gate.

  • UnknownVersionInteroperabilityProxy: Proxy resource requests to the correct peer kube-apiserver when multiple kube-apiservers exist at varied versions. See Mixed version proxy for more information.

  • UserNamespacesPodSecurityStandards: Enable Pod Security Standards policies relaxation for pods that run with namespaces. You must set the value of this feature gate consistently across all nodes in your cluster, and you must also enable UserNamespacesSupport to use this feature.

  • UserNamespacesSupport: Enable user namespace support for Pods.

  • ValidatingAdmissionPolicy: Enable ValidatingAdmissionPolicy support for CEL validations be used in Admission Control.

  • VolumeAttributesClass: Enable support for VolumeAttributesClasses. See Volume Attributes Classes for more information.

  • VolumeCapacityPriority: Enable support for prioritizing nodes in different topologies based on available PV capacity.

  • WatchBookmark: Enable support for watch bookmark events.

  • WatchList: Enable support for streaming initial state of objects in watch requests.

  • WindowsHostNetwork: Enables support for joining Windows containers to a hosts' network namespace.

  • WinDSR: Allows kube-proxy to create DSR loadbalancers for Windows.

  • WinOverlay: Allows kube-proxy to run in overlay mode for Windows.

  • ZeroLimitedNominalConcurrencyShares: Allow Priority & Fairness in the API server to use a zero value for the nominalConcurrencyShares field of the limited` section of a priority level.

What's next

  • The deprecation policy for Kubernetes explains the project's approach to removing features and components.
  • Since Kubernetes 1.24, new beta APIs are not enabled by default. When enabling a beta feature, you will also need to enable any associated API resources. For example, to enable a particular resource like storage.k8s.io/v1beta1/csistoragecapacities, set --runtime-config=storage.k8s.io/v1beta1/csistoragecapacities. See API Versioning for more details on the command line flags.

2 - Feature Gates (removed)

This page contains list of feature gates that have been removed. The information on this page is for reference. A removed feature gate is different from a GA'ed or deprecated one in that a removed one is no longer recognized as a valid feature gate. However, a GA'ed or a deprecated feature gate is still recognized by the corresponding Kubernetes components although they are unable to cause any behavior differences in a cluster.

For feature gates that are still recognized by the Kubernetes components, please refer to the Alpha/Beta feature gate table or the Graduated/Deprecated feature gate table

Feature gates that are removed

In the following table:

  • The "From" column contains the Kubernetes release when a feature is introduced or its release stage is changed.
  • The "To" column, if not empty, contains the last Kubernetes release in which you can still use a feature gate. If the feature stage is either "Deprecated" or "GA", the "To" column is the Kubernetes release when the feature is removed.
Feature Gates Removed
Feature Default Stage From To
Accelerators false Alpha 1.6 1.10
Accelerators Deprecated 1.11 1.11
AdvancedAuditing false Alpha 1.7 1.7
AdvancedAuditing true Beta 1.8 1.11
AdvancedAuditing true GA 1.12 1.27
AffinityInAnnotations false Alpha 1.6 1.7
AffinityInAnnotations Deprecated 1.8 1.8
AllowExtTrafficLocalEndpoints false Beta 1.4 1.6
AllowExtTrafficLocalEndpoints true GA 1.7 1.9
AllowInsecureBackendProxy true Beta 1.17 1.20
AllowInsecureBackendProxy true GA 1.21 1.25
AttachVolumeLimit false Alpha 1.11 1.11
AttachVolumeLimit true Beta 1.12 1.16
AttachVolumeLimit true GA 1.17 1.21
BalanceAttachedNodeVolumes false Alpha 1.11 1.21
BalanceAttachedNodeVolumes false Deprecated 1.22 1.22
BlockVolume false Alpha 1.9 1.12
BlockVolume true Beta 1.13 1.17
BlockVolume true GA 1.18 1.21
BoundServiceAccountTokenVolume false Alpha 1.13 1.20
BoundServiceAccountTokenVolume true Beta 1.21 1.21
BoundServiceAccountTokenVolume true GA 1.22 1.23
ConfigurableFSGroupPolicy false Alpha 1.18 1.19
ConfigurableFSGroupPolicy true Beta 1.20 1.22
ConfigurableFSGroupPolicy true GA 1.23 1.25
ControllerManagerLeaderMigration false Alpha 1.21 1.21
ControllerManagerLeaderMigration true Beta 1.22 1.23
ControllerManagerLeaderMigration true GA 1.24 1.26
CRIContainerLogRotation false Alpha 1.10 1.10
CRIContainerLogRotation true Beta 1.11 1.20
CRIContainerLogRotation true GA 1.21 1.22
CronJobControllerV2 false Alpha 1.20 1.20
CronJobControllerV2 true Beta 1.21 1.21
CronJobControllerV2 true GA 1.22 1.23
CronJobTimeZone false Alpha 1.24 1.24
CronJobTimeZone true Beta 1.25 1.26
CronJobTimeZone true GA 1.27 1.28
CSIBlockVolume false Alpha 1.11 1.13
CSIBlockVolume true Beta 1.14 1.17
CSIBlockVolume true GA 1.18 1.21
CSIDriverRegistry false Alpha 1.12 1.13
CSIDriverRegistry true Beta 1.14 1.17
CSIDriverRegistry true GA 1.18 1.21
CSIInlineVolume false Alpha 1.15 1.15
CSIInlineVolume true Beta 1.16 1.24
CSIInlineVolume true GA 1.25 1.26
CSIMigration false Alpha 1.14 1.16
CSIMigration true Beta 1.17 1.24
CSIMigration true GA 1.25 1.26
CSIMigrationAWS false Alpha 1.14 1.16
CSIMigrationAWS false Beta 1.17 1.22
CSIMigrationAWS true Beta 1.23 1.24
CSIMigrationAWS true GA 1.25 1.26
CSIMigrationAWSComplete false Alpha 1.17 1.20
CSIMigrationAWSComplete Deprecated 1.21 1.21
CSIMigrationAzureDisk false Alpha 1.15 1.18
CSIMigrationAzureDisk false Beta 1.19 1.22
CSIMigrationAzureDisk true Beta 1.23 1.23
CSIMigrationAzureDisk true GA 1.24 1.26
CSIMigrationAzureDiskComplete false Alpha 1.17 1.20
CSIMigrationAzureDiskComplete Deprecated 1.21 1.21
CSIMigrationAzureFileComplete false Alpha 1.17 1.20
CSIMigrationAzureFileComplete Deprecated 1.21 1.21
CSIMigrationGCE false Alpha 1.14 1.16
CSIMigrationGCE false Beta 1.17 1.22
CSIMigrationGCE true Beta 1.23 1.24
CSIMigrationGCE true GA 1.25 1.27
CSIMigrationGCEComplete false Alpha 1.17 1.20
CSIMigrationGCEComplete Deprecated 1.21 1.21
CSIMigrationOpenStack false Alpha 1.14 1.17
CSIMigrationOpenStack true Beta 1.18 1.23
CSIMigrationOpenStack true GA 1.24 1.25
CSIMigrationOpenStackComplete false Alpha 1.17 1.20
CSIMigrationOpenStackComplete Deprecated 1.21 1.21
CSIMigrationvSphere false Alpha 1.18 1.18
CSIMigrationvSphere false Beta 1.19 1.24
CSIMigrationvSphere true Beta 1.25 1.25
CSIMigrationvSphere true GA 1.26 1.28
CSIMigrationvSphereComplete false Beta 1.19 1.21
CSIMigrationvSphereComplete Deprecated 1.22 1.22
CSINodeInfo false Alpha 1.12 1.13
CSINodeInfo true Beta 1.14 1.16
CSINodeInfo true GA 1.17 1.22
CSIPersistentVolume false Alpha 1.9 1.9
CSIPersistentVolume true Beta 1.10 1.12
CSIPersistentVolume true GA 1.13 1.16
CSIServiceAccountToken false Alpha 1.20 1.20
CSIServiceAccountToken true Beta 1.21 1.21
CSIServiceAccountToken true GA 1.22 1.24
CSIStorageCapacity false Alpha 1.19 1.20
CSIStorageCapacity true Beta 1.21 1.23
CSIStorageCapacity true GA 1.24 1.27
CSIVolumeFSGroupPolicy false Alpha 1.19 1.19
CSIVolumeFSGroupPolicy true Beta 1.20 1.22
CSIVolumeFSGroupPolicy true GA 1.23 1.25
CSRDuration true Beta 1.22 1.23
CSRDuration true GA 1.24 1.25
CustomPodDNS false Alpha 1.9 1.9
CustomPodDNS true Beta 1.10 1.13
CustomPodDNS true GA 1.14 1.16
CustomResourceDefaulting false Alpha 1.15 1.15
CustomResourceDefaulting true Beta 1.16 1.16
CustomResourceDefaulting true GA 1.17 1.18
CustomResourcePublishOpenAPI false Alpha 1.14 1.14
CustomResourcePublishOpenAPI true Beta 1.15 1.15
CustomResourcePublishOpenAPI true GA 1.16 1.18
CustomResourceSubresources false Alpha 1.10 1.10
CustomResourceSubresources true Beta 1.11 1.15
CustomResourceSubresources true GA 1.16 1.18
CustomResourceValidation false Alpha 1.8 1.8
CustomResourceValidation true Beta 1.9 1.15
CustomResourceValidation true GA 1.16 1.18
CustomResourceWebhookConversion false Alpha 1.13 1.14
CustomResourceWebhookConversion true Beta 1.15 1.15
CustomResourceWebhookConversion true GA 1.16 1.18
DaemonSetUpdateSurge false Alpha 1.21 1.21
DaemonSetUpdateSurge true Beta 1.22 1.24
DaemonSetUpdateSurge true GA 1.25 1.28
DefaultPodTopologySpread false Alpha 1.19 1.19
DefaultPodTopologySpread true Beta 1.20 1.23
DefaultPodTopologySpread true GA 1.24 1.25
DelegateFSGroupToCSIDriver false Alpha 1.22 1.22
DelegateFSGroupToCSIDriver true Beta 1.23 1.25
DelegateFSGroupToCSIDriver true GA 1.26 1.27
DevicePlugins false Alpha 1.8 1.9
DevicePlugins true Beta 1.10 1.25
DevicePlugins true GA 1.26 1.27
DisableAcceleratorUsageMetrics false Alpha 1.19 1.19
DisableAcceleratorUsageMetrics true Beta 1.20 1.24
DisableAcceleratorUsageMetrics true GA 1.25 1.27
DownwardAPIHugePages false Alpha 1.20 1.20
DownwardAPIHugePages false Beta 1.21 1.21
DownwardAPIHugePages true Beta 1.22 1.26
DownwardAPIHugePages true GA 1.27 1.28
DryRun false Alpha 1.12 1.12
DryRun true Beta 1.13 1.18
DryRun true GA 1.19 1.27
DynamicAuditing false Alpha 1.13 1.18
DynamicAuditing Deprecated 1.19 1.19
DynamicKubeletConfig false Alpha 1.4 1.10
DynamicKubeletConfig true Beta 1.11 1.21
DynamicKubeletConfig false Deprecated 1.22 1.25
DynamicProvisioningScheduling false Alpha 1.11 1.11
DynamicProvisioningScheduling Deprecated 1.12
DynamicVolumeProvisioning true Alpha 1.3 1.7
DynamicVolumeProvisioning true GA 1.8 1.12
EnableAggregatedDiscoveryTimeout true Deprecated 1.16 1.17
EnableEquivalenceClassCache false Alpha 1.8 1.12
EnableEquivalenceClassCache Deprecated 1.13 1.23
EndpointSlice false Alpha 1.16 1.16
EndpointSlice false Beta 1.17 1.17
EndpointSlice true Beta 1.18 1.20
EndpointSlice true GA 1.21 1.24
EndpointSliceNodeName false Alpha 1.20 1.20
EndpointSliceNodeName true GA 1.21 1.24
EndpointSliceProxying false Alpha 1.18 1.18
EndpointSliceProxying true Beta 1.19 1.21
EndpointSliceProxying true GA 1.22 1.24
EndpointSliceTerminatingCondition false Alpha 1.20 1.21
EndpointSliceTerminatingCondition true Beta 1.22 1.25
EndpointSliceTerminatingCondition true GA 1.26 1.27
EphemeralContainers false Alpha 1.16 1.22
EphemeralContainers true Beta 1.23 1.24
EphemeralContainers true GA 1.25 1.26
EvenPodsSpread false Alpha 1.16 1.17
EvenPodsSpread true Beta 1.18 1.18
EvenPodsSpread true GA 1.19 1.21
ExpandCSIVolumes false Alpha 1.14 1.15
ExpandCSIVolumes true Beta 1.16 1.23
ExpandCSIVolumes true GA 1.24 1.26
ExpandInUsePersistentVolumes false Alpha 1.11 1.14
ExpandInUsePersistentVolumes true Beta 1.15 1.23
ExpandInUsePersistentVolumes true GA 1.24 1.26
ExpandPersistentVolumes false Alpha 1.8 1.10
ExpandPersistentVolumes true Beta 1.11 1.23
ExpandPersistentVolumes true GA 1.24 1.26
ExperimentalCriticalPodAnnotation false Alpha 1.5 1.12
ExperimentalCriticalPodAnnotation false Deprecated 1.13 1.16
ExternalPolicyForExternalIP true GA 1.18 1.22
GCERegionalPersistentDisk true Beta 1.10 1.12
GCERegionalPersistentDisk true GA 1.13 1.16
GenericEphemeralVolume false Alpha 1.19 1.20
GenericEphemeralVolume true Beta 1.21 1.22
GenericEphemeralVolume true GA 1.23 1.24
GRPCContainerProbe false Alpha 1.23 1.23
GRPCContainerProbe true Beta 1.24 1.26
GRPCContainerProbe true GA 1.27 1.28
HugePages false Alpha 1.8 1.9
HugePages true Beta 1.10 1.13
HugePages true GA 1.14 1.16
HugePageStorageMediumSize false Alpha 1.18 1.18
HugePageStorageMediumSize true Beta 1.19 1.21
HugePageStorageMediumSize true GA 1.22 1.24
HyperVContainer false Alpha 1.10 1.19
HyperVContainer false Deprecated 1.20 1.20
IdentifyPodOS false Alpha 1.23 1.23
IdentifyPodOS true Beta 1.24 1.24
IdentifyPodOS true GA 1.25 1.26
ImmutableEphemeralVolumes false Alpha 1.18 1.18
ImmutableEphemeralVolumes true Beta 1.19 1.20
ImmutableEphemeralVolumes true GA 1.21 1.24
IndexedJob false Alpha 1.21 1.21
IndexedJob true Beta 1.22 1.23
IndexedJob true GA 1.24 1.25
IngressClassNamespacedParams false Alpha 1.21 1.21
IngressClassNamespacedParams true Beta 1.22 1.22
IngressClassNamespacedParams true GA 1.23 1.24
Initializers false Alpha 1.7 1.13
Initializers Deprecated 1.14 1.14
IPv6DualStack false Alpha 1.15 1.20
IPv6DualStack true Beta 1.21 1.22
IPv6DualStack true GA 1.23 1.24
JobMutableNodeSchedulingDirectives true Beta 1.23 1.26
JobMutableNodeSchedulingDirectives true GA 1.27 1.28
JobTrackingWithFinalizers false Alpha 1.22 1.22
JobTrackingWithFinalizers false Beta 1.23 1.24
JobTrackingWithFinalizers true Beta 1.25 1.25
JobTrackingWithFinalizers true GA 1.26 1.28
KubeletConfigFile false Alpha 1.8 1.9
KubeletConfigFile Deprecated 1.10 1.10
KubeletCredentialProviders false Alpha 1.20 1.23
KubeletCredentialProviders true Beta 1.24 1.25
KubeletCredentialProviders true GA 1.26 1.28
KubeletPluginsWatcher false Alpha 1.11 1.11
KubeletPluginsWatcher true Beta 1.12 1.12
KubeletPluginsWatcher true GA 1.13 1.16
LegacyNodeRoleBehavior false Alpha 1.16 1.18
LegacyNodeRoleBehavior true Beta 1.19 1.20
LegacyNodeRoleBehavior false GA 1.21 1.22
LegacyServiceAccountTokenNoAutoGeneration true Beta 1.24 1.25
LegacyServiceAccountTokenNoAutoGeneration true GA 1.26 1.28
LocalStorageCapacityIsolation false Alpha 1.7 1.9
LocalStorageCapacityIsolation true Beta 1.10 1.24
LocalStorageCapacityIsolation true GA 1.25 1.26
MixedProtocolLBService false Alpha 1.20 1.23
MixedProtocolLBService true Beta 1.24 1.25
MixedProtocolLBService true GA 1.26 1.27
MountContainers false Alpha 1.9 1.16
MountContainers false Deprecated 1.17 1.17
MountPropagation false Alpha 1.8 1.9
MountPropagation true Beta 1.10 1.11
MountPropagation true GA 1.12 1.14
MultiCIDRRangeAllocator false Alpha 1.25 1.28
NamespaceDefaultLabelName true Beta 1.21 1.21
NamespaceDefaultLabelName true GA 1.22 1.23
NetworkPolicyEndPort false Alpha 1.21 1.21
NetworkPolicyEndPort true Beta 1.22 1.24
NetworkPolicyEndPort true GA 1.25 1.26
NetworkPolicyStatus false Alpha 1.24 1.27
NodeDisruptionExclusion false Alpha 1.16 1.18
NodeDisruptionExclusion true Beta 1.19 1.20
NodeDisruptionExclusion true GA 1.21 1.22
NodeLease false Alpha 1.12 1.13
NodeLease true Beta 1.14 1.16
NodeLease true GA 1.17 1.23
NonPreemptingPriority false Alpha 1.15 1.18
NonPreemptingPriority true Beta 1.19 1.23
NonPreemptingPriority true GA 1.24 1.25
OpenAPIV3 false Alpha 1.23 1.23
OpenAPIV3 true Beta 1.24 1.26
OpenAPIV3 true GA 1.27 1.28
PersistentLocalVolumes false Alpha 1.7 1.9
PersistentLocalVolumes true Beta 1.10 1.13
PersistentLocalVolumes true GA 1.14 1.16
PodAffinityNamespaceSelector false Alpha 1.21 1.21
PodAffinityNamespaceSelector true Beta 1.22 1.23
PodAffinityNamespaceSelector true GA 1.24 1.25
PodDisruptionBudget false Alpha 1.3 1.4
PodDisruptionBudget true Beta 1.5 1.20
PodDisruptionBudget true GA 1.21 1.25
PodHasNetworkCondition false Alpha 1.25 1.27
PodOverhead false Alpha 1.16 1.17
PodOverhead true Beta 1.18 1.23
PodOverhead true GA 1.24 1.25
PodPriority false Alpha 1.8 1.10
PodPriority true Beta 1.11 1.13
PodPriority true GA 1.14 1.18
PodReadinessGates false Alpha 1.11 1.11
PodReadinessGates true Beta 1.12 1.13
PodReadinessGates true GA 1.14 1.16
PodSecurity false Alpha 1.22 1.22
PodSecurity true Beta 1.23 1.24
PodSecurity true GA 1.25 1.27
PodShareProcessNamespace false Alpha 1.10 1.11
PodShareProcessNamespace true Beta 1.12 1.16
PodShareProcessNamespace true GA 1.17 1.19
PreferNominatedNode false Alpha 1.21 1.21
PreferNominatedNode true Beta 1.22 1.23
PreferNominatedNode true GA 1.24 1.25
ProbeTerminationGracePeriod false Alpha 1.21 1.21
ProbeTerminationGracePeriod false Beta 1.22 1.24
ProbeTerminationGracePeriod true Beta 1.25 1.27
ProbeTerminationGracePeriod true GA 1.28 1.28
PVCProtection false Alpha 1.9 1.9
PVCProtection Deprecated 1.10 1.10
RequestManagement false Alpha 1.15 1.16
RequestManagement Deprecated 1.17 1.17
ResourceLimitsPriorityFunction false Alpha 1.9 1.18
ResourceLimitsPriorityFunction Deprecated 1.19 1.19
ResourceQuotaScopeSelectors false Alpha 1.11 1.11
ResourceQuotaScopeSelectors true Beta 1.12 1.16
ResourceQuotaScopeSelectors true GA 1.17 1.18
RetroactiveDefaultStorageClass false Alpha 1.25 1.25
RetroactiveDefaultStorageClass true Beta 1.26 1.27
RetroactiveDefaultStorageClass true GA 1.28 1.28
RootCAConfigMap false Alpha 1.13 1.19
RootCAConfigMap true Beta 1.20 1.20
RootCAConfigMap true GA 1.21 1.22
RotateKubeletClientCertificate true Beta 1.8 1.18
RotateKubeletClientCertificate true GA 1.19 1.21
RunAsGroup true Beta 1.14 1.20
RunAsGroup true GA 1.21 1.22
RuntimeClass false Alpha 1.12 1.13
RuntimeClass true Beta 1.14 1.19
RuntimeClass true GA 1.20 1.24
ScheduleDaemonSetPods false Alpha 1.11 1.11
ScheduleDaemonSetPods true Beta 1.12 1.16
ScheduleDaemonSetPods true GA 1.17 1.18
SCTPSupport false Alpha 1.12 1.18
SCTPSupport true Beta 1.19 1.19
SCTPSupport true GA 1.20 1.22
SeccompDefault false Alpha 1.22 1.24
SeccompDefault true Beta 1.25 1.26
SeccompDefault true GA 1.27 1.28
SelectorIndex false Alpha 1.18 1.18
SelectorIndex true Beta 1.19 1.19
SelectorIndex true GA 1.20 1.25
ServiceAccountIssuerDiscovery false Alpha 1.18 1.19
ServiceAccountIssuerDiscovery true Beta 1.20 1.20
ServiceAccountIssuerDiscovery true GA 1.21 1.23
ServiceAppProtocol false Alpha 1.18 1.18
ServiceAppProtocol true Beta 1.19 1.19
ServiceAppProtocol true GA 1.20 1.22
ServiceInternalTrafficPolicy false Alpha 1.21 1.21
ServiceInternalTrafficPolicy true Beta 1.22 1.25
ServiceInternalTrafficPolicy true GA 1.26 1.27
ServiceIPStaticSubrange false Alpha 1.24 1.24
ServiceIPStaticSubrange true Beta 1.25 1.25
ServiceIPStaticSubrange true GA 1.26 1.27
ServiceLBNodePortControl false Alpha 1.20 1.21
ServiceLBNodePortControl true Beta 1.22 1.23
ServiceLBNodePortControl true GA 1.24 1.25
ServiceLoadBalancerClass false Alpha 1.21 1.21
ServiceLoadBalancerClass true Beta 1.22 1.23
ServiceLoadBalancerClass true GA 1.24 1.25
ServiceLoadBalancerFinalizer false Alpha 1.15 1.15
ServiceLoadBalancerFinalizer true Beta 1.16 1.16
ServiceLoadBalancerFinalizer true GA 1.17 1.20
ServiceNodeExclusion false Alpha 1.8 1.18
ServiceNodeExclusion true Beta 1.19 1.20
ServiceNodeExclusion true GA 1.21 1.22
ServiceTopology false Alpha 1.17 1.19
ServiceTopology false Deprecated 1.20 1.22
SetHostnameAsFQDN false Alpha 1.19 1.19
SetHostnameAsFQDN true Beta 1.20 1.21
SetHostnameAsFQDN true GA 1.22 1.24
StartupProbe false Alpha 1.16 1.17
StartupProbe true Beta 1.18 1.19
StartupProbe true GA 1.20 1.23
StatefulSetMinReadySeconds false Alpha 1.22 1.22
StatefulSetMinReadySeconds true Beta 1.23 1.24
StatefulSetMinReadySeconds true GA 1.25 1.26
StorageObjectInUseProtection true Beta 1.10 1.10
StorageObjectInUseProtection true GA 1.11 1.24
StreamingProxyRedirects false Beta 1.5 1.5
StreamingProxyRedirects true Beta 1.6 1.17
StreamingProxyRedirects true Deprecated 1.18 1.21
StreamingProxyRedirects false Deprecated 1.22 1.24
SupportIPVSProxyMode false Alpha 1.8 1.8
SupportIPVSProxyMode false Beta 1.9 1.9
SupportIPVSProxyMode true Beta 1.10 1.10
SupportIPVSProxyMode true GA 1.11 1.20
SupportNodePidsLimit false Alpha 1.14 1.14
SupportNodePidsLimit true Beta 1.15 1.19
SupportNodePidsLimit true GA 1.20 1.23
SupportPodPidsLimit false Alpha 1.10 1.13
SupportPodPidsLimit true Beta 1.14 1.19
SupportPodPidsLimit true GA 1.20 1.23
SuspendJob false Alpha 1.21 1.21
SuspendJob true Beta 1.22 1.23
SuspendJob true GA 1.24 1.25
Sysctls true Beta 1.11 1.20
Sysctls true GA 1.21 1.22
TaintBasedEvictions false Alpha 1.6 1.12
TaintBasedEvictions true Beta 1.13 1.17
TaintBasedEvictions true GA 1.18 1.20
TaintNodesByCondition false Alpha 1.8 1.11
TaintNodesByCondition true Beta 1.12 1.16
TaintNodesByCondition true GA 1.17 1.18
TokenRequest false Alpha 1.10 1.11
TokenRequest true Beta 1.12 1.19
TokenRequest true GA 1.20 1.21
TokenRequestProjection false Alpha 1.11 1.11
TokenRequestProjection true Beta 1.12 1.19
TokenRequestProjection true GA 1.20 1.21
TopologyManager false Alpha 1.16 1.17
TopologyManager true Beta 1.18 1.26
TopologyManager true GA 1.27 1.28
TTLAfterFinished false Alpha 1.12 1.20
TTLAfterFinished true Beta 1.21 1.22
TTLAfterFinished true GA 1.23 1.24
UserNamespacesStatelessPodsSupport false Alpha 1.25 1.27
ValidateProxyRedirects false Alpha 1.12 1.13
ValidateProxyRedirects true Beta 1.14 1.21
ValidateProxyRedirects true Deprecated 1.22 1.24
VolumePVCDataSource false Alpha 1.15 1.15
VolumePVCDataSource true Beta 1.16 1.17
VolumePVCDataSource true GA 1.18 1.21
VolumeScheduling false Alpha 1.9 1.9
VolumeScheduling true Beta 1.10 1.12
VolumeScheduling true GA 1.13 1.16
VolumeSnapshotDataSource false Alpha 1.12 1.16
VolumeSnapshotDataSource true Beta 1.17 1.19
VolumeSnapshotDataSource true GA 1.20 1.22
VolumeSubpath true GA 1.10 1.24
VolumeSubpathEnvExpansion false Alpha 1.14 1.14
VolumeSubpathEnvExpansion true Beta 1.15 1.16
VolumeSubpathEnvExpansion true GA 1.17 1.24
WarningHeaders true Beta 1.19 1.21
WarningHeaders true GA 1.22 1.24
WindowsEndpointSliceProxying false Alpha 1.19 1.20
WindowsEndpointSliceProxying true Beta 1.21 1.21
WindowsEndpointSliceProxying true GA 1.22 1.24
WindowsGMSA false Alpha 1.14 1.15
WindowsGMSA true Beta 1.16 1.17
WindowsGMSA true GA 1.18 1.20
WindowsHostProcessContainers false Alpha 1.22 1.22
WindowsHostProcessContainers true Beta 1.23 1.25
WindowsHostProcessContainers true GA 1.26 1.27
WindowsRunAsUserName false Alpha 1.16 1.16
WindowsRunAsUserName true Beta 1.17 1.17
WindowsRunAsUserName true GA 1.18 1.20

Descriptions for removed feature gates

  • Accelerators: Provided an early form of plugin to enable Nvidia GPU support when using Docker Engine; no longer available. See Device Plugins for an alternative.

  • AffinityInAnnotations: Enable setting Pod affinity or anti-affinity.

  • AdvancedAuditing: Enable advanced auditing

  • AllowExtTrafficLocalEndpoints: Enable a service to route external requests to node local endpoints.

  • AllowInsecureBackendProxy: Enable the users to skip TLS verification of kubelets on Pod log requests.

  • AttachVolumeLimit: Enable volume plugins to report limits on number of volumes that can be attached to a node. See dynamic volume limits for more details.

  • BalanceAttachedNodeVolumes: Include volume count on node to be considered for balanced resource allocation while scheduling. A node which has closer CPU, memory utilization, and volume count is favored by the scheduler while making decisions.

  • BlockVolume: Enable the definition and consumption of raw block devices in Pods. See Raw Block Volume Support for more details.

  • BoundServiceAccountTokenVolume:

    Migrate ServiceAccount volumes to use a projected volume consisting of a ServiceAccountTokenVolumeProjection. Cluster admins can use metric serviceaccount_stale_tokens_total to monitor workloads that are depending on the extended tokens. If there are no such workloads, turn off extended tokens by starting kube-apiserver with flag --service-account-extend-token-expiration=false.

    Check Bound Service Account Tokens for more details.

  • CRIContainerLogRotation: Enable container log rotation for CRI container runtime. The default max size of a log file is 10MB and the default max number of log files allowed for a container is 5. These values can be configured in the kubelet config. See logging at node level for more details.

  • CSIBlockVolume: Enable external CSI volume drivers to support block storage. See csi raw block volume support for more details.

  • CSIDriverRegistry: Enable all logic related to the CSIDriver API object in csi.storage.k8s.io.

  • CSIInlineVolume: Enable CSI Inline volumes support for pods.

  • CSIMigration: Enables shims and translation logic to route volume operations from in-tree plugins to corresponding pre-installed CSI plugins

  • CSIMigrationAWS: Enables shims and translation logic to route volume operations from the AWS-EBS in-tree plugin to EBS CSI plugin. Supports falling back to in-tree EBS plugin for mount operations to nodes that have the feature disabled or that do not have EBS CSI plugin installed and configured. Does not support falling back for provision operations, for those the CSI plugin must be installed and configured.

  • CSIMigrationAWSComplete: Stops registering the EBS in-tree plugin in kubelet and volume controllers and enables shims and translation logic to route volume operations from the AWS-EBS in-tree plugin to EBS CSI plugin. Requires CSIMigration and CSIMigrationAWS feature flags enabled and EBS CSI plugin installed and configured on all nodes in the cluster. This flag has been deprecated in favor of the InTreePluginAWSUnregister feature flag which prevents the registration of in-tree EBS plugin.

  • CSIMigrationAzureDisk: Enables shims and translation logic to route volume operations from the Azure-Disk in-tree plugin to AzureDisk CSI plugin. Supports falling back to in-tree AzureDisk plugin for mount operations to nodes that have the feature disabled or that do not have AzureDisk CSI plugin installed and configured. Does not support falling back for provision operations, for those the CSI plugin must be installed and configured. Requires CSIMigration feature flag enabled.

  • CSIMigrationAzureDiskComplete: Stops registering the Azure-Disk in-tree plugin in kubelet and volume controllers and enables shims and translation logic to route volume operations from the Azure-Disk in-tree plugin to AzureDisk CSI plugin. Requires CSIMigration and CSIMigrationAzureDisk feature flags enabled and AzureDisk CSI plugin installed and configured on all nodes in the cluster. This flag has been deprecated in favor of the InTreePluginAzureDiskUnregister feature flag which prevents the registration of in-tree AzureDisk plugin.

  • CSIMigrationAzureFileComplete: Stops registering the Azure-File in-tree plugin in kubelet and volume controllers and enables shims and translation logic to route volume operations from the Azure-File in-tree plugin to AzureFile CSI plugin. Requires CSIMigration and CSIMigrationAzureFile feature flags enabled and AzureFile CSI plugin installed and configured on all nodes in the cluster. This flag has been deprecated in favor of the InTreePluginAzureFileUnregister feature flag which prevents the registration of in-tree AzureFile plugin.

  • CSIMigrationGCE: Enables shims and translation logic to route volume operations from the GCE-PD in-tree plugin to PD CSI plugin. Supports falling back to in-tree GCE plugin for mount operations to nodes that have the feature disabled or that do not have PD CSI plugin installed and configured. Does not support falling back for provision operations, for those the CSI plugin must be installed and configured. Requires CSIMigration feature flag enabled.

  • CSIMigrationGCEComplete: Stops registering the GCE-PD in-tree plugin in kubelet and volume controllers and enables shims and translation logic to route volume operations from the GCE-PD in-tree plugin to PD CSI plugin. Requires CSIMigration and CSIMigrationGCE feature flags enabled and PD CSI plugin installed and configured on all nodes in the cluster. This flag has been deprecated in favor of the InTreePluginGCEUnregister feature flag which prevents the registration of in-tree GCE PD plugin.

  • CSIMigrationOpenStack: Enables shims and translation logic to route volume operations from the Cinder in-tree plugin to Cinder CSI plugin. Supports falling back to in-tree Cinder plugin for mount operations to nodes that have the feature disabled or that do not have Cinder CSI plugin installed and configured. Does not support falling back for provision operations, for those the CSI plugin must be installed and configured. Requires CSIMigration feature flag enabled.

  • CSIMigrationOpenStackComplete: Stops registering the Cinder in-tree plugin in kubelet and volume controllers and enables shims and translation logic to route volume operations from the Cinder in-tree plugin to Cinder CSI plugin. Requires CSIMigration and CSIMigrationOpenStack feature flags enabled and Cinder CSI plugin installed and configured on all nodes in the cluster. This flag has been deprecated in favor of the InTreePluginOpenStackUnregister feature flag which prevents the registration of in-tree openstack cinder plugin.

  • CSIMigrationvSphereComplete: Stops registering the vSphere in-tree plugin in kubelet and volume controllers and enables shims and translation logic to route volume operations from the vSphere in-tree plugin to vSphere CSI plugin. Requires CSIMigration and CSIMigrationvSphere feature flags enabled and vSphere CSI plugin installed and configured on all nodes in the cluster. This flag has been deprecated in favor of the InTreePluginvSphereUnregister feature flag which prevents the registration of in-tree vsphere plugin.

  • CSIMigrationvSphere: Enables shims and translation logic to route volume operations from the vSphere in-tree plugin to vSphere CSI plugin. Supports falling back to in-tree vSphere plugin for mount operations to nodes that have the feature disabled or that do not have vSphere CSI plugin installed and configured. Does not support falling back for provision operations, for those the CSI plugin must be installed and configured. Requires CSIMigration feature flag enabled.

  • CSINodeInfo: Enable all logic related to the CSINodeInfo API object in csi.storage.k8s.io.

  • CSIPersistentVolume: Enable discovering and mounting volumes provisioned through a CSI (Container Storage Interface) compatible volume plugin.

  • CSIServiceAccountToken: Enable CSI drivers to receive the pods' service account token that they mount volumes for. See Token Requests.

  • CSIStorageCapacity: Enables CSI drivers to publish storage capacity information and the Kubernetes scheduler to use that information when scheduling pods. See Storage Capacity. Check the csi volume type documentation for more details.

  • CSIVolumeFSGroupPolicy: Allows CSIDrivers to use the fsGroupPolicy field. This field controls whether volumes created by a CSIDriver support volume ownership and permission modifications when these volumes are mounted.

  • CSRDuration: Allows clients to request a duration for certificates issued via the Kubernetes CSR API.

  • ConfigurableFSGroupPolicy: Allows user to configure volume permission change policy for fsGroups when mounting a volume in a Pod. See Configure volume permission and ownership change policy for Pods for more details.

  • ControllerManagerLeaderMigration: Enables Leader Migration for kube-controller-manager and cloud-controller-manager which allows a cluster operator to live migrate controllers from the kube-controller-manager into an external controller-manager (e.g. the cloud-controller-manager) in an HA cluster without downtime.

  • CronJobControllerV2: Use an alternative implementation of the CronJob controller. Otherwise, version 1 of the same controller is selected.

  • CronJobTimeZone: Allow the use of the timeZone optional field in CronJobs

  • CustomPodDNS: Enable customizing the DNS settings for a Pod using its dnsConfig property. Check Pod's DNS Config for more details.

  • CustomResourceDefaulting: Enable CRD support for default values in OpenAPI v3 validation schemas.

  • CustomResourcePublishOpenAPI: Enables publishing of CRD OpenAPI specs.

  • CustomResourceSubresources: Enable /status and /scale subresources on resources created from CustomResourceDefinition.

  • CustomResourceValidation: Enable schema based validation on resources created from CustomResourceDefinition.

  • CustomResourceWebhookConversion: Enable webhook-based conversion on resources created from CustomResourceDefinition.

  • DaemonSetUpdateSurge: Enables the DaemonSet workloads to maintain availability during update per node. See Perform a Rolling Update on a DaemonSet.

  • DefaultPodTopologySpread: Enables the use of PodTopologySpread scheduling plugin to do default spreading.

  • DelegateFSGroupToCSIDriver: If supported by the CSI driver, delegates the role of applying fsGroup from a Pod's securityContext to the driver by passing fsGroup through the NodeStageVolume and NodePublishVolume CSI calls.

  • DevicePlugins: Enable the device-plugins based resource provisioning on nodes.

  • DisableAcceleratorUsageMetrics: Disable accelerator metrics collected by the kubelet.

  • DownwardAPIHugePages: Enables usage of hugepages in downward API.

  • DryRun: Enable server-side dry run requests so that validation, merging, and mutation can be tested without committing.

  • DynamicAuditing: Used to enable dynamic auditing before v1.19.

  • DynamicKubeletConfig: Enable the dynamic configuration of kubelet. The feature is no longer supported outside of supported skew policy. The feature gate was removed from kubelet in 1.24.

  • DynamicProvisioningScheduling: Extend the default scheduler to be aware of volume topology and handle PV provisioning. This feature was superseded by the VolumeScheduling feature in v1.12.

  • DynamicVolumeProvisioning: Enable the dynamic provisioning of persistent volumes to Pods.

  • EnableAggregatedDiscoveryTimeout: Enable the five second timeout on aggregated discovery calls.

  • EnableEquivalenceClassCache: Enable the scheduler to cache equivalence of nodes when scheduling Pods.

  • EndpointSlice: Enables EndpointSlices for more scalable and extensible network endpoints. See Enabling EndpointSlices.

  • EndpointSliceNodeName: Enables EndpointSlice nodeName field.

  • EndpointSliceProxying: When enabled, kube-proxy running on Linux will use EndpointSlices as the primary data source instead of Endpoints, enabling scalability and performance improvements. See Enabling Endpoint Slices.

  • EndpointSliceTerminatingCondition: Enables EndpointSlice terminating and serving condition fields.

  • EphemeralContainers: Enable the ability to add ephemeral containers to running Pods.

  • EvenPodsSpread: Enable pods to be scheduled evenly across topology domains. See Pod Topology Spread Constraints.

  • ExpandCSIVolumes: Enable the expanding of CSI volumes.

  • ExpandInUsePersistentVolumes: Enable expanding in-use PVCs. See Resizing an in-use PersistentVolumeClaim.

  • ExpandPersistentVolumes: Enable the expanding of persistent volumes. See Expanding Persistent Volumes Claims.

  • ExperimentalCriticalPodAnnotation: Enable annotating specific pods as critical so that their scheduling is guaranteed. This feature is deprecated by Pod Priority and Preemption as of v1.13.

  • ExternalPolicyForExternalIP: Fix a bug where ExternalTrafficPolicy is not applied to Service ExternalIPs.

  • GCERegionalPersistentDisk: Enable the regional PD feature on GCE.

  • GRPCContainerProbe: Enables the gRPC probe method for {Liveness,Readiness,Startup}Probe. See Configure Liveness, Readiness and Startup Probes.

  • GenericEphemeralVolume: Enables ephemeral, inline volumes that support all features of normal volumes (can be provided by third-party storage vendors, storage capacity tracking, restore from snapshot, etc.). See Ephemeral Volumes.

  • HugePageStorageMediumSize: Enable support for multiple sizes pre-allocated huge pages.

  • HugePages: Enable the allocation and consumption of pre-allocated huge pages.

  • HyperVContainer: Enable Hyper-V isolation for Windows containers.

  • IPv6DualStack: Enable dual stack support for IPv6.

  • IdentifyPodOS: Allows the Pod OS field to be specified. This helps in identifying the OS of the pod authoritatively during the API server admission time. In Kubernetes 1.29, the allowed values for the pod.spec.os.name are windows and linux.

  • ImmutableEphemeralVolumes: Allows for marking individual Secrets and ConfigMaps as immutable for better safety and performance.

  • IndexedJob: Allows the Job controller to manage Pod completions per completion index.

  • IngressClassNamespacedParams: Allow namespace-scoped parameters reference in IngressClass resource. This feature adds two fields - Scope and Namespace to IngressClass.spec.parameters.

  • Initializers: Allow asynchronous coordination of object creation using the Initializers admission plugin.

  • JobMutableNodeSchedulingDirectives: Allows updating node scheduling directives in the pod template of Job.

  • JobTrackingWithFinalizers: Enables tracking Job completions without relying on Pods remaining in the cluster indefinitely. The Job controller uses Pod finalizers and a field in the Job status to keep track of the finished Pods to count towards completion.

  • KubeletConfigFile: Enable loading kubelet configuration from a file specified using a config file. See setting kubelet parameters via a config file for more details.

  • KubeletCredentialProviders: Enable kubelet exec credential providers for image pull credentials.

  • KubeletPluginsWatcher: Enable probe-based plugin watcher utility to enable kubelet to discover plugins such as CSI volume drivers.

  • LegacyNodeRoleBehavior: When disabled, legacy behavior in service load balancers and node disruption will ignore the node-role.kubernetes.io/master label in favor of the feature-specific labels provided by NodeDisruptionExclusion and ServiceNodeExclusion.

  • LegacyServiceAccountTokenNoAutoGeneration: Stop auto-generation of Secret-based service account tokens.

  • LocalStorageCapacityIsolation: Enable the consumption of local ephemeral storage and also the sizeLimit property of an emptyDir volume.

  • MixedProtocolLBService: Enable using different protocols in the same LoadBalancer type Service instance.

  • MountContainers: Enable using utility containers on host as the volume mounter.

  • MountPropagation: Enable sharing volume mounted by one container to other containers or pods. For more details, please see mount propagation.

  • MultiCIDRRangeAllocator: Enables the MultiCIDR range allocator.

  • NamespaceDefaultLabelName: Configure the API Server to set an immutable label kubernetes.io/metadata.name on all namespaces, containing the namespace name.

  • NetworkPolicyStatus: Enable the status subresource for NetworkPolicy objects.

  • NodeDisruptionExclusion: Enable use of the Node label node.kubernetes.io/exclude-disruption which prevents nodes from being evacuated during zone failures.

  • NodeLease: Enable the new Lease API to report node heartbeats, which could be used as a node health signal.

  • NonPreemptingPriority: Enable preemptionPolicy field for PriorityClass and Pod.

  • OpenAPIV3: Enables the API server to publish OpenAPI v3.

  • PVCProtection: Enable the prevention of a PersistentVolumeClaim (PVC) from being deleted when it is still used by any Pod.

  • PersistentLocalVolumes: Enable the usage of local volume type in Pods. Pod affinity has to be specified if requesting a local volume.

  • PodAffinityNamespaceSelector: Enable the Pod Affinity Namespace Selector and CrossNamespacePodAffinity quota scope features.

  • PodDisruptionBudget: Enable the PodDisruptionBudget feature.

  • PodHasNetworkCondition: Enable the kubelet to mark the PodHasNetwork condition on pods. This was renamed to PodReadyToStartContainersCondition in 1.28.

  • PodOverhead: Enable the PodOverhead feature to account for pod overheads.

  • PodPriority: Enable the descheduling and preemption of Pods based on their priorities.

  • PodReadinessGates: Enable the setting of PodReadinessGate field for extending Pod readiness evaluation. See Pod readiness gate for more details.

  • PodSecurity: Enables the PodSecurity admission plugin.

  • PodShareProcessNamespace: Enable the setting of shareProcessNamespace in a Pod for sharing a single process namespace between containers running in a pod. More details can be found in Share Process Namespace between Containers in a Pod.

  • PreferNominatedNode: This flag tells the scheduler whether the nominated nodes will be checked first before looping through all the other nodes in the cluster.

  • ProbeTerminationGracePeriod: Enable setting probe-level terminationGracePeriodSeconds on pods. See the enhancement proposal for more details.

  • RequestManagement: Enables managing request concurrency with prioritization and fairness at each API server. Deprecated by APIPriorityAndFairness since 1.17.

  • ResourceLimitsPriorityFunction: Enable a scheduler priority function that assigns a lowest possible score of 1 to a node that satisfies at least one of the input Pod's cpu and memory limits. The intent is to break ties between nodes with same scores.

  • ResourceQuotaScopeSelectors: Enable resource quota scope selectors.

  • RetroactiveDefaultStorageClass: Allow assigning StorageClass to unbound PVCs retroactively.

  • RootCAConfigMap: Configure the kube-controller-manager to publish a ConfigMap named kube-root-ca.crt to every namespace. This ConfigMap contains a CA bundle used for verifying connections to the kube-apiserver. See Bound Service Account Tokens for more details.

  • RotateKubeletClientCertificate: Enable the rotation of the client TLS certificate on the kubelet. See kubelet configuration for more details.

  • RunAsGroup: Enable control over the primary group ID set on the init processes of containers.

  • RuntimeClass: Enable the RuntimeClass feature for selecting container runtime configurations.

  • SCTPSupport: Enables the SCTP protocol value in Pod, Service, Endpoints, EndpointSlice, and NetworkPolicy definitions.

  • ScheduleDaemonSetPods: Enable DaemonSet Pods to be scheduled by the default scheduler instead of the DaemonSet controller.

  • SeccompDefault: Enables the use of RuntimeDefault as the default seccomp profile for all workloads. The seccomp profile is specified in the securityContext of a Pod and/or a Container.

  • SelectorIndex: Allows label and field based indexes in API server watch cache to accelerate list operations.

  • ServiceAccountIssuerDiscovery: Enable OIDC discovery endpoints (issuer and JWKS URLs) for the service account issuer in the API server. See Configure Service Accounts for Pods for more details.

  • ServiceAppProtocol: Enables the appProtocol field on Services and Endpoints.

  • ServiceIPStaticSubrange: Enables a strategy for Services ClusterIP allocations, whereby the ClusterIP range is subdivided. Dynamic allocated ClusterIP addresses will be allocated preferently from the upper range allowing users to assign static ClusterIPs from the lower range with a low risk of collision. See Avoiding collisions for more details.

  • ServiceInternalTrafficPolicy: Enables the internalTrafficPolicy field on Services

  • ServiceLoadBalancerClass: Enables the loadBalancerClass field on Services. See Specifying class of load balancer implementation for more details.

  • ServiceLoadBalancerFinalizer: Enable finalizer protection for Service load balancers.

  • ServiceLBNodePortControl: Enables the allocateLoadBalancerNodePorts field on Services.

  • ServiceNodeExclusion: Enable the exclusion of nodes from load balancers created by a cloud provider. A node is eligible for exclusion if labelled with "node.kubernetes.io/exclude-from-external-load-balancers".

  • ServiceTopology: Enable service to route traffic based upon the Node topology of the cluster.

  • SetHostnameAsFQDN: Enable the ability of setting Fully Qualified Domain Name(FQDN) as the hostname of a pod. See Pod's setHostnameAsFQDN field.

  • StartupProbe: Enable the startup probe in the kubelet.

  • StatefulSetMinReadySeconds: Allows minReadySeconds to be respected by the StatefulSet controller.

  • StorageObjectInUseProtection: Postpone the deletion of PersistentVolume or PersistentVolumeClaim objects if they are still being used.

  • StreamingProxyRedirects: Instructs the API server to intercept (and follow) redirects from the backend (kubelet) for streaming requests. Examples of streaming requests include the exec, attach and port-forward requests.

  • SupportIPVSProxyMode: Enable providing in-cluster service load balancing using IPVS. See service proxies for more details.

  • SupportNodePidsLimit: Enable the support to limiting PIDs on the Node. The parameter pid=<number> in the --system-reserved and --kube-reserved options can be specified to ensure that the specified number of process IDs will be reserved for the system as a whole and for Kubernetes system daemons respectively.

  • SupportPodPidsLimit: Enable the support to limiting PIDs in Pods.

  • SuspendJob: Enable support to suspend and resume Jobs. For more details, see the Jobs docs.

  • Sysctls: Enable support for namespaced kernel parameters (sysctls) that can be set for each pod. See sysctls for more details.

  • TTLAfterFinished: Allow a TTL controller to clean up resources after they finish execution.

  • TaintBasedEvictions: Enable evicting pods from nodes based on taints on Nodes and tolerations on Pods. See taints and tolerations for more details.

  • TaintNodesByCondition: Enable automatic tainting nodes based on node conditions.

  • TokenRequest: Enable the TokenRequest endpoint on service account resources.

  • TokenRequestProjection: Enable the injection of service account tokens into a Pod through a projected volume.

  • TopologyManager: Enable a mechanism to coordinate fine-grained hardware resource assignments for different components in Kubernetes. See Control Topology Management Policies on a node.

  • UserNamespacesStatelessPodsSupport: Enable user namespace support for stateless Pods. This feature gate was superseded by the UserNamespacesSupport feature gate in the Kubernetes v1.28 release.

  • ValidateProxyRedirects: This flag controls whether the API server should validate that redirects are only followed to the same host. Only used if the StreamingProxyRedirects flag is enabled.

  • VolumePVCDataSource: Enable support for specifying an existing PVC as a DataSource.

  • VolumeScheduling: Enable volume topology aware scheduling and make the PersistentVolumeClaim (PVC) binding aware of scheduling decisions. It also enables the usage of local volume type when used together with the PersistentLocalVolumes feature gate.

  • VolumeSnapshotDataSource: Enable volume snapshot data source support.

  • VolumeSubpath: Allow mounting a subpath of a volume in a container.

  • VolumeSubpathEnvExpansion: Enable subPathExpr field for expanding environment variables into a subPath.

  • WarningHeaders: Allow sending warning headers in API responses.

  • WindowsEndpointSliceProxying: When enabled, kube-proxy running on Windows will use EndpointSlices as the primary data source instead of Endpoints, enabling scalability and performance improvements. See Enabling Endpoint Slices.

  • WindowsGMSA: Enables passing of GMSA credential specs from pods to container runtimes.

  • WindowsHostProcessContainers: Enables support for Windows HostProcess containers.

  • WindowsRunAsUserName: Enable support for running applications in Windows containers with as a non-default user. See Configuring RunAsUserName for more details.

3 - kubelet

Synopsis

The kubelet is the primary "node agent" that runs on each node. It can register the node with the apiserver using one of: the hostname; a flag to override the hostname; or specific logic for a cloud provider.

The kubelet works in terms of a PodSpec. A PodSpec is a YAML or JSON object that describes a pod. The kubelet takes a set of PodSpecs that are provided through various mechanisms (primarily through the apiserver) and ensures that the containers described in those PodSpecs are running and healthy. The kubelet doesn't manage containers which were not created by Kubernetes.

Other than from a PodSpec from the apiserver, there are two ways that a container manifest can be provided to the kubelet.

  • File: Path passed as a flag on the command line. Files under this path will be monitored periodically for updates. The monitoring period is 20s by default and is configurable via a flag.
  • HTTP endpoint: HTTP endpoint passed as a parameter on the command line. This endpoint is checked every 20 seconds (also configurable with a flag).
kubelet [flags]

Options

--address string     Default: 0.0.0.0
The IP address for the kubelet to serve on (set to 0.0.0.0 or :: for listening on all interfaces and IP address families) (DEPRECATED: This parameter should be set via the config file specified by the kubelet's --config flag. See kubelet-config-file for more information.)
--allowed-unsafe-sysctls strings
Comma-separated whitelist of unsafe sysctls or unsafe sysctl patterns (ending in *). Use these at your own risk. (DEPRECATED: This parameter should be set via the config file specified by the kubelet's --config flag. See kubelet-config-file for more information.)
--anonymous-auth     Default: true
Enables anonymous requests to the kubelet server. Requests that are not rejected by another authentication method are treated as anonymous requests. Anonymous requests have a username of system:anonymous, and a group name of system:unauthenticated. (DEPRECATED: This parameter should be set via the config file specified by the kubelet's --config flag. See kubelet-config-file for more information.)
--authentication-token-webhook
Use the TokenReview API to determine authentication for bearer tokens. (DEPRECATED: This parameter should be set via the config file specified by the kubelet's --config flag. See kubelet-config-file for more information.)
--authentication-token-webhook-cache-ttl duration     Default: 2m0s
The duration to cache responses from the webhook token authenticator. (DEPRECATED: This parameter should be set via the config file specified by the kubelet's --config flag. See kubelet-config-file for more information.)
--authorization-mode string     Default: AlwaysAllow
Authorization mode for kubelet server. Valid options are "AlwaysAllow" or "Webhook". Webhook mode uses the SubjectAccessReview API to determine authorization. (DEPRECATED: This parameter should be set via the config file specified by the kubelet's --config flag. See kubelet-config-file for more information.)
--authorization-webhook-cache-authorized-ttl duration     Default: 5m0s
The duration to cache 'authorized' responses from the webhook authorizer. (DEPRECATED: This parameter should be set via the config file specified by the kubelet's --config flag. See kubelet-config-file for more information.)
--authorization-webhook-cache-unauthorized-ttl duration     Default: 30s
The duration to cache 'unauthorized' responses from the webhook authorizer. (DEPRECATED: This parameter should be set via the config file specified by the kubelet's --config flag. See kubelet-config-file for more information.)
--bootstrap-kubeconfig string
Path to a kubeconfig file that will be used to get client certificate for kubelet. If the file specified by --kubeconfig does not exist, the bootstrap kubeconfig is used to request a client certificate from the API server. On success, a kubeconfig file referencing the generated client certificate and key is written to the path specified by --kubeconfig. The client certificate and key file will be stored in the directory pointed by --cert-dir.
--cert-dir string     Default: /var/lib/kubelet/pki
The directory where the TLS certs are located. If --tls-cert-file and --tls-private-key-file are provided, this flag will be ignored.
--cgroup-driver string     Default: cgroupfs
Driver that the kubelet uses to manipulate cgroups on the host. Possible values: "cgroupfs", "systemd". (DEPRECATED: This parameter should be set via the config file specified by the kubelet's --config flag. See kubelet-config-file for more information.)
--cgroup-root string     Default: ''
Optional root cgroup to use for pods. This is handled by the container runtime on a best effort basis. Default: '', which means use the container runtime default. (DEPRECATED: This parameter should be set via the config file specified by the kubelet's --config flag. See kubelet-config-file for more information.)
--cgroups-per-qos     Default: true
Enable creation of QoS cgroup hierarchy, if true, top level QoS and pod cgroups are created. (DEPRECATED: This parameter should be set via the config file specified by the kubelet's --config flag. See kubelet-config-file for more information.)
--client-ca-file string
If set, any request presenting a client certificate signed by one of the authorities in the client-ca-file is authenticated with an identity corresponding to the CommonName of the client certificate. (DEPRECATED: This parameter should be set via the config file specified by the kubelet's --config flag. See kubelet-config-file for more information.)
--cloud-config string
The path to the cloud provider configuration file. Empty string for no configuration file. (DEPRECATED: will be removed in 1.25 or later, in favor of removing cloud providers code from kubelet.)
--cloud-provider string
The provider for cloud services. Set to empty string for running with no cloud provider. Set to 'external' for running with an external cloud provider. If set, the cloud provider determines the name of the node (consult cloud provider documentation to determine if and how the hostname is used).
--cluster-dns strings
Comma-separated list of DNS server IP address. This value is used for containers DNS server in case of Pods with "dnsPolicy: ClusterFirst".
Note: all DNS servers appearing in the list MUST serve the same set of records otherwise name resolution within the cluster may not work correctly. There is no guarantee as to which DNS server may be contacted for name resolution. (DEPRECATED: This parameter should be set via the config file specified by the kubelet's --config flag. See kubelet-config-file for more information.)
--cluster-domain string
Domain for this cluster. If set, kubelet will configure all containers to search this domain in addition to the host's search domains. (DEPRECATED: This parameter should be set via the config file specified by the kubelet's --config flag. See kubelet-config-file for more information.)
--config string
The kubelet will load its initial configuration from this file. The path may be absolute or relative; relative paths start at the kubelet's current working directory. Omit this flag to use the built-in default configuration values. Command-line flags override configuration from this file.
--config-dir string     Default: ''
Path to a directory to specify drop-ins, allows the user to optionally specify additional configs to overwrite what is provided by default and in the `--config` flag.
Note: Set the 'KUBELET_CONFIG_DROPIN_DIR_ALPHA' environment variable to specify the directory.
--container-log-max-files int32     Default: 5
<Warning: Beta feature> Set the maximum number of container log files that can be present for a container. The number must be >= 2. (DEPRECATED: This parameter should be set via the config file specified by the kubelet's --config flag. See kubelet-config-file for more information.)
--container-log-max-size string     Default: 10Mi
<Warning: Beta feature> Set the maximum size (e.g. 10Mi) of container log file before it is rotated. (DEPRECATED: This parameter should be set via the config file specified by the kubelet's --config flag. See kubelet-config-file for more information.)
--container-runtime-endpoint string     Default: "unix:///run/containerd/containerd.sock"
The endpoint of remote runtime service. UNIX domain sockets are supported on Linux, while 'npipe' and 'tcp' endpoints are supported on windows. Examples: 'unix:///path/to/runtime.sock', 'npipe:////./pipe/runtime'. (DEPRECATED: This parameter should be set via the config file specified by the kubelet's --config flag. See kubelet-config-file for more information.)
--contention-profiling
Enable block profiling, if profiling is enabled. (DEPRECATED: This parameter should be set via the config file specified by the kubelet's --config flag. See kubelet-config-file for more information.)
--cpu-cfs-quota     Default: true
Enable CPU CFS quota enforcement for containers that specify CPU limits. (DEPRECATED: This parameter should be set via the config file specified by the kubelet's --config flag. See kubelet-config-file for more information.)
--cpu-cfs-quota-period duration     Default: 100ms
Sets CPU CFS quota period value, cpu.cfs_period_us, defaults to Linux Kernel default. (DEPRECATED: This parameter should be set via the config file specified by the kubelet's --config flag. See kubelet-config-file for more information.)
--cpu-manager-policy string     Default: none
The CPU manager policy to use. Possible values: "none", "static". (DEPRECATED: This parameter should be set via the config file specified by the kubelet's --config flag. See kubelet-config-file for more information.)
--cpu-manager-policy-options string
A set of 'key=value' CPU manager policy options to use, to fine tune their behaviour. If not supplied, keep the default behaviour. (DEPRECATED: This parameter should be set via the config file specified by the kubelet's --config flag. See kubelet-config-file for more information.)
--cpu-manager-reconcile-period duration     Default: 10s
<Warning: Alpha feature> CPU manager reconciliation period. Examples: "10s", or "1m". If not supplied, defaults to node status update frequency. (DEPRECATED: This parameter should be set via the config file specified by the kubelet's --config flag. See kubelet-config-file for more information.)
--enable-controller-attach-detach     Default: true
Enables the Attach/Detach controller to manage attachment/detachment of volumes scheduled to this node, and disables kubelet from executing any attach/detach operations. (DEPRECATED: This parameter should be set via the config file specified by the kubelet's --config flag. See kubelet-config-file for more information.)
--enable-debugging-handlers     Default: true
Enables server endpoints for log collection and local running of containers and commands. (DEPRECATED: This parameter should be set via the config file specified by the kubelet's --config flag. See kubelet-config-file for more information.)
--enable-server     Default: true
Enable the kubelet's server. (DEPRECATED: This parameter should be set via the config file specified by the kubelet's --config flag. See kubelet-config-file for more information.)
--enforce-node-allocatable strings     Default: pods
A comma separated list of levels of node allocatable enforcement to be enforced by kubelet. Acceptable options are "none", "pods", "system-reserved", and "kube-reserved". If the latter two options are specified, --system-reserved-cgroup and --kube-reserved-cgroup must also be set, respectively. If "none" is specified, no additional options should be set. See official documentation for more details. (DEPRECATED: This parameter should be set via the config file specified by the kubelet's --config flag. See kubelet-config-file for more information.)
--event-burst int32     Default: 100
Maximum size of a bursty event records, temporarily allows event records to burst to this number, while still not exceeding --event-qps. The number must be >= 0. If 0 will use default burst (100). (DEPRECATED: This parameter should be set via the config file specified by the kubelet's --config flag. See kubelet-config-file for more information.)
--event-qps int32     Default: 50
QPS to limit event creations. The number must be >= 0. If 0 will use default QPS (50). (DEPRECATED: This parameter should be set via the config file specified by the kubelet's --config flag. See kubelet-config-file for more information.)
--eviction-hard strings     Default: imagefs.available<15%,memory.available<100Mi,nodefs.available<10%
A set of eviction thresholds (e.g. "memory.available<1Gi") that if met would trigger a pod eviction. On a Linux node, the default value also includes "nodefs.inodesFree<5%". (DEPRECATED: This parameter should be set via the config file specified by the kubelet's --config flag. See kubelet-config-file for more information.)
--eviction-max-pod-grace-period int32
Maximum allowed grace period (in seconds) to use when terminating pods in response to a soft eviction threshold being met. If negative, defer to pod specified value. (DEPRECATED: This parameter should be set via the config file specified by the kubelet's --config flag. See kubelet-config-file for more information.)
--eviction-minimum-reclaim strings
A set of minimum reclaims (e.g. "imagefs.available=2Gi") that describes the minimum amount of resource the kubelet will reclaim when performing a pod eviction if that resource is under pressure. (DEPRECATED: This parameter should be set via the config file specified by the kubelet's --config flag. See kubelet-config-file for more information.)
--eviction-pressure-transition-period duration     Default: 5m0s
Duration for which the kubelet has to wait before transitioning out of an eviction pressure condition. (DEPRECATED: This parameter should be set via the config file specified by the kubelet's --config flag. See kubelet-config-file for more information.)
--eviction-soft strings
A set of eviction thresholds (e.g. "memory.available<1.5Gi") that if met over a corresponding grace period would trigger a pod eviction. (DEPRECATED: This parameter should be set via the config file specified by the kubelet's --config flag. See kubelet-config-file for more information.)
--eviction-soft-grace-period strings
A set of eviction grace periods (e.g. "memory.available=1m30s") that correspond to how long a soft eviction threshold must hold before triggering a pod eviction. (DEPRECATED: This parameter should be set via the config file specified by the kubelet's --config flag. See kubelet-config-file for more information.)
--exit-on-lock-contention
Whether kubelet should exit upon lock-file contention.
--experimental-allocatable-ignore-eviction     Default: false
When set to true, hard eviction thresholds will be ignored while calculating node allocatable. See here for more details. (DEPRECATED: will be removed in 1.25 or later)
--experimental-mounter-path string     Default: mount
[Experimental] Path of mounter binary. Leave empty to use the default mount. (DEPRECATED: will be removed in 1.24 or later, in favor of using CSI.)
--fail-swap-on     Default: true
Makes the kubelet fail to start if swap is enabled on the node. (DEPRECATED: This parameter should be set via the config file specified by the kubelet's --config flag. See kubelet-config-file for more information.)
--feature-gates <A list of 'key=true/false' pairs>
A set of key=value pairs that describe feature gates for alpha/experimental features. Options are:
APIResponseCompression=true|false (BETA - default=true)
APIServerIdentity=true|false (BETA - default=true)
APIServerTracing=true|false (BETA - default=true)
AdmissionWebhookMatchConditions=true|false (BETA - default=true)
AggregatedDiscoveryEndpoint=true|false (BETA - default=true)
AllAlpha=true|false (ALPHA - default=false)
AllBeta=true|false (BETA - default=false)
AnyVolumeDataSource=true|false (BETA - default=true)
AppArmor=true|false (BETA - default=true)
CPUManagerPolicyAlphaOptions=true|false (ALPHA - default=false)
CPUManagerPolicyBetaOptions=true|false (BETA - default=true)
CPUManagerPolicyOptions=true|false (BETA - default=true)
CRDValidationRatcheting=true|false (ALPHA - default=false)
CSIMigrationPortworx=true|false (BETA - default=false)
CSIVolumeHealth=true|false (ALPHA - default=false)
CloudControllerManagerWebhook=true|false (ALPHA - default=false)
CloudDualStackNodeIPs=true|false (BETA - default=true)
ClusterTrustBundle=true|false (ALPHA - default=false)
ClusterTrustBundleProjection=true|false (ALPHA - default=false)
ComponentSLIs=true|false (BETA - default=true)
ConsistentListFromCache=true|false (ALPHA - default=false)
ContainerCheckpoint=true|false (ALPHA - default=false)
ContextualLogging=true|false (ALPHA - default=false)
CronJobsScheduledAnnotation=true|false (BETA - default=true)
CrossNamespaceVolumeDataSource=true|false (ALPHA - default=false)
CustomCPUCFSQuotaPeriod=true|false (ALPHA - default=false)
DevicePluginCDIDevices=true|false (BETA - default=true)
DisableCloudProviders=true|false (BETA - default=true)
DisableKubeletCloudCredentialProviders=true|false (BETA - default=true)
DisableNodeKubeProxyVersion=true|false (ALPHA - default=false)
DynamicResourceAllocation=true|false (ALPHA - default=false)
ElasticIndexedJob=true|false (BETA - default=true)
EventedPLEG=true|false (BETA - default=false)
GracefulNodeShutdown=true|false (BETA - default=true)
GracefulNodeShutdownBasedOnPodPriority=true|false (BETA - default=true)
HPAContainerMetrics=true|false (BETA - default=true)
HPAScaleToZero=true|false (ALPHA - default=false)
HonorPVReclaimPolicy=true|false (ALPHA - default=false)
ImageMaximumGCAge=true|false (ALPHA - default=false)
InPlacePodVerticalScaling=true|false (ALPHA - default=false)
InTreePluginAWSUnregister=true|false (ALPHA - default=false)
InTreePluginAzureDiskUnregister=true|false (ALPHA - default=false)
InTreePluginAzureFileUnregister=true|false (ALPHA - default=false)
InTreePluginGCEUnregister=true|false (ALPHA - default=false)
InTreePluginOpenStackUnregister=true|false (ALPHA - default=false)
InTreePluginPortworxUnregister=true|false (ALPHA - default=false)
InTreePluginvSphereUnregister=true|false (ALPHA - default=false)
JobBackoffLimitPerIndex=true|false (BETA - default=true)
JobPodFailurePolicy=true|false (BETA - default=true)
JobPodReplacementPolicy=true|false (BETA - default=true)
KubeProxyDrainingTerminatingNodes=true|false (ALPHA - default=false)
KubeletCgroupDriverFromCRI=true|false (ALPHA - default=false)
KubeletInUserNamespace=true|false (ALPHA - default=false)
KubeletPodResourcesDynamicResources=true|false (ALPHA - default=false)
KubeletPodResourcesGet=true|false (ALPHA - default=false)
KubeletSeparateDiskGC=true|false (ALPHA - default=false)
KubeletTracing=true|false (BETA - default=true)
LegacyServiceAccountTokenCleanUp=true|false (BETA - default=true)
LoadBalancerIPMode=true|false (ALPHA - default=false)
LocalStorageCapacityIsolationFSQuotaMonitoring=true|false (ALPHA - default=false)
LogarithmicScaleDown=true|false (BETA - default=true)
LoggingAlphaOptions=true|false (ALPHA - default=false)
LoggingBetaOptions=true|false (BETA - default=true)
MatchLabelKeysInPodAffinity=true|false (ALPHA - default=false)
MatchLabelKeysInPodTopologySpread=true|false (BETA - default=true)
MaxUnavailableStatefulSet=true|false (ALPHA - default=false)
MemoryManager=true|false (BETA - default=true)
MemoryQoS=true|false (ALPHA - default=false)
MinDomainsInPodTopologySpread=true|false (BETA - default=true)
MultiCIDRServiceAllocator=true|false (ALPHA - default=false)
NFTablesProxyMode=true|false (ALPHA - default=false)
NewVolumeManagerReconstruction=true|false (BETA - default=true)
NodeInclusionPolicyInPodTopologySpread=true|false (BETA - default=true)
NodeLogQuery=true|false (ALPHA - default=false)
NodeSwap=true|false (BETA - default=false)
OpenAPIEnums=true|false (BETA - default=true)
PDBUnhealthyPodEvictionPolicy=true|false (BETA - default=true)
PersistentVolumeLastPhaseTransitionTime=true|false (BETA - default=true)
PodAndContainerStatsFromCRI=true|false (ALPHA - default=false)
PodDeletionCost=true|false (BETA - default=true)
PodDisruptionConditions=true|false (BETA - default=true)
PodHostIPs=true|false (BETA - default=true)
PodIndexLabel=true|false (BETA - default=true)
PodLifecycleSleepAction=true|false (ALPHA - default=false)
PodReadyToStartContainersCondition=true|false (BETA - default=true)
PodSchedulingReadiness=true|false (BETA - default=true)
ProcMountType=true|false (ALPHA - default=false)
QOSReserved=true|false (ALPHA - default=false)
RecoverVolumeExpansionFailure=true|false (ALPHA - default=false)
RotateKubeletServerCertificate=true|false (BETA - default=true)
RuntimeClassInImageCriApi=true|false (ALPHA - default=false)
SELinuxMountReadWriteOncePod=true|false (BETA - default=true)
SchedulerQueueingHints=true|false (BETA - default=false)
SecurityContextDeny=true|false (ALPHA - default=false)
SeparateTaintEvictionController=true|false (BETA - default=true)
ServiceAccountTokenJTI=true|false (ALPHA - default=false)
ServiceAccountTokenNodeBinding=true|false (ALPHA - default=false)
ServiceAccountTokenNodeBindingValidation=true|false (ALPHA - default=false)
ServiceAccountTokenPodNodeInfo=true|false (ALPHA - default=false)
SidecarContainers=true|false (BETA - default=true)
SizeMemoryBackedVolumes=true|false (BETA - default=true)
StableLoadBalancerNodeSet=true|false (BETA - default=true)
StatefulSetAutoDeletePVC=true|false (BETA - default=true)
StatefulSetStartOrdinal=true|false (BETA - default=true)
StorageVersionAPI=true|false (ALPHA - default=false)
StorageVersionHash=true|false (BETA - default=true)
StructuredAuthenticationConfiguration=true|false (ALPHA - default=false)
StructuredAuthorizationConfiguration=true|false (ALPHA - default=false)
TopologyAwareHints=true|false (BETA - default=true)
TopologyManagerPolicyAlphaOptions=true|false (ALPHA - default=false)
TopologyManagerPolicyBetaOptions=true|false (BETA - default=true)
TopologyManagerPolicyOptions=true|false (BETA - default=true)
TranslateStreamCloseWebsocketRequests=true|false (ALPHA - default=false)
UnauthenticatedHTTP2DOSMitigation=true|false (BETA - default=true)
UnknownVersionInteroperabilityProxy=true|false (ALPHA - default=false)
UserNamespacesPodSecurityStandards=true|false (ALPHA - default=false)
UserNamespacesSupport=true|false (ALPHA - default=false)
ValidatingAdmissionPolicy=true|false (BETA - default=false)
VolumeAttributesClass=true|false (ALPHA - default=false)
VolumeCapacityPriority=true|false (ALPHA - default=false)
WatchList=true|false (ALPHA - default=false)
WinDSR=true|false (ALPHA - default=false)
WinOverlay=true|false (BETA - default=true)
WindowsHostNetwork=true|false (ALPHA - default=true)
ZeroLimitedNominalConcurrencyShares=true|false (BETA - default=false)
(DEPRECATED: This parameter should be set via the config file specified by the kubelet's --config flag. See kubelet-config-file for more information.)
--file-check-frequency duration     Default: 20s
Duration between checking config files for new data. (DEPRECATED: This parameter should be set via the config file specified by the kubelet's --config flag. See kubelet-config-file for more information.)
--hairpin-mode string     Default: promiscuous-bridge
How should the kubelet setup hairpin NAT. This allows endpoints of a Service to load balance back to themselves if they should try to access their own Service. Valid values are "promiscuous-bridge", "hairpin-veth" and "none". (DEPRECATED: This parameter should be set via the config file specified by the kubelet's --config flag. See kubelet-config-file for more information.)
--healthz-bind-address string     Default: 127.0.0.1
The IP address for the healthz server to serve on (set to "0.0.0.0" or "::" for listening in all interfaces and IP families). (DEPRECATED: This parameter should be set via the config file specified by the kubelet's --config flag. See kubelet-config-file for more information.)
--healthz-port int32     Default: 10248
The port of the localhost healthz endpoint (set to 0 to disable). (DEPRECATED: This parameter should be set via the config file specified by the kubelet's --config flag. See kubelet-config-file for more information.)
-h, --help
help for kubelet
--hostname-override string
If non-empty, will use this string as identification instead of the actual hostname. If --cloud-provider is set, the cloud provider determines the name of the node (consult cloud provider documentation to determine if and how the hostname is used).
--http-check-frequency duration     Default: 20s
Duration between checking HTTP for new data. (DEPRECATED: This parameter should be set via the config file specified by the kubelet's --config flag. See kubelet-config-file for more information.)
--image-credential-provider-bin-dir string
The path to the directory where credential provider plugin binaries are located.
--image-credential-provider-config string
The path to the credential provider plugin config file.
--image-gc-high-threshold int32     Default: 85
The percent of disk usage after which image garbage collection is always run. Values must be within the range [0, 100], To disable image garbage collection, set to 100. (DEPRECATED: This parameter should be set via the config file specified by the kubelet's --config flag. See kubelet-config-file for more information.)
--image-gc-low-threshold int32     Default: 80
The percent of disk usage before which image garbage collection is never run. Lowest disk usage to garbage collect to. Values must be within the range [0, 100] and should not be larger than that of --image-gc-high-threshold. (DEPRECATED: This parameter should be set via the config file specified by the kubelet's --config flag. See kubelet-config-file for more information.)
--image-service-endpoint string
The endpoint of remote image service. If not specified, it will be the same with --container-runtime-endpoint by default. UNIX domain socket are supported on Linux, while `npipe` and `tcp` endpoints are supported on Windows. Examples: unix:///path/to/runtime.sock, npipe:////./pipe/runtime. (DEPRECATED: This parameter should be set via the config file specified by the kubelet's --config flag. See kubelet-config-file for more information.)
--keep-terminated-pod-volumes
Keep terminated pod volumes mounted to the node after the pod terminates. Can be useful for debugging volume related issues. (DEPRECATED: will be removed in a future version)
--kernel-memcg-notification
If enabled, the kubelet will integrate with the kernel memcg notification to determine if memory eviction thresholds are crossed rather than polling. (DEPRECATED: This parameter should be set via the config file specified by the kubelet's --config flag. See kubelet-config-file for more information.)
--kube-api-burst int32     Default: 100
Burst to use while talking with kubernetes API server. The number must be >= 0. If 0 will use default burst (100). Doesn't cover events and node heartbeat apis which rate limiting is controlled by a different set of flags. (DEPRECATED: This parameter should be set via the config file specified by the kubelet's --config flag. See kubelet-config-file for more information.)
--kube-api-content-type string     Default: application/vnd.kubernetes.protobuf
Content type of requests sent to apiserver. (DEPRECATED: This parameter should be set via the config file specified by the kubelet's --config flag. See kubelet-config-file for more information.)
--kube-api-qps int32     Default: 50
QPS to use while talking with kubernetes API server. The number must be >= 0. If 0 will use default QPS (50). Doesn't cover events and node heartbeat apis which rate limiting is controlled by a different set of flags. (DEPRECATED: This parameter should be set via the config file specified by the kubelet's --config flag. See kubelet-config-file for more information.)
--kube-reserved strings     Default: <None>
A set of <resource name>=<resource quantity> (e.g. "cpu=200m,memory=500Mi,ephemeral-storage=1Gi,pid='100'&auot;) pairs that describe resources reserved for kubernetes system components. Currently cpu, memory and local ephemeral-storage for root file system are supported. See here for more detail. (DEPRECATED: This parameter should be set via the config file specified by the kubelet's --config flag. See kubelet-config-file for more information.)
--kube-reserved-cgroup string     Default: ''
Absolute name of the top level cgroup that is used to manage kubernetes components for which compute resources were reserved via --kube-reserved flag. Ex. "/kube-reserved". (DEPRECATED: This parameter should be set via the config file specified by the kubelet's --config flag. See kubelet-config-file for more information.)
--kubeconfig string
Path to a kubeconfig file, specifying how to connect to the API server. Providing --kubeconfig enables API server mode, omitting --kubeconfig enables standalone mode.
--kubelet-cgroups string
Optional absolute name of cgroups to create and run the kubelet in. (DEPRECATED: This parameter should be set via the config file specified by the kubelet's --config flag. See kubelet-config-file for more information.)
--local-storage-capacity-isolation>     Default: true
If true, local ephemeral storage isolation is enabled. Otherwise, local storage isolation feature will be disabled. (DEPRECATED: This parameter should be set via the config file specified by the kubelet's --config flag. See kubelet-config-file for more information.)
--lock-file string
<Warning: Alpha feature> The path to file for kubelet to use as a lock file.
--log-flush-frequency duration     Default: 5s
Maximum number of seconds between log flushes.
--log-json-info-buffer-size string     Default: '0'
[Alpha] In JSON format with split output streams, the info messages can be buffered for a while to increase performance. The default value of zero bytes disables buffering. The size can be specified as number of bytes (512), multiples of 1000 (1K), multiples of 1024 (2Ki), or powers of those (3M, 4G, 5Mi, 6Gi). Enable the LoggingAlphaOptions feature gate to use this. (DEPRECATED: This parameter should be set via the config file specified by the kubelet's --config flag. See kubelet-config-file for more information.)
--log-json-split-stream
[Alpha] In JSON format, write error messages to stderr and info messages to stdout. The default is to write a single stream to stdout. Enable the LoggingAlphaOptions feature gate to use this. (DEPRECATED: This parameter should be set via the config file specified by the kubelet's --config flag. See kubelet-config-file for more information.)
--logging-format string     Default: text
Sets the log format. Permitted formats: "json" (gated by LoggingBetaOptions, "text"). (DEPRECATED: This parameter should be set via the config file specified by the kubelet's --config flag. See kubelet-config-file for more information.)
--make-iptables-util-chains     Default: true
If true, kubelet will ensure iptables utility rules are present on host. (DEPRECATED: This parameter should be set via the config file specified by the kubelet's --config flag. See kubelet-config-file for more information.)
--manifest-url string
URL for accessing additional Pod specifications to run. (DEPRECATED: This parameter should be set via the config file specified by the kubelet's --config flag. See kubelet-config-file for more information.)
--manifest-url-header strings
Comma-separated list of HTTP headers to use when accessing the URL provided to --manifest-url. Multiple headers with the same name will be added in the same order provided. This flag can be repeatedly invoked. For example: --manifest-url-header 'a:hello,b:again,c:world' --manifest-url-header 'b:beautiful' (DEPRECATED: This parameter should be set via the config file specified by the kubelet's --config flag. See kubelet-config-file for more information.)
--max-open-files int     Default: 1000000
Number of files that can be opened by kubelet process. (DEPRECATED: This parameter should be set via the config file specified by the kubelet's --config flag. See kubelet-config-file for more information.)
--max-pods int32     Default: 110
Number of Pods that can run on this kubelet. (DEPRECATED: This parameter should be set via the config file specified by the kubelet's --config flag. See kubelet-config-file for more information.)
--maximum-dead-containers int32     Default: -1
Maximum number of old instances of containers to retain globally. Each container takes up some disk space. To disable, set to a negative number. (DEPRECATED: Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.)
--maximum-dead-containers-per-container int32     Default: 1
Maximum number of old instances to retain per container. Each container takes up some disk space. (DEPRECATED: Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.)
--memory-manager-policy string     Default: None
Memory Manager policy to use. Possible values: "None", "Static". (DEPRECATED: This parameter should be set via the config file specified by the kubelet's --config flag. See kubelet-config-file for more information.)
--minimum-container-ttl-duration duration
Minimum age for a finished container before it is garbage collected. Examples: "300ms", "10s" or "2h45m". (DEPRECATED: Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.)
--minimum-image-ttl-duration duration     Default: 2m0s
Minimum age for an unused image before it is garbage collected. Examples: "300ms", "10s" or "2h45m". (DEPRECATED: This parameter should be set via the config file specified by the kubelet's --config flag. See kubelet-config-file for more information.)
--node-ip string
IP address (or comma-separated dual-stack IP addresses) of the node. If unset, kubelet will use the node's default IPv4 address, if any, or its default IPv6 address if it has no IPv4 addresses. You can pass "::" to make it prefer the default IPv6 address rather than the default IPv4 address.
--node-labels <key=value pairs>
<Warning: Alpha feature>Labels to add when registering the node in the cluster. Labels must be key=value pairs separated by ','. Labels in the 'kubernetes.io' namespace must begin with an allowed prefix ('kubelet.kubernetes.io', 'node.kubernetes.io') or be in the specifically allowed set ('beta.kubernetes.io/arch', 'beta.kubernetes.io/instance-type', 'beta.kubernetes.io/os', 'failure-domain.beta.kubernetes.io/region', 'failure-domain.beta.kubernetes.io/zone', 'kubernetes.io/arch', 'kubernetes.io/hostname', 'kubernetes.io/os', 'node.kubernetes.io/instance-type', 'topology.kubernetes.io/region', 'topology.kubernetes.io/zone')
--node-status-max-images int32     Default: 50
The maximum number of images to report in node.status.images. If -1 is specified, no cap will be applied. (DEPRECATED: This parameter should be set via the config file specified by the kubelet's --config flag. See kubelet-config-file for more information.)
--node-status-update-frequency duration     Default: 10s
Specifies how often kubelet posts node status to master. Note: be cautious when changing the constant, it must work with nodeMonitorGracePeriod in Node controller. (DEPRECATED: This parameter should be set via the config file specified by the kubelet's --config flag. See kubelet-config-file for more information.)
--oom-score-adj int32     Default: -999
The oom-score-adj value for kubelet process. Values must be within the range [-1000, 1000]. (DEPRECATED: This parameter should be set via the config file specified by the kubelet's --config flag. See kubelet-config-file for more information.)
--pod-cidr string
The CIDR to use for pod IP addresses, only used in standalone mode. In cluster mode, this is obtained from the master. For IPv6, the maximum number of IP's allocated is 65536 (DEPRECATED: This parameter should be set via the config file specified by the kubelet's --config flag. See kubelet-config-file for more information.)
--pod-infra-container-image string     Default: registry.k8s.io/pause:3.9
Specified image will not be pruned by the image garbage collector. CRI implementations have their own configuration to set this image. (DEPRECATED: will be removed in 1.27. Image garbage collector will get sandbox image information from CRI.)
--pod-manifest-path string
Path to the directory containing static pod files to run, or the path to a single static pod file. Files starting with dots will be ignored. (DEPRECATED: This parameter should be set via the config file specified by the kubelet's --config flag. See kubelet-config-file for more information.)
--pod-max-pids int     Default: -1
Set the maximum number of processes per pod. If -1, the kubelet defaults to the node allocatable PID capacity. (DEPRECATED: This parameter should be set via the config file specified by the kubelet's --config flag. See kubelet-config-file for more information.)
--pods-per-core int32
Number of Pods per core that can run on this kubelet. The total number of pods on this kubelet cannot exceed --max-pods, so --max-pods will be used if this calculation results in a larger number of pods allowed on the kubelet. A value of 0 disables this limit. (DEPRECATED: This parameter should be set via the config file specified by the kubelet's --config flag. See kubelet-config-file for more information.)
--port int32     Default: 10250
The port for the kubelet to serve on. (DEPRECATED: This parameter should be set via the config file specified by the kubelet's --config flag. See kubelet-config-file for more information.)
--protect-kernel-defaults
Default kubelet behaviour for kernel tuning. If set, kubelet errors if any of kernel tunables is different than kubelet defaults. (DEPRECATED: This parameter should be set via the config file specified by the kubelet's --config flag. See kubelet-config-file for more information.)
--provider-id string
Unique identifier for identifying the node in a machine database, i.e cloud provider.
--qos-reserved string
<Warning: Alpha feature> A set of <resource name>=<percentage> (e.g. "memory=50%") pairs that describe how pod resource requests are reserved at the QoS level. Currently only memory is supported. Requires the QOSReserved feature gate to be enabled. (DEPRECATED: This parameter should be set via the config file specified by the kubelet's --config flag. See kubelet-config-file for more information.)
--read-only-port int32     Default: 10255
The read-only port for the kubelet to serve on with no authentication/authorization (set to 0 to disable). (DEPRECATED: This parameter should be set via the config file specified by the kubelet's --config flag. See kubelet-config-file for more information.)
--register-node     Default: true
Register the node with the API server. If --kubeconfig is not provided, this flag is irrelevant, as the kubelet won't have an API server to register with. (DEPRECATED: This parameter should be set via the config file specified by the kubelet's --config flag. See kubelet-config-file for more information.)
--register-schedulable     Default: true
Register the node as schedulable. Won't have any effect if --register-node is false. (DEPRECATED: will be removed in a future version)
--register-with-taints string
Register the node with the given list of taints (comma separated <key>=<value>:<effect>). No-op if --register-node is false. (DEPRECATED: This parameter should be set via the config file specified by the kubelet's --config flag. See kubelet-config-file for more information.)
--registry-burst int32     Default: 10
Maximum size of a bursty pulls, temporarily allows pulls to burst to this number, while still not exceeding --registry-qps. Only used if --registry-qps is greater than 0. (DEPRECATED: This parameter should be set via the config file specified by the kubelet's --config flag. See kubelet-config-file for more information.)
--registry-qps int32     Default: 5
If > 0, limit registry pull QPS to this value. If 0, unlimited. (DEPRECATED: This parameter should be set via the config file specified by the kubelet's --config flag. See kubelet-config-file for more information.)
--reserved-cpus string
A comma-separated list of CPUs or CPU ranges that are reserved for system and kubernetes usage. This specific list will supersede cpu counts in --system-reserved and --kube-reserved. (DEPRECATED: This parameter should be set via the config file specified by the kubelet's --config flag. See kubelet-config-file for more information.)
--reserved-memory string
A comma-separated list of memory reservations for NUMA nodes. (e.g. "--reserved-memory 0:memory=1Gi,hugepages-1M=2Gi --reserved-memory 1:memory=2Gi"). The total sum for each memory type should be equal to the sum of --kube-reserved, --system-reserved and --eviction-threshold. See here for more details. (DEPRECATED: This parameter should be set via the config file specified by the kubelet's --config flag. See kubelet-config-file for more information.)
--resolv-conf string     Default: /etc/resolv.conf
Resolver configuration file used as the basis for the container DNS resolution configuration. (DEPRECATED: This parameter should be set via the config file specified by the kubelet's --config flag. See kubelet-config-file for more information.)
--root-dir string     Default: /var/lib/kubelet
Directory path for managing kubelet files (volume mounts, etc).
--rotate-certificates
Auto rotate the kubelet client certificates by requesting new certificates from the kube-apiserver when the certificate expiration approaches. (DEPRECATED: This parameter should be set via the config file specified by the kubelet's --config flag. See kubelet-config-file for more information.)
--rotate-server-certificates
Auto-request and rotate the kubelet serving certificates by requesting new certificates from the kube-apiserver when the certificate expiration approaches. Requires the RotateKubeletServerCertificate feature gate to be enabled, and approval of the submitted CertificateSigningRequest objects. (DEPRECATED: This parameter should be set via the config file specified by the kubelet's --config flag. See kubelet-config-file for more information.)
--runonce
If true, exit after spawning pods from local manifests or remote urls. Exclusive with --enable-server (DEPRECATED: This parameter should be set via the config file specified by the kubelet's --config flag. See kubelet-config-file for more information.)
--runtime-cgroups string
Optional absolute name of cgroups to create and run the runtime in.
--runtime-request-timeout duration     Default: 2m0s
Timeout of all runtime requests except long running request - pull, logs, exec and attach. When timeout exceeded, kubelet will cancel the request, throw out an error and retry later. (DEPRECATED: This parameter should be set via the config file specified by the kubelet's --config flag. See kubelet-config-file for more information.)
--seccomp-default
Enable the use of RuntimeDefault as the default seccomp profile for all workloads.
--serialize-image-pulls     Default: true
Pull images one at a time. We recommend *not* changing the default value on nodes that run docker daemon with version < 1.9 or an aufs storage backend. Issue #10959 has more details. (DEPRECATED: This parameter should be set via the config file specified by the kubelet's --config flag. See kubelet-config-file for more information.)
--streaming-connection-idle-timeout duration     Default: 4h0m0s
Maximum time a streaming connection can be idle before the connection is automatically closed. 0 indicates no timeout. Example: 5m. Note: All connections to the kubelet server have a maximum duration of 4 hours. (DEPRECATED: This parameter should be set via the config file specified by the kubelet's --config flag. See kubelet-config-file for more information.)
--sync-frequency duration     Default: 1m0s
Max period between synchronizing running containers and config. (DEPRECATED: This parameter should be set via the config file specified by the kubelet's --config flag. See kubelet-config-file for more information.)
--system-cgroups string
Optional absolute name of cgroups in which to place all non-kernel processes that are not already inside a cgroup under '/'. Empty for no container. Rolling back the flag requires a reboot. (DEPRECATED: This parameter should be set via the config file specified by the kubelet's --config flag. See kubelet-config-file for more information.)
--system-reserved string     Default: <none>
A set of <resource name>=<resource quantity> (e.g. "cpu=200m,memory=500Mi,ephemeral-storage=1Gi,pid='100'") pairs that describe resources reserved for non-kubernetes components. Currently only cpu and memory and local ephemeral storage for root file system are supported. See here for more detail. (DEPRECATED: This parameter should be set via the config file specified by the kubelet's --config flag. See kubelet-config-file for more information.)
--system-reserved-cgroup string     Default: ''
Absolute name of the top level cgroup that is used to manage non-kubernetes components for which compute resources were reserved via --system-reserved flag. Ex. /system-reserved. (DEPRECATED: This parameter should be set via the config file specified by the kubelet's --config flag. See kubelet-config-file for more information.)
--tls-cert-file string
File containing x509 certificate used for serving HTTPS (with intermediate certs, if any, concatenated after server cert). If --tls-cert-file and --tls-private-key-file are not provided, a self-signed certificate and key are generated for the public address and saved to the directory passed to --cert-dir. (DEPRECATED: This parameter should be set via the config file specified by the kubelet's --config flag. See kubelet-config-file for more information.)
--tls-cipher-suites string
Comma-separated list of cipher suites for the server. If omitted, the default Go cipher suites will be used.
Preferred values: TLS_AES_128_GCM_SHA256, TLS_AES_256_GCM_SHA384, TLS_CHACHA20_POLY1305_SHA256, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305, TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305, TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256, TLS_RSA_WITH_AES_128_CBC_SHA, TLS_RSA_WITH_AES_128_GCM_SHA256, TLS_RSA_WITH_AES_256_CBC_SHA, TLS_RSA_WITH_AES_256_GCM_SHA384
Insecure values: TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_ECDSA_WITH_RC4_128_SHA, TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_RSA_WITH_RC4_128_SHA, TLS_RSA_WITH_3DES_EDE_CBC_SHA, TLS_RSA_WITH_AES_128_CBC_SHA256, TLS_RSA_WITH_RC4_128_SHA.
(DEPRECATED: This parameter should be set via the config file specified by the kubelet's --config flag. See kubelet-config-file for more information.)
--tls-min-version string
Minimum TLS version supported. Possible values: "VersionTLS10", "VersionTLS11", "VersionTLS12", "VersionTLS13". (DEPRECATED: This parameter should be set via the config file specified by the kubelet's --config flag. See kubelet-config-file for more information.)
--tls-private-key-file string
File containing x509 private key matching --tls-cert-file. (DEPRECATED: This parameter should be set via the config file specified by the kubelet's --config flag. See kubelet-config-file for more information.)
--topology-manager-policy string     Default: 'none'
Topology Manager policy to use. Possible values: "none", "best-effort", "restricted", "single-numa-node". (DEPRECATED: This parameter should be set via the config file specified by the kubelet's --config flag. See kubelet-config-file for more information.)
--topology-manager-policy-options string
A set of <key>=<value> topology manager policy options to use, to fine tune their behaviour. If not supplied, keep the default behaviour. (DEPRECATED: This parameter should be set via the config file specified by the kubelet's --config flag. See kubelet-config-file for more information.)
--topology-manager-scope string     Default: container
Scope to which topology hints are applied. Topology manager collects hints from hint providers and applies them to the defined scope to ensure the pod admission. Possible values: "container", "pod". (DEPRECATED: This parameter should be set via the config file specified by the kubelet's --config flag. See kubelet-config-file for more information.)
-v, --v Level
Number for the log level verbosity
--version version[=true]
Print version information and quit; --version=vX.Y.Z... sets the reported version.
--vmodule <A list of 'pattern=N' strings>
Comma-separated list of pattern=N settings for file-filtered logging (only works for text log format).
--volume-plugin-dir string     Default: /usr/libexec/kubernetes/kubelet-plugins/volume/exec/
The full path of the directory in which to search for additional third party volume plugins. (DEPRECATED: This parameter should be set via the config file specified by the kubelet's --config flag. See kubelet-config-file for more information.)
--volume-stats-agg-period duration     Default: 1m0s
Specifies interval for kubelet to calculate and cache the volume disk usage for all pods and volumes. To disable volume calculations, set to a negative number. (DEPRECATED: This parameter should be set via the config file specified by the kubelet's --config flag. See kubelet-config-file for more information.)

4 - kube-apiserver

Synopsis

The Kubernetes API server validates and configures data for the api objects which include pods, services, replicationcontrollers, and others. The API Server services REST operations and provides the frontend to the cluster's shared state through which all other components interact.

kube-apiserver [flags]

Options

--admission-control-config-file string

File with admission control configuration.

--advertise-address string

The IP address on which to advertise the apiserver to members of the cluster. This address must be reachable by the rest of the cluster. If blank, the --bind-address will be used. If --bind-address is unspecified, the host's default interface will be used.

--aggregator-reject-forwarding-redirect     Default: true

Aggregator reject forwarding redirect response back to client.

--allow-metric-labels stringToString     Default: []

The map from metric-label to value allow-list of this label. The key's format is <MetricName>,<LabelName>. The value's format is <allowed_value>,<allowed_value>...e.g. metric1,label1='v1,v2,v3', metric1,label2='v1,v2,v3' metric2,label1='v1,v2,v3'.

--allow-metric-labels-manifest string

The path to the manifest file that contains the allow-list mapping. The format of the file is the same as the flag --allow-metric-labels. Note that the flag --allow-metric-labels will override the manifest file.

--allow-privileged

If true, allow privileged containers. [default=false]

--anonymous-auth     Default: true

Enables anonymous requests to the secure port of the API server. Requests that are not rejected by another authentication method are treated as anonymous requests. Anonymous requests have a username of system:anonymous, and a group name of system:unauthenticated.

--api-audiences strings

Identifiers of the API. The service account token authenticator will validate that tokens used against the API are bound to at least one of these audiences. If the --service-account-issuer flag is configured and this flag is not, this field defaults to a single element list containing the issuer URL.

--audit-log-batch-buffer-size int     Default: 10000

The size of the buffer to store events before batching and writing. Only used in batch mode.

--audit-log-batch-max-size int     Default: 1

The maximum size of a batch. Only used in batch mode.

--audit-log-batch-max-wait duration

The amount of time to wait before force writing the batch that hadn't reached the max size. Only used in batch mode.

--audit-log-batch-throttle-burst int

Maximum number of requests sent at the same moment if ThrottleQPS was not utilized before. Only used in batch mode.

--audit-log-batch-throttle-enable

Whether batching throttling is enabled. Only used in batch mode.

--audit-log-batch-throttle-qps float

Maximum average number of batches per second. Only used in batch mode.

--audit-log-compress

If set, the rotated log files will be compressed using gzip.

--audit-log-format string     Default: "json"

Format of saved audits. "legacy" indicates 1-line text format for each event. "json" indicates structured json format. Known formats are legacy,json.

--audit-log-maxage int

The maximum number of days to retain old audit log files based on the timestamp encoded in their filename.

--audit-log-maxbackup int

The maximum number of old audit log files to retain. Setting a value of 0 will mean there's no restriction on the number of files.

--audit-log-maxsize int

The maximum size in megabytes of the audit log file before it gets rotated.

--audit-log-mode string     Default: "blocking"

Strategy for sending audit events. Blocking indicates sending events should block server responses. Batch causes the backend to buffer and write events asynchronously. Known modes are batch,blocking,blocking-strict.

--audit-log-path string

If set, all requests coming to the apiserver will be logged to this file. '-' means standard out.

--audit-log-truncate-enabled

Whether event and batch truncating is enabled.

--audit-log-truncate-max-batch-size int     Default: 10485760

Maximum size of the batch sent to the underlying backend. Actual serialized size can be several hundreds of bytes greater. If a batch exceeds this limit, it is split into several batches of smaller size.

--audit-log-truncate-max-event-size int     Default: 102400

Maximum size of the audit event sent to the underlying backend. If the size of an event is greater than this number, first request and response are removed, and if this doesn't reduce the size enough, event is discarded.

--audit-log-version string     Default: "audit.k8s.io/v1"

API group and version used for serializing audit events written to log.

--audit-policy-file string

Path to the file that defines the audit policy configuration.

--audit-webhook-batch-buffer-size int     Default: 10000

The size of the buffer to store events before batching and writing. Only used in batch mode.

--audit-webhook-batch-max-size int     Default: 400

The maximum size of a batch. Only used in batch mode.

--audit-webhook-batch-max-wait duration     Default: 30s

The amount of time to wait before force writing the batch that hadn't reached the max size. Only used in batch mode.

--audit-webhook-batch-throttle-burst int     Default: 15

Maximum number of requests sent at the same moment if ThrottleQPS was not utilized before. Only used in batch mode.

--audit-webhook-batch-throttle-enable     Default: true

Whether batching throttling is enabled. Only used in batch mode.

--audit-webhook-batch-throttle-qps float     Default: 10

Maximum average number of batches per second. Only used in batch mode.

--audit-webhook-config-file string

Path to a kubeconfig formatted file that defines the audit webhook configuration.

--audit-webhook-initial-backoff duration     Default: 10s

The amount of time to wait before retrying the first failed request.

--audit-webhook-mode string     Default: "batch"

Strategy for sending audit events. Blocking indicates sending events should block server responses. Batch causes the backend to buffer and write events asynchronously. Known modes are batch,blocking,blocking-strict.

--audit-webhook-truncate-enabled

Whether event and batch truncating is enabled.

--audit-webhook-truncate-max-batch-size int     Default: 10485760

Maximum size of the batch sent to the underlying backend. Actual serialized size can be several hundreds of bytes greater. If a batch exceeds this limit, it is split into several batches of smaller size.

--audit-webhook-truncate-max-event-size int     Default: 102400

Maximum size of the audit event sent to the underlying backend. If the size of an event is greater than this number, first request and response are removed, and if this doesn't reduce the size enough, event is discarded.

--audit-webhook-version string     Default: "audit.k8s.io/v1"

API group and version used for serializing audit events written to webhook.

--authentication-config string

File with Authentication Configuration to configure the JWT Token authenticator. Note: This feature is in Alpha since v1.29.--feature-gate=StructuredAuthenticationConfiguration=true needs to be set for enabling this feature.This feature is mutually exclusive with the oidc-* flags.

--authentication-token-webhook-cache-ttl duration     Default: 2m0s

The duration to cache responses from the webhook token authenticator.

--authentication-token-webhook-config-file string

File with webhook configuration for token authentication in kubeconfig format. The API server will query the remote service to determine authentication for bearer tokens.

--authentication-token-webhook-version string     Default: "v1beta1"

The API version of the authentication.k8s.io TokenReview to send to and expect from the webhook.

--authorization-config string

File with Authorization Configuration to configure the authorizer chain.Note: This feature is in Alpha since v1.29.--feature-gate=StructuredAuthorizationConfiguration=true feature flag needs to be set to true for enabling the functionality.This feature is mutually exclusive with the other --authorization-mode and --authorization-webhook-* flags.

--authorization-mode strings

Ordered list of plug-ins to do authorization on secure port. Defaults to AlwaysAllow if --authorization-config is not used. Comma-delimited list of: AlwaysAllow,AlwaysDeny,ABAC,Webhook,RBAC,Node.

--authorization-policy-file string

File with authorization policy in json line by line format, used with --authorization-mode=ABAC, on the secure port.

--authorization-webhook-cache-authorized-ttl duration     Default: 5m0s

The duration to cache 'authorized' responses from the webhook authorizer.

--authorization-webhook-cache-unauthorized-ttl duration     Default: 30s

The duration to cache 'unauthorized' responses from the webhook authorizer.

--authorization-webhook-config-file string

File with webhook configuration in kubeconfig format, used with --authorization-mode=Webhook. The API server will query the remote service to determine access on the API server's secure port.

--authorization-webhook-version string     Default: "v1beta1"

The API version of the authorization.k8s.io SubjectAccessReview to send to and expect from the webhook.

--azure-container-registry-config string

Path to the file containing Azure container registry configuration information.

--bind-address string     Default: 0.0.0.0

The IP address on which to listen for the --secure-port port. The associated interface(s) must be reachable by the rest of the cluster, and by CLI/web clients. If blank or an unspecified address (0.0.0.0 or ::), all interfaces and IP address families will be used.

--cert-dir string     Default: "/var/run/kubernetes"

The directory where the TLS certs are located. If --tls-cert-file and --tls-private-key-file are provided, this flag will be ignored.

--client-ca-file string

If set, any request presenting a client certificate signed by one of the authorities in the client-ca-file is authenticated with an identity corresponding to the CommonName of the client certificate.

--cloud-provider-gce-l7lb-src-cidrs cidrs     Default: 130.211.0.0/22,35.191.0.0/16

CIDRs opened in GCE firewall for L7 LB traffic proxy & health checks

--contention-profiling

Enable block profiling, if profiling is enabled

--cors-allowed-origins strings

List of allowed origins for CORS, comma separated. An allowed origin can be a regular expression to support subdomain matching. If this list is empty CORS will not be enabled. Please ensure each expression matches the entire hostname by anchoring to the start with '^' or including the '//' prefix, and by anchoring to the end with '$' or including the ':' port separator suffix. Examples of valid expressions are '//example.com(:|$)' and '^https://example.com(:|$)'

--debug-socket-path string

Use an unprotected (no authn/authz) unix-domain socket for profiling with the given path

--default-not-ready-toleration-seconds int     Default: 300

Indicates the tolerationSeconds of the toleration for notReady:NoExecute that is added by default to every pod that does not already have such a toleration.

--default-unreachable-toleration-seconds int     Default: 300

Indicates the tolerationSeconds of the toleration for unreachable:NoExecute that is added by default to every pod that does not already have such a toleration.

--delete-collection-workers int     Default: 1

Number of workers spawned for DeleteCollection call. These are used to speed up namespace cleanup.

--disable-admission-plugins strings

admission plugins that should be disabled although they are in the default enabled plugins list (NamespaceLifecycle, LimitRanger, ServiceAccount, TaintNodesByCondition, PodSecurity, Priority, DefaultTolerationSeconds, DefaultStorageClass, StorageObjectInUseProtection, PersistentVolumeClaimResize, RuntimeClass, CertificateApproval, CertificateSigning, ClusterTrustBundleAttest, CertificateSubjectRestriction, DefaultIngressClass, MutatingAdmissionWebhook, ValidatingAdmissionPolicy, ValidatingAdmissionWebhook, ResourceQuota). Comma-delimited list of admission plugins: AlwaysAdmit, AlwaysDeny, AlwaysPullImages, CertificateApproval, CertificateSigning, CertificateSubjectRestriction, ClusterTrustBundleAttest, DefaultIngressClass, DefaultStorageClass, DefaultTolerationSeconds, DenyServiceExternalIPs, EventRateLimit, ExtendedResourceToleration, ImagePolicyWebhook, LimitPodHardAntiAffinityTopology, LimitRanger, MutatingAdmissionWebhook, NamespaceAutoProvision, NamespaceExists, NamespaceLifecycle, NodeRestriction, OwnerReferencesPermissionEnforcement, PersistentVolumeClaimResize, PersistentVolumeLabel, PodNodeSelector, PodSecurity, PodTolerationRestriction, Priority, ResourceQuota, RuntimeClass, SecurityContextDeny, ServiceAccount, StorageObjectInUseProtection, TaintNodesByCondition, ValidatingAdmissionPolicy, ValidatingAdmissionWebhook. The order of plugins in this flag does not matter.

--disabled-metrics strings

This flag provides an escape hatch for misbehaving metrics. You must provide the fully qualified metric name in order to disable it. Disclaimer: disabling metrics is higher in precedence than showing hidden metrics.

--egress-selector-config-file string

File with apiserver egress selector configuration.

--enable-admission-plugins strings

admission plugins that should be enabled in addition to default enabled ones (NamespaceLifecycle, LimitRanger, ServiceAccount, TaintNodesByCondition, PodSecurity, Priority, DefaultTolerationSeconds, DefaultStorageClass, StorageObjectInUseProtection, PersistentVolumeClaimResize, RuntimeClass, CertificateApproval, CertificateSigning, ClusterTrustBundleAttest, CertificateSubjectRestriction, DefaultIngressClass, MutatingAdmissionWebhook, ValidatingAdmissionPolicy, ValidatingAdmissionWebhook, ResourceQuota). Comma-delimited list of admission plugins: AlwaysAdmit, AlwaysDeny, AlwaysPullImages, CertificateApproval, CertificateSigning, CertificateSubjectRestriction, ClusterTrustBundleAttest, DefaultIngressClass, DefaultStorageClass, DefaultTolerationSeconds, DenyServiceExternalIPs, EventRateLimit, ExtendedResourceToleration, ImagePolicyWebhook, LimitPodHardAntiAffinityTopology, LimitRanger, MutatingAdmissionWebhook, NamespaceAutoProvision, NamespaceExists, NamespaceLifecycle, NodeRestriction, OwnerReferencesPermissionEnforcement, PersistentVolumeClaimResize, PersistentVolumeLabel, PodNodeSelector, PodSecurity, PodTolerationRestriction, Priority, ResourceQuota, RuntimeClass, SecurityContextDeny, ServiceAccount, StorageObjectInUseProtection, TaintNodesByCondition, ValidatingAdmissionPolicy, ValidatingAdmissionWebhook. The order of plugins in this flag does not matter.

--enable-aggregator-routing

Turns on aggregator routing requests to endpoints IP rather than cluster IP.

--enable-bootstrap-token-auth

Enable to allow secrets of type 'bootstrap.kubernetes.io/token' in the 'kube-system' namespace to be used for TLS bootstrapping authentication.

--enable-garbage-collector     Default: true

Enables the generic garbage collector. MUST be synced with the corresponding flag of the kube-controller-manager.

--enable-priority-and-fairness     Default: true

If true, replace the max-in-flight handler with an enhanced one that queues and dispatches with priority and fairness

--encryption-provider-config string

The file containing configuration for encryption providers to be used for storing secrets in etcd

--encryption-provider-config-automatic-reload

Determines if the file set by --encryption-provider-config should be automatically reloaded if the disk contents change. Setting this to true disables the ability to uniquely identify distinct KMS plugins via the API server healthz endpoints.

--endpoint-reconciler-type string     Default: "lease"

Use an endpoint reconciler (master-count, lease, none) master-count is deprecated, and will be removed in a future version.

--etcd-cafile string

SSL Certificate Authority file used to secure etcd communication.

--etcd-certfile string

SSL certification file used to secure etcd communication.

--etcd-compaction-interval duration     Default: 5m0s

The interval of compaction requests. If 0, the compaction request from apiserver is disabled.

--etcd-count-metric-poll-period duration     Default: 1m0s

Frequency of polling etcd for number of resources per type. 0 disables the metric collection.

--etcd-db-metric-poll-interval duration     Default: 30s

The interval of requests to poll etcd and update metric. 0 disables the metric collection

--etcd-healthcheck-timeout duration     Default: 2s

The timeout to use when checking etcd health.

--etcd-keyfile string

SSL key file used to secure etcd communication.

--etcd-prefix string     Default: "/registry"

The prefix to prepend to all resource paths in etcd.

--etcd-readycheck-timeout duration     Default: 2s

The timeout to use when checking etcd readiness

--etcd-servers strings

List of etcd servers to connect with (scheme://ip:port), comma separated.

--etcd-servers-overrides strings

Per-resource etcd servers overrides, comma separated. The individual override format: group/resource#servers, where servers are URLs, semicolon separated. Note that this applies only to resources compiled into this server binary.

--event-ttl duration     Default: 1h0m0s

Amount of time to retain events.

--external-hostname string

The hostname to use when generating externalized URLs for this master (e.g. Swagger API Docs or OpenID Discovery).

--feature-gates <comma-separated 'key=True|False' pairs>

A set of key=value pairs that describe feature gates for alpha/experimental features. Options are:
APIResponseCompression=true|false (BETA - default=true)
APIServerIdentity=true|false (BETA - default=true)
APIServerTracing=true|false (BETA - default=true)
AdmissionWebhookMatchConditions=true|false (BETA - default=true)
AggregatedDiscoveryEndpoint=true|false (BETA - default=true)
AllAlpha=true|false (ALPHA - default=false)
AllBeta=true|false (BETA - default=false)
AnyVolumeDataSource=true|false (BETA - default=true)
AppArmor=true|false (BETA - default=true)
CPUManagerPolicyAlphaOptions=true|false (ALPHA - default=false)
CPUManagerPolicyBetaOptions=true|false (BETA - default=true)
CPUManagerPolicyOptions=true|false (BETA - default=true)
CRDValidationRatcheting=true|false (ALPHA - default=false)
CSIMigrationPortworx=true|false (BETA - default=false)
CSIVolumeHealth=true|false (ALPHA - default=false)
CloudControllerManagerWebhook=true|false (ALPHA - default=false)
CloudDualStackNodeIPs=true|false (BETA - default=true)
ClusterTrustBundle=true|false (ALPHA - default=false)
ClusterTrustBundleProjection=true|false (ALPHA - default=false)
ComponentSLIs=true|false (BETA - default=true)
ConsistentListFromCache=true|false (ALPHA - default=false)
ContainerCheckpoint=true|false (ALPHA - default=false)
ContextualLogging=true|false (ALPHA - default=false)
CronJobsScheduledAnnotation=true|false (BETA - default=true)
CrossNamespaceVolumeDataSource=true|false (ALPHA - default=false)
CustomCPUCFSQuotaPeriod=true|false (ALPHA - default=false)
DevicePluginCDIDevices=true|false (BETA - default=true)
DisableCloudProviders=true|false (BETA - default=true)
DisableKubeletCloudCredentialProviders=true|false (BETA - default=true)
DisableNodeKubeProxyVersion=true|false (ALPHA - default=false)
DynamicResourceAllocation=true|false (ALPHA - default=false)
ElasticIndexedJob=true|false (BETA - default=true)
EventedPLEG=true|false (BETA - default=false)
GracefulNodeShutdown=true|false (BETA - default=true)
GracefulNodeShutdownBasedOnPodPriority=true|false (BETA - default=true)
HPAContainerMetrics=true|false (BETA - default=true)
HPAScaleToZero=true|false (ALPHA - default=false)
HonorPVReclaimPolicy=true|false (ALPHA - default=false)
ImageMaximumGCAge=true|false (ALPHA - default=false)
InPlacePodVerticalScaling=true|false (ALPHA - default=false)
InTreePluginAWSUnregister=true|false (ALPHA - default=false)
InTreePluginAzureDiskUnregister=true|false (ALPHA - default=false)
InTreePluginAzureFileUnregister=true|false (ALPHA - default=false)
InTreePluginGCEUnregister=true|false (ALPHA - default=false)
InTreePluginOpenStackUnregister=true|false (ALPHA - default=false)
InTreePluginPortworxUnregister=true|false (ALPHA - default=false)
InTreePluginvSphereUnregister=true|false (ALPHA - default=false)
JobBackoffLimitPerIndex=true|false (BETA - default=true)
JobPodFailurePolicy=true|false (BETA - default=true)
JobPodReplacementPolicy=true|false (BETA - default=true)
KubeProxyDrainingTerminatingNodes=true|false (ALPHA - default=false)
KubeletCgroupDriverFromCRI=true|false (ALPHA - default=false)
KubeletInUserNamespace=true|false (ALPHA - default=false)
KubeletPodResourcesDynamicResources=true|false (ALPHA - default=false)
KubeletPodResourcesGet=true|false (ALPHA - default=false)
KubeletSeparateDiskGC=true|false (ALPHA - default=false)
KubeletTracing=true|false (BETA - default=true)
LegacyServiceAccountTokenCleanUp=true|false (BETA - default=true)
LoadBalancerIPMode=true|false (ALPHA - default=false)
LocalStorageCapacityIsolationFSQuotaMonitoring=true|false (ALPHA - default=false)
LogarithmicScaleDown=true|false (BETA - default=true)
LoggingAlphaOptions=true|false (ALPHA - default=false)
LoggingBetaOptions=true|false (BETA - default=true)
MatchLabelKeysInPodAffinity=true|false (ALPHA - default=false)
MatchLabelKeysInPodTopologySpread=true|false (BETA - default=true)
MaxUnavailableStatefulSet=true|false (ALPHA - default=false)
MemoryManager=true|false (BETA - default=true)
MemoryQoS=true|false (ALPHA - default=false)
MinDomainsInPodTopologySpread=true|false (BETA - default=true)
MultiCIDRServiceAllocator=true|false (ALPHA - default=false)
NFTablesProxyMode=true|false (ALPHA - default=false)
NewVolumeManagerReconstruction=true|false (BETA - default=true)
NodeInclusionPolicyInPodTopologySpread=true|false (BETA - default=true)
NodeLogQuery=true|false (ALPHA - default=false)
NodeSwap=true|false (BETA - default=false)
OpenAPIEnums=true|false (BETA - default=true)
PDBUnhealthyPodEvictionPolicy=true|false (BETA - default=true)
PersistentVolumeLastPhaseTransitionTime=true|false (BETA - default=true)
PodAndContainerStatsFromCRI=true|false (ALPHA - default=false)
PodDeletionCost=true|false (BETA - default=true)
PodDisruptionConditions=true|false (BETA - default=true)
PodHostIPs=true|false (BETA - default=true)
PodIndexLabel=true|false (BETA - default=true)
PodLifecycleSleepAction=true|false (ALPHA - default=false)
PodReadyToStartContainersCondition=true|false (BETA - default=true)
PodSchedulingReadiness=true|false (BETA - default=true)
ProcMountType=true|false (ALPHA - default=false)
QOSReserved=true|false (ALPHA - default=false)
RecoverVolumeExpansionFailure=true|false (ALPHA - default=false)
RotateKubeletServerCertificate=true|false (BETA - default=true)
RuntimeClassInImageCriApi=true|false (ALPHA - default=false)
SELinuxMountReadWriteOncePod=true|false (BETA - default=true)
SchedulerQueueingHints=true|false (BETA - default=false)
SecurityContextDeny=true|false (ALPHA - default=false)
SeparateTaintEvictionController=true|false (BETA - default=true)
ServiceAccountTokenJTI=true|false (ALPHA - default=false)
ServiceAccountTokenNodeBinding=true|false (ALPHA - default=false)
ServiceAccountTokenNodeBindingValidation=true|false (ALPHA - default=false)
ServiceAccountTokenPodNodeInfo=true|false (ALPHA - default=false)
SidecarContainers=true|false (BETA - default=true)
SizeMemoryBackedVolumes=true|false (BETA - default=true)
StableLoadBalancerNodeSet=true|false (BETA - default=true)
StatefulSetAutoDeletePVC=true|false (BETA - default=true)
StatefulSetStartOrdinal=true|false (BETA - default=true)
StorageVersionAPI=true|false (ALPHA - default=false)
StorageVersionHash=true|false (BETA - default=true)
StructuredAuthenticationConfiguration=true|false (ALPHA - default=false)
StructuredAuthorizationConfiguration=true|false (ALPHA - default=false)
TopologyAwareHints=true|false (BETA - default=true)
TopologyManagerPolicyAlphaOptions=true|false (ALPHA - default=false)
TopologyManagerPolicyBetaOptions=true|false (BETA - default=true)
TopologyManagerPolicyOptions=true|false (BETA - default=true)
TranslateStreamCloseWebsocketRequests=true|false (ALPHA - default=false)
UnauthenticatedHTTP2DOSMitigation=true|false (BETA - default=true)
UnknownVersionInteroperabilityProxy=true|false (ALPHA - default=false)
UserNamespacesPodSecurityStandards=true|false (ALPHA - default=false)
UserNamespacesSupport=true|false (ALPHA - default=false)
ValidatingAdmissionPolicy=true|false (BETA - default=false)
VolumeAttributesClass=true|false (ALPHA - default=false)
VolumeCapacityPriority=true|false (ALPHA - default=false)
WatchList=true|false (ALPHA - default=false)
WinDSR=true|false (ALPHA - default=false)
WinOverlay=true|false (BETA - default=true)
WindowsHostNetwork=true|false (ALPHA - default=true)
ZeroLimitedNominalConcurrencyShares=true|false (BETA - default=false)

--goaway-chance float

To prevent HTTP/2 clients from getting stuck on a single apiserver, randomly close a connection (GOAWAY). The client's other in-flight requests won't be affected, and the client will reconnect, likely landing on a different apiserver after going through the load balancer again. This argument sets the fraction of requests that will be sent a GOAWAY. Clusters with single apiservers, or which don't use a load balancer, should NOT enable this. Min is 0 (off), Max is .02 (1/50 requests); .001 (1/1000) is a recommended starting point.

-h, --help

help for kube-apiserver

--http2-max-streams-per-connection int

The limit that the server gives to clients for the maximum number of streams in an HTTP/2 connection. Zero means to use golang's default.

--kubelet-certificate-authority string

Path to a cert file for the certificate authority.

--kubelet-client-certificate string

Path to a client cert file for TLS.

--kubelet-client-key string

Path to a client key file for TLS.

--kubelet-preferred-address-types strings     Default: "Hostname,InternalDNS,InternalIP,ExternalDNS,ExternalIP"

List of the preferred NodeAddressTypes to use for kubelet connections.

--kubelet-timeout duration     Default: 5s

Timeout for kubelet operations.

--kubernetes-service-node-port int

If non-zero, the Kubernetes master service (which apiserver creates/maintains) will be of type NodePort, using this as the value of the port. If zero, the Kubernetes master service will be of type ClusterIP.

--lease-reuse-duration-seconds int     Default: 60

The time in seconds that each lease is reused. A lower value could avoid large number of objects reusing the same lease. Notice that a too small value may cause performance problems at storage layer.

--livez-grace-period duration

This option represents the maximum amount of time it should take for apiserver to complete its startup sequence and become live. From apiserver's start time to when this amount of time has elapsed, /livez will assume that unfinished post-start hooks will complete successfully and therefore return true.

--log-flush-frequency duration     Default: 5s

Maximum number of seconds between log flushes

--logging-format string     Default: "text"

Sets the log format. Permitted formats: "text".

--max-connection-bytes-per-sec int

If non-zero, throttle each user connection to this number of bytes/sec. Currently only applies to long-running requests.

--max-mutating-requests-inflight int     Default: 200

This and --max-requests-inflight are summed to determine the server's total concurrency limit (which must be positive) if --enable-priority-and-fairness is true. Otherwise, this flag limits the maximum number of mutating requests in flight, or a zero value disables the limit completely.

--max-requests-inflight int     Default: 400

This and --max-mutating-requests-inflight are summed to determine the server's total concurrency limit (which must be positive) if --enable-priority-and-fairness is true. Otherwise, this flag limits the maximum number of non-mutating requests in flight, or a zero value disables the limit completely.

--min-request-timeout int     Default: 1800

An optional field indicating the minimum number of seconds a handler must keep a request open before timing it out. Currently only honored by the watch request handler, which picks a randomized value above this number as the connection timeout, to spread out load.

--oidc-ca-file string

If set, the OpenID server's certificate will be verified by one of the authorities in the oidc-ca-file, otherwise the host's root CA set will be used.

--oidc-client-id string

The client ID for the OpenID Connect client, must be set if oidc-issuer-url is set.

--oidc-groups-claim string

If provided, the name of a custom OpenID Connect claim for specifying user groups. The claim value is expected to be a string or array of strings. This flag is experimental, please see the authentication documentation for further details.

--oidc-groups-prefix string

If provided, all groups will be prefixed with this value to prevent conflicts with other authentication strategies.

--oidc-issuer-url string

The URL of the OpenID issuer, only HTTPS scheme will be accepted. If set, it will be used to verify the OIDC JSON Web Token (JWT).

--oidc-required-claim <comma-separated 'key=value' pairs>

A key=value pair that describes a required claim in the ID Token. If set, the claim is verified to be present in the ID Token with a matching value. Repeat this flag to specify multiple claims.

--oidc-signing-algs strings     Default: "RS256"

Comma-separated list of allowed JOSE asymmetric signing algorithms. JWTs with a supported 'alg' header values are: RS256, RS384, RS512, ES256, ES384, ES512, PS256, PS384, PS512. Values are defined by RFC 7518 https://tools.ietf.org/html/rfc7518#section-3.1.

--oidc-username-claim string     Default: "sub"

The OpenID claim to use as the user name. Note that claims other than the default ('sub') is not guaranteed to be unique and immutable. This flag is experimental, please see the authentication documentation for further details.

--oidc-username-prefix string

If provided, all usernames will be prefixed with this value. If not provided, username claims other than 'email' are prefixed by the issuer URL to avoid clashes. To skip any prefixing, provide the value '-'.

--peer-advertise-ip string

If set and the UnknownVersionInteroperabilityProxy feature gate is enabled, this IP will be used by peer kube-apiservers to proxy requests to this kube-apiserver when the request cannot be handled by the peer due to version skew between the kube-apiservers. This flag is only used in clusters configured with multiple kube-apiservers for high availability.

--peer-advertise-port string

If set and the UnknownVersionInteroperabilityProxy feature gate is enabled, this port will be used by peer kube-apiservers to proxy requests to this kube-apiserver when the request cannot be handled by the peer due to version skew between the kube-apiservers. This flag is only used in clusters configured with multiple kube-apiservers for high availability.

--peer-ca-file string

If set and the UnknownVersionInteroperabilityProxy feature gate is enabled, this file will be used to verify serving certificates of peer kube-apiservers. This flag is only used in clusters configured with multiple kube-apiservers for high availability.

--permit-address-sharing

If true, SO_REUSEADDR will be used when binding the port. This allows binding to wildcard IPs like 0.0.0.0 and specific IPs in parallel, and it avoids waiting for the kernel to release sockets in TIME_WAIT state. [default=false]

--permit-port-sharing

If true, SO_REUSEPORT will be used when binding the port, which allows more than one instance to bind on the same address and port. [default=false]

--profiling     Default: true

Enable profiling via web interface host:port/debug/pprof/

--proxy-client-cert-file string

Client certificate used to prove the identity of the aggregator or kube-apiserver when it must call out during a request. This includes proxying requests to a user api-server and calling out to webhook admission plugins. It is expected that this cert includes a signature from the CA in the --requestheader-client-ca-file flag. That CA is published in the 'extension-apiserver-authentication' configmap in the kube-system namespace. Components receiving calls from kube-aggregator should use that CA to perform their half of the mutual TLS verification.

--proxy-client-key-file string

Private key for the client certificate used to prove the identity of the aggregator or kube-apiserver when it must call out during a request. This includes proxying requests to a user api-server and calling out to webhook admission plugins.

--request-timeout duration     Default: 1m0s

An optional field indicating the duration a handler must keep a request open before timing it out. This is the default request timeout for requests but may be overridden by flags such as --min-request-timeout for specific types of requests.

--requestheader-allowed-names strings

List of client certificate common names to allow to provide usernames in headers specified by --requestheader-username-headers. If empty, any client certificate validated by the authorities in --requestheader-client-ca-file is allowed.

--requestheader-client-ca-file string

Root certificate bundle to use to verify client certificates on incoming requests before trusting usernames in headers specified by --requestheader-username-headers. WARNING: generally do not depend on authorization being already done for incoming requests.

--requestheader-extra-headers-prefix strings

List of request header prefixes to inspect. X-Remote-Extra- is suggested.

--requestheader-group-headers strings

List of request headers to inspect for groups. X-Remote-Group is suggested.

--requestheader-username-headers strings

List of request headers to inspect for usernames. X-Remote-User is common.

--runtime-config <comma-separated 'key=value' pairs>

A set of key=value pairs that enable or disable built-in APIs. Supported options are:
v1=true|false for the core API group
<group>/<version>=true|false for a specific API group and version (e.g. apps/v1=true)
api/all=true|false controls all API versions
api/ga=true|false controls all API versions of the form v[0-9]+
api/beta=true|false controls all API versions of the form v[0-9]+beta[0-9]+
api/alpha=true|false controls all API versions of the form v[0-9]+alpha[0-9]+
api/legacy is deprecated, and will be removed in a future version

--secure-port int     Default: 6443

The port on which to serve HTTPS with authentication and authorization. It cannot be switched off with 0.

--service-account-extend-token-expiration     Default: true

Turns on projected service account expiration extension during token generation, which helps safe transition from legacy token to bound service account token feature. If this flag is enabled, admission injected tokens would be extended up to 1 year to prevent unexpected failure during transition, ignoring value of service-account-max-token-expiration.

--service-account-issuer strings

Identifier of the service account token issuer. The issuer will assert this identifier in "iss" claim of issued tokens. This value is a string or URI. If this option is not a valid URI per the OpenID Discovery 1.0 spec, the ServiceAccountIssuerDiscovery feature will remain disabled, even if the feature gate is set to true. It is highly recommended that this value comply with the OpenID spec: https://openid.net/specs/openid-connect-discovery-1_0.html. In practice, this means that service-account-issuer must be an https URL. It is also highly recommended that this URL be capable of serving OpenID discovery documents at {service-account-issuer}/.well-known/openid-configuration. When this flag is specified multiple times, the first is used to generate tokens and all are used to determine which issuers are accepted.

--service-account-jwks-uri string

Overrides the URI for the JSON Web Key Set in the discovery doc served at /.well-known/openid-configuration. This flag is useful if the discovery docand key set are served to relying parties from a URL other than the API server's external (as auto-detected or overridden with external-hostname).

--service-account-key-file strings

File containing PEM-encoded x509 RSA or ECDSA private or public keys, used to verify ServiceAccount tokens. The specified file can contain multiple keys, and the flag can be specified multiple times with different files. If unspecified, --tls-private-key-file is used. Must be specified when --service-account-signing-key-file is provided

--service-account-lookup     Default: true

If true, validate ServiceAccount tokens exist in etcd as part of authentication.

--service-account-max-token-expiration duration

The maximum validity duration of a token created by the service account token issuer. If an otherwise valid TokenRequest with a validity duration larger than this value is requested, a token will be issued with a validity duration of this value.

--service-account-signing-key-file string

Path to the file that contains the current private key of the service account token issuer. The issuer will sign issued ID tokens with this private key.

--service-cluster-ip-range string

A CIDR notation IP range from which to assign service cluster IPs. This must not overlap with any IP ranges assigned to nodes or pods. Max of two dual-stack CIDRs is allowed.

--service-node-port-range <a string in the form 'N1-N2'>     Default: 30000-32767

A port range to reserve for services with NodePort visibility. This must not overlap with the ephemeral port range on nodes. Example: '30000-32767'. Inclusive at both ends of the range.

--show-hidden-metrics-for-version string

The previous version for which you want to show hidden metrics. Only the previous minor version is meaningful, other values will not be allowed. The format is <major>.<minor>, e.g.: '1.16'. The purpose of this format is make sure you have the opportunity to notice if the next release hides additional metrics, rather than being surprised when they are permanently removed in the release after that.

--shutdown-delay-duration duration

Time to delay the termination. During that time the server keeps serving requests normally. The endpoints /healthz and /livez will return success, but /readyz immediately returns failure. Graceful termination starts after this delay has elapsed. This can be used to allow load balancer to stop sending traffic to this server.

--shutdown-send-retry-after

If true the HTTP Server will continue listening until all non long running request(s) in flight have been drained, during this window all incoming requests will be rejected with a status code 429 and a 'Retry-After' response header, in addition 'Connection: close' response header is set in order to tear down the TCP connection when idle.

--shutdown-watch-termination-grace-period duration

This option, if set, represents the maximum amount of grace period the apiserver will wait for active watch request(s) to drain during the graceful server shutdown window.

--storage-backend string

The storage backend for persistence. Options: 'etcd3' (default).

--storage-media-type string     Default: "application/vnd.kubernetes.protobuf"

The media type to use to store objects in storage. Some resources or storage backends may only support a specific media type and will ignore this setting. Supported media types: [application/json, application/yaml, application/vnd.kubernetes.protobuf]

--strict-transport-security-directives strings

List of directives for HSTS, comma separated. If this list is empty, then HSTS directives will not be added. Example: 'max-age=31536000,includeSubDomains,preload'

--tls-cert-file string

File containing the default x509 Certificate for HTTPS. (CA cert, if any, concatenated after server cert). If HTTPS serving is enabled, and --tls-cert-file and --tls-private-key-file are not provided, a self-signed certificate and key are generated for the public address and saved to the directory specified by --cert-dir.

--tls-cipher-suites strings

Comma-separated list of cipher suites for the server. If omitted, the default Go cipher suites will be used.
Preferred values: TLS_AES_128_GCM_SHA256, TLS_AES_256_GCM_SHA384, TLS_CHACHA20_POLY1305_SHA256, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305, TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305, TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256, TLS_RSA_WITH_AES_128_CBC_SHA, TLS_RSA_WITH_AES_128_GCM_SHA256, TLS_RSA_WITH_AES_256_CBC_SHA, TLS_RSA_WITH_AES_256_GCM_SHA384.
Insecure values: TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_ECDSA_WITH_RC4_128_SHA, TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_RSA_WITH_RC4_128_SHA, TLS_RSA_WITH_3DES_EDE_CBC_SHA, TLS_RSA_WITH_AES_128_CBC_SHA256, TLS_RSA_WITH_RC4_128_SHA.

--tls-min-version string

Minimum TLS version supported. Possible values: VersionTLS10, VersionTLS11, VersionTLS12, VersionTLS13

--tls-private-key-file string

File containing the default x509 private key matching --tls-cert-file.

--tls-sni-cert-key string

A pair of x509 certificate and private key file paths, optionally suffixed with a list of domain patterns which are fully qualified domain names, possibly with prefixed wildcard segments. The domain patterns also allow IP addresses, but IPs should only be used if the apiserver has visibility to the IP address requested by a client. If no domain patterns are provided, the names of the certificate are extracted. Non-wildcard matches trump over wildcard matches, explicit domain patterns trump over extracted names. For multiple key/certificate pairs, use the --tls-sni-cert-key multiple times. Examples: "example.crt,example.key" or "foo.crt,foo.key:*.foo.com,foo.com".

--token-auth-file string

If set, the file that will be used to secure the secure port of the API server via token authentication.

--tracing-config-file string

File with apiserver tracing configuration.

-v, --v int

number for the log level verbosity

--version version[=true]

--version, --version=raw prints version information and quits; --version=vX.Y.Z... sets the reported version

--vmodule pattern=N,...

comma-separated list of pattern=N settings for file-filtered logging (only works for text log format)

--watch-cache     Default: true

Enable watch caching in the apiserver

--watch-cache-sizes strings

Watch cache size settings for some resources (pods, nodes, etc.), comma separated. The individual setting format: resource[.group]#size, where resource is lowercase plural (no version), group is omitted for resources of apiVersion v1 (the legacy core API) and included for others, and size is a number. This option is only meaningful for resources built into the apiserver, not ones defined by CRDs or aggregated from external servers, and is only consulted if the watch-cache is enabled. The only meaningful size setting to supply here is zero, which means to disable watch caching for the associated resource; all non-zero values are equivalent and mean to not disable watch caching for that resource

5 - kube-controller-manager

Synopsis

The Kubernetes controller manager is a daemon that embeds the core control loops shipped with Kubernetes. In applications of robotics and automation, a control loop is a non-terminating loop that regulates the state of the system. In Kubernetes, a controller is a control loop that watches the shared state of the cluster through the apiserver and makes changes attempting to move the current state towards the desired state. Examples of controllers that ship with Kubernetes today are the replication controller, endpoints controller, namespace controller, and serviceaccounts controller.

kube-controller-manager [flags]

Options

--allocate-node-cidrs

Should CIDRs for Pods be allocated and set on the cloud provider.

--allow-metric-labels stringToString     Default: []

The map from metric-label to value allow-list of this label. The key's format is <MetricName>,<LabelName>. The value's format is <allowed_value>,<allowed_value>...e.g. metric1,label1='v1,v2,v3', metric1,label2='v1,v2,v3' metric2,label1='v1,v2,v3'.

--allow-metric-labels-manifest string

The path to the manifest file that contains the allow-list mapping. The format of the file is the same as the flag --allow-metric-labels. Note that the flag --allow-metric-labels will override the manifest file.

--attach-detach-reconcile-sync-period duration     Default: 1m0s

The reconciler sync wait time between volume attach detach. This duration must be larger than one second, and increasing this value from the default may allow for volumes to be mismatched with pods.

--authentication-kubeconfig string

kubeconfig file pointing at the 'core' kubernetes server with enough rights to create tokenreviews.authentication.k8s.io. This is optional. If empty, all token requests are considered to be anonymous and no client CA is looked up in the cluster.

--authentication-skip-lookup

If false, the authentication-kubeconfig will be used to lookup missing authentication configuration from the cluster.

--authentication-token-webhook-cache-ttl duration     Default: 10s

The duration to cache responses from the webhook token authenticator.

--authentication-tolerate-lookup-failure

If true, failures to look up missing authentication configuration from the cluster are not considered fatal. Note that this can result in authentication that treats all requests as anonymous.

--authorization-always-allow-paths strings     Default: "/healthz,/readyz,/livez"

A list of HTTP paths to skip during authorization, i.e. these are authorized without contacting the 'core' kubernetes server.

--authorization-kubeconfig string

kubeconfig file pointing at the 'core' kubernetes server with enough rights to create subjectaccessreviews.authorization.k8s.io. This is optional. If empty, all requests not skipped by authorization are forbidden.

--authorization-webhook-cache-authorized-ttl duration     Default: 10s

The duration to cache 'authorized' responses from the webhook authorizer.

--authorization-webhook-cache-unauthorized-ttl duration     Default: 10s

The duration to cache 'unauthorized' responses from the webhook authorizer.

--azure-container-registry-config string

Path to the file containing Azure container registry configuration information.

--bind-address string     Default: 0.0.0.0

The IP address on which to listen for the --secure-port port. The associated interface(s) must be reachable by the rest of the cluster, and by CLI/web clients. If blank or an unspecified address (0.0.0.0 or ::), all interfaces and IP address families will be used.

--cert-dir string

The directory where the TLS certs are located. If --tls-cert-file and --tls-private-key-file are provided, this flag will be ignored.

--cidr-allocator-type string     Default: "RangeAllocator"

Type of CIDR allocator to use

--client-ca-file string

If set, any request presenting a client certificate signed by one of the authorities in the client-ca-file is authenticated with an identity corresponding to the CommonName of the client certificate.

--cloud-config string

The path to the cloud provider configuration file. Empty string for no configuration file.

--cloud-provider string

The provider for cloud services. Empty string for no provider.

--cluster-cidr string

CIDR Range for Pods in cluster. Requires --allocate-node-cidrs to be true

--cluster-name string     Default: "kubernetes"

The instance prefix for the cluster.

--cluster-signing-cert-file string

Filename containing a PEM-encoded X509 CA certificate used to issue cluster-scoped certificates. If specified, no more specific --cluster-signing-* flag may be specified.

--cluster-signing-duration duration     Default: 8760h0m0s

The max length of duration signed certificates will be given. Individual CSRs may request shorter certs by setting spec.expirationSeconds.

--cluster-signing-key-file string

Filename containing a PEM-encoded RSA or ECDSA private key used to sign cluster-scoped certificates. If specified, no more specific --cluster-signing-* flag may be specified.

--cluster-signing-kube-apiserver-client-cert-file string

Filename containing a PEM-encoded X509 CA certificate used to issue certificates for the kubernetes.io/kube-apiserver-client signer. If specified, --cluster-signing-{cert,key}-file must not be set.

--cluster-signing-kube-apiserver-client-key-file string

Filename containing a PEM-encoded RSA or ECDSA private key used to sign certificates for the kubernetes.io/kube-apiserver-client signer. If specified, --cluster-signing-{cert,key}-file must not be set.

--cluster-signing-kubelet-client-cert-file string

Filename containing a PEM-encoded X509 CA certificate used to issue certificates for the kubernetes.io/kube-apiserver-client-kubelet signer. If specified, --cluster-signing-{cert,key}-file must not be set.

--cluster-signing-kubelet-client-key-file string

Filename containing a PEM-encoded RSA or ECDSA private key used to sign certificates for the kubernetes.io/kube-apiserver-client-kubelet signer. If specified, --cluster-signing-{cert,key}-file must not be set.

--cluster-signing-kubelet-serving-cert-file string

Filename containing a PEM-encoded X509 CA certificate used to issue certificates for the kubernetes.io/kubelet-serving signer. If specified, --cluster-signing-{cert,key}-file must not be set.

--cluster-signing-kubelet-serving-key-file string

Filename containing a PEM-encoded RSA or ECDSA private key used to sign certificates for the kubernetes.io/kubelet-serving signer. If specified, --cluster-signing-{cert,key}-file must not be set.

--cluster-signing-legacy-unknown-cert-file string

Filename containing a PEM-encoded X509 CA certificate used to issue certificates for the kubernetes.io/legacy-unknown signer. If specified, --cluster-signing-{cert,key}-file must not be set.

--cluster-signing-legacy-unknown-key-file string

Filename containing a PEM-encoded RSA or ECDSA private key used to sign certificates for the kubernetes.io/legacy-unknown signer. If specified, --cluster-signing-{cert,key}-file must not be set.

--concurrent-cron-job-syncs int32     Default: 5

The number of cron job objects that are allowed to sync concurrently. Larger number = more responsive jobs, but more CPU (and network) load

--concurrent-deployment-syncs int32     Default: 5

The number of deployment objects that are allowed to sync concurrently. Larger number = more responsive deployments, but more CPU (and network) load

--concurrent-endpoint-syncs int32     Default: 5

The number of endpoint syncing operations that will be done concurrently. Larger number = faster endpoint updating, but more CPU (and network) load

--concurrent-ephemeralvolume-syncs int32     Default: 5

The number of ephemeral volume syncing operations that will be done concurrently. Larger number = faster ephemeral volume updating, but more CPU (and network) load

--concurrent-gc-syncs int32     Default: 20

The number of garbage collector workers that are allowed to sync concurrently.

--concurrent-horizontal-pod-autoscaler-syncs int32     Default: 5

The number of horizontal pod autoscaler objects that are allowed to sync concurrently. Larger number = more responsive horizontal pod autoscaler objects processing, but more CPU (and network) load.

--concurrent-job-syncs int32     Default: 5

The number of job objects that are allowed to sync concurrently. Larger number = more responsive jobs, but more CPU (and network) load

--concurrent-namespace-syncs int32     Default: 10

The number of namespace objects that are allowed to sync concurrently. Larger number = more responsive namespace termination, but more CPU (and network) load

--concurrent-rc-syncs int32     Default: 5

The number of replication controllers that are allowed to sync concurrently. Larger number = more responsive replica management, but more CPU (and network) load

--concurrent-replicaset-syncs int32     Default: 5

The number of replica sets that are allowed to sync concurrently. Larger number = more responsive replica management, but more CPU (and network) load

--concurrent-resource-quota-syncs int32     Default: 5

The number of resource quotas that are allowed to sync concurrently. Larger number = more responsive quota management, but more CPU (and network) load

--concurrent-service-endpoint-syncs int32     Default: 5

The number of service endpoint syncing operations that will be done concurrently. Larger number = faster endpoint slice updating, but more CPU (and network) load. Defaults to 5.

--concurrent-service-syncs int32     Default: 1

The number of services that are allowed to sync concurrently. Larger number = more responsive service management, but more CPU (and network) load

--concurrent-serviceaccount-token-syncs int32     Default: 5

The number of service account token objects that are allowed to sync concurrently. Larger number = more responsive token generation, but more CPU (and network) load

--concurrent-statefulset-syncs int32     Default: 5

The number of statefulset objects that are allowed to sync concurrently. Larger number = more responsive statefulsets, but more CPU (and network) load

--concurrent-ttl-after-finished-syncs int32     Default: 5

The number of ttl-after-finished-controller workers that are allowed to sync concurrently.

--concurrent-validating-admission-policy-status-syncs int32     Default: 5

The number of ValidatingAdmissionPolicyStatusController workers that are allowed to sync concurrently.

--configure-cloud-routes     Default: true

Should CIDRs allocated by allocate-node-cidrs be configured on the cloud provider.

--contention-profiling

Enable block profiling, if profiling is enabled

--controller-start-interval duration

Interval between starting controller managers.

--controllers strings     Default: "*"

A list of controllers to enable. '*' enables all on-by-default controllers, 'foo' enables the controller named 'foo', '-foo' disables the controller named 'foo'.
All controllers: bootstrap-signer-controller, certificatesigningrequest-approving-controller, certificatesigningrequest-cleaner-controller, certificatesigningrequest-signing-controller, cloud-node-lifecycle-controller, clusterrole-aggregation-controller, cronjob-controller, daemonset-controller, deployment-controller, disruption-controller, endpoints-controller, endpointslice-controller, endpointslice-mirroring-controller, ephemeral-volume-controller, garbage-collector-controller, horizontal-pod-autoscaler-controller, job-controller, legacy-serviceaccount-token-cleaner-controller, namespace-controller, node-ipam-controller, node-lifecycle-controller, node-route-controller, persistentvolume-attach-detach-controller, persistentvolume-binder-controller, persistentvolume-expander-controller, persistentvolume-protection-controller, persistentvolumeclaim-protection-controller, pod-garbage-collector-controller, replicaset-controller, replicationcontroller-controller, resourceclaim-controller, resourcequota-controller, root-ca-certificate-publisher-controller, service-cidr-controller, service-lb-controller, serviceaccount-controller, serviceaccount-token-controller, statefulset-controller, storageversion-garbage-collector-controller, taint-eviction-controller, token-cleaner-controller, ttl-after-finished-controller, ttl-controller, validatingadmissionpolicy-status-controller
Disabled-by-default controllers: bootstrap-signer-controller, token-cleaner-controller

--disable-attach-detach-reconcile-sync

Disable volume attach detach reconciler sync. Disabling this may cause volumes to be mismatched with pods. Use wisely.

--disabled-metrics strings

This flag provides an escape hatch for misbehaving metrics. You must provide the fully qualified metric name in order to disable it. Disclaimer: disabling metrics is higher in precedence than showing hidden metrics.

--enable-dynamic-provisioning     Default: true

Enable dynamic provisioning for environments that support it.

--enable-garbage-collector     Default: true

Enables the generic garbage collector. MUST be synced with the corresponding flag of the kube-apiserver.

--enable-hostpath-provisioner

Enable HostPath PV provisioning when running without a cloud provider. This allows testing and development of provisioning features. HostPath provisioning is not supported in any way, won't work in a multi-node cluster, and should not be used for anything other than testing or development.

--enable-leader-migration

Whether to enable controller leader migration.

--endpoint-updates-batch-period duration

The length of endpoint updates batching period. Processing of pod changes will be delayed by this duration to join them with potential upcoming updates and reduce the overall number of endpoints updates. Larger number = higher endpoint programming latency, but lower number of endpoints revision generated

--endpointslice-updates-batch-period duration

The length of endpoint slice updates batching period. Processing of pod changes will be delayed by this duration to join them with potential upcoming updates and reduce the overall number of endpoints updates. Larger number = higher endpoint programming latency, but lower number of endpoints revision generated

--external-cloud-volume-plugin string

The plugin to use when cloud provider is set to external. Can be empty, should only be set when cloud-provider is external. Currently used to allow node-ipam-controller, persistentvolume-binder-controller, persistentvolume-expander-controller and attach-detach-controller to work for in tree cloud providers.

--feature-gates <comma-separated 'key=True|False' pairs>

A set of key=value pairs that describe feature gates for alpha/experimental features. Options are:
APIResponseCompression=true|false (BETA - default=true)
APIServerIdentity=true|false (BETA - default=true)
APIServerTracing=true|false (BETA - default=true)
AdmissionWebhookMatchConditions=true|false (BETA - default=true)
AggregatedDiscoveryEndpoint=true|false (BETA - default=true)
AllAlpha=true|false (ALPHA - default=false)
AllBeta=true|false (BETA - default=false)
AnyVolumeDataSource=true|false (BETA - default=true)
AppArmor=true|false (BETA - default=true)
CPUManagerPolicyAlphaOptions=true|false (ALPHA - default=false)
CPUManagerPolicyBetaOptions=true|false (BETA - default=true)
CPUManagerPolicyOptions=true|false (BETA - default=true)
CRDValidationRatcheting=true|false (ALPHA - default=false)
CSIMigrationPortworx=true|false (BETA - default=false)
CSIVolumeHealth=true|false (ALPHA - default=false)
CloudControllerManagerWebhook=true|false (ALPHA - default=false)
CloudDualStackNodeIPs=true|false (BETA - default=true)
ClusterTrustBundle=true|false (ALPHA - default=false)
ClusterTrustBundleProjection=true|false (ALPHA - default=false)
ComponentSLIs=true|false (BETA - default=true)
ConsistentListFromCache=true|false (ALPHA - default=false)
ContainerCheckpoint=true|false (ALPHA - default=false)
ContextualLogging=true|false (ALPHA - default=false)
CronJobsScheduledAnnotation=true|false (BETA - default=true)
CrossNamespaceVolumeDataSource=true|false (ALPHA - default=false)
CustomCPUCFSQuotaPeriod=true|false (ALPHA - default=false)
DevicePluginCDIDevices=true|false (BETA - default=true)
DisableCloudProviders=true|false (BETA - default=true)
DisableKubeletCloudCredentialProviders=true|false (BETA - default=true)
DisableNodeKubeProxyVersion=true|false (ALPHA - default=false)
DynamicResourceAllocation=true|false (ALPHA - default=false)
ElasticIndexedJob=true|false (BETA - default=true)
EventedPLEG=true|false (BETA - default=false)
GracefulNodeShutdown=true|false (BETA - default=true)
GracefulNodeShutdownBasedOnPodPriority=true|false (BETA - default=true)
HPAContainerMetrics=true|false (BETA - default=true)
HPAScaleToZero=true|false (ALPHA - default=false)
HonorPVReclaimPolicy=true|false (ALPHA - default=false)
ImageMaximumGCAge=true|false (ALPHA - default=false)
InPlacePodVerticalScaling=true|false (ALPHA - default=false)
InTreePluginAWSUnregister=true|false (ALPHA - default=false)
InTreePluginAzureDiskUnregister=true|false (ALPHA - default=false)
InTreePluginAzureFileUnregister=true|false (ALPHA - default=false)
InTreePluginGCEUnregister=true|false (ALPHA - default=false)
InTreePluginOpenStackUnregister=true|false (ALPHA - default=false)
InTreePluginPortworxUnregister=true|false (ALPHA - default=false)
InTreePluginvSphereUnregister=true|false (ALPHA - default=false)
JobBackoffLimitPerIndex=true|false (BETA - default=true)
JobPodFailurePolicy=true|false (BETA - default=true)
JobPodReplacementPolicy=true|false (BETA - default=true)
KubeProxyDrainingTerminatingNodes=true|false (ALPHA - default=false)
KubeletCgroupDriverFromCRI=true|false (ALPHA - default=false)
KubeletInUserNamespace=true|false (ALPHA - default=false)
KubeletPodResourcesDynamicResources=true|false (ALPHA - default=false)
KubeletPodResourcesGet=true|false (ALPHA - default=false)
KubeletSeparateDiskGC=true|false (ALPHA - default=false)
KubeletTracing=true|false (BETA - default=true)
LegacyServiceAccountTokenCleanUp=true|false (BETA - default=true)
LoadBalancerIPMode=true|false (ALPHA - default=false)
LocalStorageCapacityIsolationFSQuotaMonitoring=true|false (ALPHA - default=false)
LogarithmicScaleDown=true|false (BETA - default=true)
LoggingAlphaOptions=true|false (ALPHA - default=false)
LoggingBetaOptions=true|false (BETA - default=true)
MatchLabelKeysInPodAffinity=true|false (ALPHA - default=false)
MatchLabelKeysInPodTopologySpread=true|false (BETA - default=true)
MaxUnavailableStatefulSet=true|false (ALPHA - default=false)
MemoryManager=true|false (BETA - default=true)
MemoryQoS=true|false (ALPHA - default=false)
MinDomainsInPodTopologySpread=true|false (BETA - default=true)
MultiCIDRServiceAllocator=true|false (ALPHA - default=false)
NFTablesProxyMode=true|false (ALPHA - default=false)
NewVolumeManagerReconstruction=true|false (BETA - default=true)
NodeInclusionPolicyInPodTopologySpread=true|false (BETA - default=true)
NodeLogQuery=true|false (ALPHA - default=false)
NodeSwap=true|false (BETA - default=false)
OpenAPIEnums=true|false (BETA - default=true)
PDBUnhealthyPodEvictionPolicy=true|false (BETA - default=true)
PersistentVolumeLastPhaseTransitionTime=true|false (BETA - default=true)
PodAndContainerStatsFromCRI=true|false (ALPHA - default=false)
PodDeletionCost=true|false (BETA - default=true)
PodDisruptionConditions=true|false (BETA - default=true)
PodHostIPs=true|false (BETA - default=true)
PodIndexLabel=true|false (BETA - default=true)
PodLifecycleSleepAction=true|false (ALPHA - default=false)
PodReadyToStartContainersCondition=true|false (BETA - default=true)
PodSchedulingReadiness=true|false (BETA - default=true)
ProcMountType=true|false (ALPHA - default=false)
QOSReserved=true|false (ALPHA - default=false)
RecoverVolumeExpansionFailure=true|false (ALPHA - default=false)
RotateKubeletServerCertificate=true|false (BETA - default=true)
RuntimeClassInImageCriApi=true|false (ALPHA - default=false)
SELinuxMountReadWriteOncePod=true|false (BETA - default=true)
SchedulerQueueingHints=true|false (BETA - default=false)
SecurityContextDeny=true|false (ALPHA - default=false)
SeparateTaintEvictionController=true|false (BETA - default=true)
ServiceAccountTokenJTI=true|false (ALPHA - default=false)
ServiceAccountTokenNodeBinding=true|false (ALPHA - default=false)
ServiceAccountTokenNodeBindingValidation=true|false (ALPHA - default=false)
ServiceAccountTokenPodNodeInfo=true|false (ALPHA - default=false)
SidecarContainers=true|false (BETA - default=true)
SizeMemoryBackedVolumes=true|false (BETA - default=true)
StableLoadBalancerNodeSet=true|false (BETA - default=true)
StatefulSetAutoDeletePVC=true|false (BETA - default=true)
StatefulSetStartOrdinal=true|false (BETA - default=true)
StorageVersionAPI=true|false (ALPHA - default=false)
StorageVersionHash=true|false (BETA - default=true)
StructuredAuthenticationConfiguration=true|false (ALPHA - default=false)
StructuredAuthorizationConfiguration=true|false (ALPHA - default=false)
TopologyAwareHints=true|false (BETA - default=true)
TopologyManagerPolicyAlphaOptions=true|false (ALPHA - default=false)
TopologyManagerPolicyBetaOptions=true|false (BETA - default=true)
TopologyManagerPolicyOptions=true|false (BETA - default=true)
TranslateStreamCloseWebsocketRequests=true|false (ALPHA - default=false)
UnauthenticatedHTTP2DOSMitigation=true|false (BETA - default=true)
UnknownVersionInteroperabilityProxy=true|false (ALPHA - default=false)
UserNamespacesPodSecurityStandards=true|false (ALPHA - default=false)
UserNamespacesSupport=true|false (ALPHA - default=false)
ValidatingAdmissionPolicy=true|false (BETA - default=false)
VolumeAttributesClass=true|false (ALPHA - default=false)
VolumeCapacityPriority=true|false (ALPHA - default=false)
WatchList=true|false (ALPHA - default=false)
WinDSR=true|false (ALPHA - default=false)
WinOverlay=true|false (BETA - default=true)
WindowsHostNetwork=true|false (ALPHA - default=true)
ZeroLimitedNominalConcurrencyShares=true|false (BETA - default=false)

--flex-volume-plugin-dir string     Default: "/usr/libexec/kubernetes/kubelet-plugins/volume/exec/"

Full path of the directory in which the flex volume plugin should search for additional third party volume plugins.

-h, --help

help for kube-controller-manager

--horizontal-pod-autoscaler-cpu-initialization-period duration     Default: 5m0s

The period after pod start when CPU samples might be skipped.

--horizontal-pod-autoscaler-downscale-stabilization duration     Default: 5m0s

The period for which autoscaler will look backwards and not scale down below any recommendation it made during that period.

--horizontal-pod-autoscaler-initial-readiness-delay duration     Default: 30s

The period after pod start during which readiness changes will be treated as initial readiness.

--horizontal-pod-autoscaler-sync-period duration     Default: 15s

The period for syncing the number of pods in horizontal pod autoscaler.

--horizontal-pod-autoscaler-tolerance float     Default: 0.1

The minimum change (from 1.0) in the desired-to-actual metrics ratio for the horizontal pod autoscaler to consider scaling.

--http2-max-streams-per-connection int

The limit that the server gives to clients for the maximum number of streams in an HTTP/2 connection. Zero means to use golang's default.

--kube-api-burst int32     Default: 30

Burst to use while talking with kubernetes apiserver.

--kube-api-content-type string     Default: "application/vnd.kubernetes.protobuf"

Content type of requests sent to apiserver.

--kube-api-qps float     Default: 20

QPS to use while talking with kubernetes apiserver.

--kubeconfig string

Path to kubeconfig file with authorization and master location information (the master location can be overridden by the master flag).

--large-cluster-size-threshold int32     Default: 50

Number of nodes from which node-lifecycle-controller treats the cluster as large for the eviction logic purposes. --secondary-node-eviction-rate is implicitly overridden to 0 for clusters this size or smaller. Notice: If nodes reside in multiple zones, this threshold will be considered as zone node size threshold for each zone to determine node eviction rate independently.

--leader-elect     Default: true

Start a leader election client and gain leadership before executing the main loop. Enable this when running replicated components for high availability.

--leader-elect-lease-duration duration     Default: 15s

The duration that non-leader candidates will wait after observing a leadership renewal until attempting to acquire leadership of a led but unrenewed leader slot. This is effectively the maximum duration that a leader can be stopped before it is replaced by another candidate. This is only applicable if leader election is enabled.

--leader-elect-renew-deadline duration     Default: 10s

The interval between attempts by the acting master to renew a leadership slot before it stops leading. This must be less than the lease duration. This is only applicable if leader election is enabled.

--leader-elect-resource-lock string     Default: "leases"

The type of resource object that is used for locking during leader election. Supported options are 'leases', 'endpointsleases' and 'configmapsleases'.

--leader-elect-resource-name string     Default: "kube-controller-manager"

The name of resource object that is used for locking during leader election.

--leader-elect-resource-namespace string     Default: "kube-system"

The namespace of resource object that is used for locking during leader election.

--leader-elect-retry-period duration     Default: 2s

The duration the clients should wait between attempting acquisition and renewal of a leadership. This is only applicable if leader election is enabled.

--leader-migration-config string

Path to the config file for controller leader migration, or empty to use the value that reflects default configuration of the controller manager. The config file should be of type LeaderMigrationConfiguration, group controllermanager.config.k8s.io, version v1alpha1.

--legacy-service-account-token-clean-up-period duration     Default: 8760h0m0s

The period of time since the last usage of an legacy service account token before it can be deleted.

--log-flush-frequency duration     Default: 5s

Maximum number of seconds between log flushes

--logging-format string     Default: "text"

Sets the log format. Permitted formats: "text".

--master string

The address of the Kubernetes API server (overrides any value in kubeconfig).

--max-endpoints-per-slice int32     Default: 100

The maximum number of endpoints that will be added to an EndpointSlice. More endpoints per slice will result in less endpoint slices, but larger resources. Defaults to 100.

--min-resync-period duration     Default: 12h0m0s

The resync period in reflectors will be random between MinResyncPeriod and 2*MinResyncPeriod.

--mirroring-concurrent-service-endpoint-syncs int32     Default: 5

The number of service endpoint syncing operations that will be done concurrently by the endpointslice-mirroring-controller. Larger number = faster endpoint slice updating, but more CPU (and network) load. Defaults to 5.

--mirroring-endpointslice-updates-batch-period duration

The length of EndpointSlice updates batching period for endpointslice-mirroring-controller. Processing of EndpointSlice changes will be delayed by this duration to join them with potential upcoming updates and reduce the overall number of EndpointSlice updates. Larger number = higher endpoint programming latency, but lower number of endpoints revision generated

--mirroring-max-endpoints-per-subset int32     Default: 1000

The maximum number of endpoints that will be added to an EndpointSlice by the endpointslice-mirroring-controller. More endpoints per slice will result in less endpoint slices, but larger resources. Defaults to 100.

--namespace-sync-period duration     Default: 5m0s

The period for syncing namespace life-cycle updates

--node-cidr-mask-size int32

Mask size for node cidr in cluster. Default is 24 for IPv4 and 64 for IPv6.

--node-cidr-mask-size-ipv4 int32

Mask size for IPv4 node cidr in dual-stack cluster. Default is 24.

--node-cidr-mask-size-ipv6 int32

Mask size for IPv6 node cidr in dual-stack cluster. Default is 64.

--node-eviction-rate float     Default: 0.1

Number of nodes per second on which pods are deleted in case of node failure when a zone is healthy (see --unhealthy-zone-threshold for definition of healthy/unhealthy). Zone refers to entire cluster in non-multizone clusters.

--node-monitor-grace-period duration     Default: 40s

Amount of time which we allow running Node to be unresponsive before marking it unhealthy. Must be N times more than kubelet's nodeStatusUpdateFrequency, where N means number of retries allowed for kubelet to post node status.

--node-monitor-period duration     Default: 5s

The period for syncing NodeStatus in cloud-node-lifecycle-controller.

--node-startup-grace-period duration     Default: 1m0s

Amount of time which we allow starting Node to be unresponsive before marking it unhealthy.

--permit-address-sharing

If true, SO_REUSEADDR will be used when binding the port. This allows binding to wildcard IPs like 0.0.0.0 and specific IPs in parallel, and it avoids waiting for the kernel to release sockets in TIME_WAIT state. [default=false]

--permit-port-sharing

If true, SO_REUSEPORT will be used when binding the port, which allows more than one instance to bind on the same address and port. [default=false]

--profiling     Default: true

Enable profiling via web interface host:port/debug/pprof/

--pv-recycler-increment-timeout-nfs int32     Default: 30

the increment of time added per Gi to ActiveDeadlineSeconds for an NFS scrubber pod

--pv-recycler-minimum-timeout-hostpath int32     Default: 60

The minimum ActiveDeadlineSeconds to use for a HostPath Recycler pod. This is for development and testing only and will not work in a multi-node cluster.

--pv-recycler-minimum-timeout-nfs int32     Default: 300

The minimum ActiveDeadlineSeconds to use for an NFS Recycler pod

--pv-recycler-pod-template-filepath-hostpath string

The file path to a pod definition used as a template for HostPath persistent volume recycling. This is for development and testing only and will not work in a multi-node cluster.

--pv-recycler-pod-template-filepath-nfs string

The file path to a pod definition used as a template for NFS persistent volume recycling

--pv-recycler-timeout-increment-hostpath int32     Default: 30

the increment of time added per Gi to ActiveDeadlineSeconds for a HostPath scrubber pod. This is for development and testing only and will not work in a multi-node cluster.

--pvclaimbinder-sync-period duration     Default: 15s

The period for syncing persistent volumes and persistent volume claims

--requestheader-allowed-names strings

List of client certificate common names to allow to provide usernames in headers specified by --requestheader-username-headers. If empty, any client certificate validated by the authorities in --requestheader-client-ca-file is allowed.

--requestheader-client-ca-file string

Root certificate bundle to use to verify client certificates on incoming requests before trusting usernames in headers specified by --requestheader-username-headers. WARNING: generally do not depend on authorization being already done for incoming requests.

--requestheader-extra-headers-prefix strings     Default: "x-remote-extra-"

List of request header prefixes to inspect. X-Remote-Extra- is suggested.

--requestheader-group-headers strings     Default: "x-remote-group"

List of request headers to inspect for groups. X-Remote-Group is suggested.

--requestheader-username-headers strings     Default: "x-remote-user"

List of request headers to inspect for usernames. X-Remote-User is common.

--resource-quota-sync-period duration     Default: 5m0s

The period for syncing quota usage status in the system

--root-ca-file string

If set, this root certificate authority will be included in service account's token secret. This must be a valid PEM-encoded CA bundle.

--route-reconciliation-period duration     Default: 10s

The period for reconciling routes created for Nodes by cloud provider.

--secondary-node-eviction-rate float     Default: 0.01

Number of nodes per second on which pods are deleted in case of node failure when a zone is unhealthy (see --unhealthy-zone-threshold for definition of healthy/unhealthy). Zone refers to entire cluster in non-multizone clusters. This value is implicitly overridden to 0 if the cluster size is smaller than --large-cluster-size-threshold.

--secure-port int     Default: 10257

The port on which to serve HTTPS with authentication and authorization. If 0, don't serve HTTPS at all.

--service-account-private-key-file string

Filename containing a PEM-encoded private RSA or ECDSA key used to sign service account tokens.

--service-cluster-ip-range string

CIDR Range for Services in cluster. Requires --allocate-node-cidrs to be true

--show-hidden-metrics-for-version string

The previous version for which you want to show hidden metrics. Only the previous minor version is meaningful, other values will not be allowed. The format is <major>.<minor>, e.g.: '1.16'. The purpose of this format is make sure you have the opportunity to notice if the next release hides additional metrics, rather than being surprised when they are permanently removed in the release after that.

--terminated-pod-gc-threshold int32     Default: 12500

Number of terminated pods that can exist before the terminated pod garbage collector starts deleting terminated pods. If <= 0, the terminated pod garbage collector is disabled.

--tls-cert-file string

File containing the default x509 Certificate for HTTPS. (CA cert, if any, concatenated after server cert). If HTTPS serving is enabled, and --tls-cert-file and --tls-private-key-file are not provided, a self-signed certificate and key are generated for the public address and saved to the directory specified by --cert-dir.

--tls-cipher-suites strings

Comma-separated list of cipher suites for the server. If omitted, the default Go cipher suites will be used.
Preferred values: TLS_AES_128_GCM_SHA256, TLS_AES_256_GCM_SHA384, TLS_CHACHA20_POLY1305_SHA256, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305, TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305, TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256, TLS_RSA_WITH_AES_128_CBC_SHA, TLS_RSA_WITH_AES_128_GCM_SHA256, TLS_RSA_WITH_AES_256_CBC_SHA, TLS_RSA_WITH_AES_256_GCM_SHA384.
Insecure values: TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_ECDSA_WITH_RC4_128_SHA, TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_RSA_WITH_RC4_128_SHA, TLS_RSA_WITH_3DES_EDE_CBC_SHA, TLS_RSA_WITH_AES_128_CBC_SHA256, TLS_RSA_WITH_RC4_128_SHA.

--tls-min-version string

Minimum TLS version supported. Possible values: VersionTLS10, VersionTLS11, VersionTLS12, VersionTLS13

--tls-private-key-file string

File containing the default x509 private key matching --tls-cert-file.

--tls-sni-cert-key string

A pair of x509 certificate and private key file paths, optionally suffixed with a list of domain patterns which are fully qualified domain names, possibly with prefixed wildcard segments. The domain patterns also allow IP addresses, but IPs should only be used if the apiserver has visibility to the IP address requested by a client. If no domain patterns are provided, the names of the certificate are extracted. Non-wildcard matches trump over wildcard matches, explicit domain patterns trump over extracted names. For multiple key/certificate pairs, use the --tls-sni-cert-key multiple times. Examples: "example.crt,example.key" or "foo.crt,foo.key:*.foo.com,foo.com".

--unhealthy-zone-threshold float     Default: 0.55

Fraction of Nodes in a zone which needs to be not Ready (minimum 3) for zone to be treated as unhealthy.

--use-service-account-credentials

If true, use individual service account credentials for each controller.

-v, --v int

number for the log level verbosity

--version version[=true]

--version, --version=raw prints version information and quits; --version=vX.Y.Z... sets the reported version

--vmodule pattern=N,...

comma-separated list of pattern=N settings for file-filtered logging (only works for text log format)

6 - kube-proxy

Synopsis

The Kubernetes network proxy runs on each node. This reflects services as defined in the Kubernetes API on each node and can do simple TCP, UDP, and SCTP stream forwarding or round robin TCP, UDP, and SCTP forwarding across a set of backends. Service cluster IPs and ports are currently found through Docker-links-compatible environment variables specifying ports opened by the service proxy. There is an optional addon that provides cluster DNS for these cluster IPs. The user must create a service with the apiserver API to configure the proxy.

kube-proxy [flags]

Options

--add_dir_header

If true, adds the file directory to the header of the log messages

--alsologtostderr

log to standard error as well as files (no effect when -logtostderr=true)

--bind-address string     Default: 0.0.0.0

Overrides kube-proxy's idea of what its node's primary IP is. Note that the name is a historical artifact, and kube-proxy does not actually bind any sockets to this IP. This parameter is ignored if a config file is specified by --config.

--bind-address-hard-fail

If true kube-proxy will treat failure to bind to a port as fatal and exit

--boot_id_file string     Default: "/proc/sys/kernel/random/boot_id"

Comma-separated list of files to check for boot-id. Use the first one that exists.

--cleanup

If true cleanup iptables and ipvs rules and exit.

--cluster-cidr string

The CIDR range of the pods in the cluster. (For dual-stack clusters, this can be a comma-separated dual-stack pair of CIDR ranges.). When --detect-local-mode is set to ClusterCIDR, kube-proxy will consider traffic to be local if its source IP is in this range. (Otherwise it is not used.) This parameter is ignored if a config file is specified by --config.

--config string

The path to the configuration file.

--config-sync-period duration     Default: 15m0s

How often configuration from the apiserver is refreshed. Must be greater than 0.

--conntrack-max-per-core int32     Default: 32768

Maximum number of NAT connections to track per CPU core (0 to leave the limit as-is and ignore conntrack-min).

--conntrack-min int32     Default: 131072

Minimum number of conntrack entries to allocate, regardless of conntrack-max-per-core (set conntrack-max-per-core=0 to leave the limit as-is).

--conntrack-tcp-be-liberal

Enable liberal mode for tracking TCP packets by setting nf_conntrack_tcp_be_liberal to 1

--conntrack-tcp-timeout-close-wait duration     Default: 1h0m0s

NAT timeout for TCP connections in the CLOSE_WAIT state

--conntrack-tcp-timeout-established duration     Default: 24h0m0s

Idle timeout for established TCP connections (0 to leave as-is)

--conntrack-udp-timeout duration

Idle timeout for UNREPLIED UDP connections (0 to leave as-is)

--conntrack-udp-timeout-stream duration

Idle timeout for ASSURED UDP connections (0 to leave as-is)

--detect-local-mode LocalMode

Mode to use to detect local traffic. This parameter is ignored if a config file is specified by --config.

--feature-gates <comma-separated 'key=True|False' pairs>

A set of key=value pairs that describe feature gates for alpha/experimental features. Options are:
APIResponseCompression=true|false (BETA - default=true)
APIServerIdentity=true|false (BETA - default=true)
APIServerTracing=true|false (BETA - default=true)
AdmissionWebhookMatchConditions=true|false (BETA - default=true)
AggregatedDiscoveryEndpoint=true|false (BETA - default=true)
AllAlpha=true|false (ALPHA - default=false)
AllBeta=true|false (BETA - default=false)
AnyVolumeDataSource=true|false (BETA - default=true)
AppArmor=true|false (BETA - default=true)
CPUManagerPolicyAlphaOptions=true|false (ALPHA - default=false)
CPUManagerPolicyBetaOptions=true|false (BETA - default=true)
CPUManagerPolicyOptions=true|false (BETA - default=true)
CRDValidationRatcheting=true|false (ALPHA - default=false)
CSIMigrationPortworx=true|false (BETA - default=false)
CSIVolumeHealth=true|false (ALPHA - default=false)
CloudControllerManagerWebhook=true|false (ALPHA - default=false)
CloudDualStackNodeIPs=true|false (BETA - default=true)
ClusterTrustBundle=true|false (ALPHA - default=false)
ClusterTrustBundleProjection=true|false (ALPHA - default=false)
ComponentSLIs=true|false (BETA - default=true)
ConsistentListFromCache=true|false (ALPHA - default=false)
ContainerCheckpoint=true|false (ALPHA - default=false)
ContextualLogging=true|false (ALPHA - default=false)
CronJobsScheduledAnnotation=true|false (BETA - default=true)
CrossNamespaceVolumeDataSource=true|false (ALPHA - default=false)
CustomCPUCFSQuotaPeriod=true|false (ALPHA - default=false)
DevicePluginCDIDevices=true|false (BETA - default=true)
DisableCloudProviders=true|false (BETA - default=true)
DisableKubeletCloudCredentialProviders=true|false (BETA - default=true)
DisableNodeKubeProxyVersion=true|false (ALPHA - default=false)
DynamicResourceAllocation=true|false (ALPHA - default=false)
ElasticIndexedJob=true|false (BETA - default=true)
EventedPLEG=true|false (BETA - default=false)
GracefulNodeShutdown=true|false (BETA - default=true)
GracefulNodeShutdownBasedOnPodPriority=true|false (BETA - default=true)
HPAContainerMetrics=true|false (BETA - default=true)
HPAScaleToZero=true|false (ALPHA - default=false)
HonorPVReclaimPolicy=true|false (ALPHA - default=false)
ImageMaximumGCAge=true|false (ALPHA - default=false)
InPlacePodVerticalScaling=true|false (ALPHA - default=false)
InTreePluginAWSUnregister=true|false (ALPHA - default=false)
InTreePluginAzureDiskUnregister=true|false (ALPHA - default=false)
InTreePluginAzureFileUnregister=true|false (ALPHA - default=false)
InTreePluginGCEUnregister=true|false (ALPHA - default=false)
InTreePluginOpenStackUnregister=true|false (ALPHA - default=false)
InTreePluginPortworxUnregister=true|false (ALPHA - default=false)
InTreePluginvSphereUnregister=true|false (ALPHA - default=false)
JobBackoffLimitPerIndex=true|false (BETA - default=true)
JobPodFailurePolicy=true|false (BETA - default=true)
JobPodReplacementPolicy=true|false (BETA - default=true)
KubeProxyDrainingTerminatingNodes=true|false (ALPHA - default=false)
KubeletCgroupDriverFromCRI=true|false (ALPHA - default=false)
KubeletInUserNamespace=true|false (ALPHA - default=false)
KubeletPodResourcesDynamicResources=true|false (ALPHA - default=false)
KubeletPodResourcesGet=true|false (ALPHA - default=false)
KubeletSeparateDiskGC=true|false (ALPHA - default=false)
KubeletTracing=true|false (BETA - default=true)
LegacyServiceAccountTokenCleanUp=true|false (BETA - default=true)
LoadBalancerIPMode=true|false (ALPHA - default=false)
LocalStorageCapacityIsolationFSQuotaMonitoring=true|false (ALPHA - default=false)
LogarithmicScaleDown=true|false (BETA - default=true)
LoggingAlphaOptions=true|false (ALPHA - default=false)
LoggingBetaOptions=true|false (BETA - default=true)
MatchLabelKeysInPodAffinity=true|false (ALPHA - default=false)
MatchLabelKeysInPodTopologySpread=true|false (BETA - default=true)
MaxUnavailableStatefulSet=true|false (ALPHA - default=false)
MemoryManager=true|false (BETA - default=true)
MemoryQoS=true|false (ALPHA - default=false)
MinDomainsInPodTopologySpread=true|false (BETA - default=true)
MultiCIDRServiceAllocator=true|false (ALPHA - default=false)
NFTablesProxyMode=true|false (ALPHA - default=false)
NewVolumeManagerReconstruction=true|false (BETA - default=true)
NodeInclusionPolicyInPodTopologySpread=true|false (BETA - default=true)
NodeLogQuery=true|false (ALPHA - default=false)
NodeSwap=true|false (BETA - default=false)
OpenAPIEnums=true|false (BETA - default=true)
PDBUnhealthyPodEvictionPolicy=true|false (BETA - default=true)
PersistentVolumeLastPhaseTransitionTime=true|false (BETA - default=true)
PodAndContainerStatsFromCRI=true|false (ALPHA - default=false)
PodDeletionCost=true|false (BETA - default=true)
PodDisruptionConditions=true|false (BETA - default=true)
PodHostIPs=true|false (BETA - default=true)
PodIndexLabel=true|false (BETA - default=true)
PodLifecycleSleepAction=true|false (ALPHA - default=false)
PodReadyToStartContainersCondition=true|false (BETA - default=true)
PodSchedulingReadiness=true|false (BETA - default=true)
ProcMountType=true|false (ALPHA - default=false)
QOSReserved=true|false (ALPHA - default=false)
RecoverVolumeExpansionFailure=true|false (ALPHA - default=false)
RotateKubeletServerCertificate=true|false (BETA - default=true)
RuntimeClassInImageCriApi=true|false (ALPHA - default=false)
SELinuxMountReadWriteOncePod=true|false (BETA - default=true)
SchedulerQueueingHints=true|false (BETA - default=false)
SecurityContextDeny=true|false (ALPHA - default=false)
SeparateTaintEvictionController=true|false (BETA - default=true)
ServiceAccountTokenJTI=true|false (ALPHA - default=false)
ServiceAccountTokenNodeBinding=true|false (ALPHA - default=false)
ServiceAccountTokenNodeBindingValidation=true|false (ALPHA - default=false)
ServiceAccountTokenPodNodeInfo=true|false (ALPHA - default=false)
SidecarContainers=true|false (BETA - default=true)
SizeMemoryBackedVolumes=true|false (BETA - default=true)
StableLoadBalancerNodeSet=true|false (BETA - default=true)
StatefulSetAutoDeletePVC=true|false (BETA - default=true)
StatefulSetStartOrdinal=true|false (BETA - default=true)
StorageVersionAPI=true|false (ALPHA - default=false)
StorageVersionHash=true|false (BETA - default=true)
StructuredAuthenticationConfiguration=true|false (ALPHA - default=false)
StructuredAuthorizationConfiguration=true|false (ALPHA - default=false)
TopologyAwareHints=true|false (BETA - default=true)
TopologyManagerPolicyAlphaOptions=true|false (ALPHA - default=false)
TopologyManagerPolicyBetaOptions=true|false (BETA - default=true)
TopologyManagerPolicyOptions=true|false (BETA - default=true)
TranslateStreamCloseWebsocketRequests=true|false (ALPHA - default=false)
UnauthenticatedHTTP2DOSMitigation=true|false (BETA - default=true)
UnknownVersionInteroperabilityProxy=true|false (ALPHA - default=false)
UserNamespacesPodSecurityStandards=true|false (ALPHA - default=false)
UserNamespacesSupport=true|false (ALPHA - default=false)
ValidatingAdmissionPolicy=true|false (BETA - default=false)
VolumeAttributesClass=true|false (ALPHA - default=false)
VolumeCapacityPriority=true|false (ALPHA - default=false)
WatchList=true|false (ALPHA - default=false)
WinDSR=true|false (ALPHA - default=false)
WinOverlay=true|false (BETA - default=true)
WindowsHostNetwork=true|false (ALPHA - default=true)
ZeroLimitedNominalConcurrencyShares=true|false (BETA - default=false)
This parameter is ignored if a config file is specified by --config.

--healthz-bind-address ipport     Default: 0.0.0.0:10256

The IP address and port for the health check server to serve on, defaulting to "0.0.0.0:10256" (if --bind-address is unset or IPv4), or "[::]:10256" (if --bind-address is IPv6). Set empty to disable. This parameter is ignored if a config file is specified by --config.

-h, --help

help for kube-proxy

--hostname-override string

If non-empty, will be used as the name of the Node that kube-proxy is running on. If unset, the node name is assumed to be the same as the node's hostname.

--init-only

If true, perform any initialization steps that must be done with full root privileges, and then exit. After doing this, you can run kube-proxy again with only the CAP_NET_ADMIN capability.

--iptables-localhost-nodeports     Default: true

If false, kube-proxy will disable the legacy behavior of allowing NodePort services to be accessed via localhost. (Applies only to iptables mode and IPv4; localhost NodePorts are never allowed with other proxy modes or with IPv6.)

--iptables-masquerade-bit int32     Default: 14

If using the iptables or ipvs proxy mode, the bit of the fwmark space to mark packets requiring SNAT with. Must be within the range [0, 31].

--iptables-min-sync-period duration     Default: 1s

The minimum period between iptables rule resyncs (e.g. '5s', '1m', '2h22m'). A value of 0 means every Service or EndpointSlice change will result in an immediate iptables resync.

--iptables-sync-period duration     Default: 30s

An interval (e.g. '5s', '1m', '2h22m') indicating how frequently various re-synchronizing and cleanup operations are performed. Must be greater than 0.

--ipvs-exclude-cidrs strings

A comma-separated list of CIDRs which the ipvs proxier should not touch when cleaning up IPVS rules.

--ipvs-min-sync-period duration

The minimum period between IPVS rule resyncs (e.g. '5s', '1m', '2h22m'). A value of 0 means every Service or EndpointSlice change will result in an immediate IPVS resync.

--ipvs-scheduler string

The ipvs scheduler type when proxy mode is ipvs

--ipvs-strict-arp

Enable strict ARP by setting arp_ignore to 1 and arp_announce to 2

--ipvs-sync-period duration     Default: 30s

An interval (e.g. '5s', '1m', '2h22m') indicating how frequently various re-synchronizing and cleanup operations are performed. Must be greater than 0.

--ipvs-tcp-timeout duration

The timeout for idle IPVS TCP connections, 0 to leave as-is. (e.g. '5s', '1m', '2h22m').

--ipvs-tcpfin-timeout duration

The timeout for IPVS TCP connections after receiving a FIN packet, 0 to leave as-is. (e.g. '5s', '1m', '2h22m').

--ipvs-udp-timeout duration

The timeout for IPVS UDP packets, 0 to leave as-is. (e.g. '5s', '1m', '2h22m').

--kube-api-burst int32     Default: 10

Burst to use while talking with kubernetes apiserver

--kube-api-content-type string     Default: "application/vnd.kubernetes.protobuf"

Content type of requests sent to apiserver.

--kube-api-qps float     Default: 5

QPS to use while talking with kubernetes apiserver

--kubeconfig string

Path to kubeconfig file with authorization information (the master location can be overridden by the master flag).

--log-flush-frequency duration     Default: 5s

Maximum number of seconds between log flushes

--log_backtrace_at <a string in the form 'file:N'>     Default: :0

when logging hits line file:N, emit a stack trace

--log_dir string

If non-empty, write log files in this directory (no effect when -logtostderr=true)

--log_file string

If non-empty, use this log file (no effect when -logtostderr=true)

--log_file_max_size uint     Default: 1800

Defines the maximum size a log file can grow to (no effect when -logtostderr=true). Unit is megabytes. If the value is 0, the maximum file size is unlimited.

--logging-format string     Default: "text"

Sets the log format. Permitted formats: "text".

--logtostderr     Default: true

log to standard error instead of files

--machine_id_file string     Default: "/etc/machine-id,/var/lib/dbus/machine-id"

Comma-separated list of files to check for machine-id. Use the first one that exists.

--masquerade-all

If using the iptables or ipvs proxy mode, SNAT all traffic sent via Service cluster IPs. This may be required with some CNI plugins.

--master string

The address of the Kubernetes API server (overrides any value in kubeconfig)

--metrics-bind-address ipport     Default: 127.0.0.1:10249

The IP address and port for the metrics server to serve on, defaulting to "127.0.0.1:10249" (if --bind-address is unset or IPv4), or "[::1]:10249" (if --bind-address is IPv6). (Set to "0.0.0.0:10249" / "[::]:10249" to bind on all interfaces.) Set empty to disable. This parameter is ignored if a config file is specified by --config.

--nodeport-addresses strings

A list of CIDR ranges that contain valid node IPs. If set, connections to NodePort services will only be accepted on node IPs in one of the indicated ranges. If unset, NodePort connections will be accepted on all local IPs. This parameter is ignored if a config file is specified by --config.

--one_output

If true, only write logs to their native severity level (vs also writing to each lower severity level; no effect when -logtostderr=true)

--oom-score-adj int32     Default: -999

The oom-score-adj value for kube-proxy process. Values must be within the range [-1000, 1000]. This parameter is ignored if a config file is specified by --config.

--pod-bridge-interface string

A bridge interface name. When --detect-local-mode is set to BridgeInterface, kube-proxy will consider traffic to be local if it originates from this bridge.

--pod-interface-name-prefix string

An interface name prefix. When --detect-local-mode is set to InterfaceNamePrefix, kube-proxy will consider traffic to be local if it originates from any interface whose name begins with this prefix.

--profiling

If true enables profiling via web interface on /debug/pprof handler. This parameter is ignored if a config file is specified by --config.

--proxy-mode ProxyMode

Which proxy mode to use: on Linux this can be 'iptables' (default) or 'ipvs'. On Windows the only supported value is 'kernelspace'.This parameter is ignored if a config file is specified by --config.

--show-hidden-metrics-for-version string

The previous version for which you want to show hidden metrics. Only the previous minor version is meaningful, other values will not be allowed. The format is <major>.<minor>, e.g.: '1.16'. The purpose of this format is make sure you have the opportunity to notice if the next release hides additional metrics, rather than being surprised when they are permanently removed in the release after that. This parameter is ignored if a config file is specified by --config.

--skip_headers

If true, avoid header prefixes in the log messages

--skip_log_headers

If true, avoid headers when opening log files (no effect when -logtostderr=true)

--stderrthreshold int     Default: 2

logs at or above this threshold go to stderr when writing to files and stderr (no effect when -logtostderr=true or -alsologtostderr=true)

-v, --v int

number for the log level verbosity

--version version[=true]

--version, --version=raw prints version information and quits; --version=vX.Y.Z... sets the reported version

--vmodule pattern=N,...

comma-separated list of pattern=N settings for file-filtered logging (only works for text log format)

--write-config-to string

If set, write the default configuration values to this file and exit.

7 - kube-scheduler

Synopsis

The Kubernetes scheduler is a control plane process which assigns Pods to Nodes. The scheduler determines which Nodes are valid placements for each Pod in the scheduling queue according to constraints and available resources. The scheduler then ranks each valid Node and binds the Pod to a suitable Node. Multiple different schedulers may be used within a cluster; kube-scheduler is the reference implementation. See scheduling for more information about scheduling and the kube-scheduler component.

kube-scheduler [flags]

Options

--allow-metric-labels stringToString     Default: []

The map from metric-label to value allow-list of this label. The key's format is <MetricName>,<LabelName>. The value's format is <allowed_value>,<allowed_value>...e.g. metric1,label1='v1,v2,v3', metric1,label2='v1,v2,v3' metric2,label1='v1,v2,v3'.

--allow-metric-labels-manifest string

The path to the manifest file that contains the allow-list mapping. The format of the file is the same as the flag --allow-metric-labels. Note that the flag --allow-metric-labels will override the manifest file.

--authentication-kubeconfig string

kubeconfig file pointing at the 'core' kubernetes server with enough rights to create tokenreviews.authentication.k8s.io. This is optional. If empty, all token requests are considered to be anonymous and no client CA is looked up in the cluster.

--authentication-skip-lookup

If false, the authentication-kubeconfig will be used to lookup missing authentication configuration from the cluster.

--authentication-token-webhook-cache-ttl duration     Default: 10s

The duration to cache responses from the webhook token authenticator.

--authentication-tolerate-lookup-failure     Default: true

If true, failures to look up missing authentication configuration from the cluster are not considered fatal. Note that this can result in authentication that treats all requests as anonymous.

--authorization-always-allow-paths strings     Default: "/healthz,/readyz,/livez"

A list of HTTP paths to skip during authorization, i.e. these are authorized without contacting the 'core' kubernetes server.

--authorization-kubeconfig string

kubeconfig file pointing at the 'core' kubernetes server with enough rights to create subjectaccessreviews.authorization.k8s.io. This is optional. If empty, all requests not skipped by authorization are forbidden.

--authorization-webhook-cache-authorized-ttl duration     Default: 10s

The duration to cache 'authorized' responses from the webhook authorizer.

--authorization-webhook-cache-unauthorized-ttl duration     Default: 10s

The duration to cache 'unauthorized' responses from the webhook authorizer.

--azure-container-registry-config string

Path to the file containing Azure container registry configuration information.

--bind-address string     Default: 0.0.0.0

The IP address on which to listen for the --secure-port port. The associated interface(s) must be reachable by the rest of the cluster, and by CLI/web clients. If blank or an unspecified address (0.0.0.0 or ::), all interfaces and IP address families will be used.

--cert-dir string

The directory where the TLS certs are located. If --tls-cert-file and --tls-private-key-file are provided, this flag will be ignored.

--client-ca-file string

If set, any request presenting a client certificate signed by one of the authorities in the client-ca-file is authenticated with an identity corresponding to the CommonName of the client certificate.

--config string

The path to the configuration file.

--contention-profiling     Default: true

DEPRECATED: enable block profiling, if profiling is enabled. This parameter is ignored if a config file is specified in --config.

--disabled-metrics strings

This flag provides an escape hatch for misbehaving metrics. You must provide the fully qualified metric name in order to disable it. Disclaimer: disabling metrics is higher in precedence than showing hidden metrics.

--feature-gates <comma-separated 'key=True|False' pairs>

A set of key=value pairs that describe feature gates for alpha/experimental features. Options are:
APIResponseCompression=true|false (BETA - default=true)
APIServerIdentity=true|false (BETA - default=true)
APIServerTracing=true|false (BETA - default=true)
AdmissionWebhookMatchConditions=true|false (BETA - default=true)
AggregatedDiscoveryEndpoint=true|false (BETA - default=true)
AllAlpha=true|false (ALPHA - default=false)
AllBeta=true|false (BETA - default=false)
AnyVolumeDataSource=true|false (BETA - default=true)
AppArmor=true|false (BETA - default=true)
CPUManagerPolicyAlphaOptions=true|false (ALPHA - default=false)
CPUManagerPolicyBetaOptions=true|false (BETA - default=true)
CPUManagerPolicyOptions=true|false (BETA - default=true)
CRDValidationRatcheting=true|false (ALPHA - default=false)
CSIMigrationPortworx=true|false (BETA - default=false)
CSIVolumeHealth=true|false (ALPHA - default=false)
CloudControllerManagerWebhook=true|false (ALPHA - default=false)
CloudDualStackNodeIPs=true|false (BETA - default=true)
ClusterTrustBundle=true|false (ALPHA - default=false)
ClusterTrustBundleProjection=true|false (ALPHA - default=false)
ComponentSLIs=true|false (BETA - default=true)
ConsistentListFromCache=true|false (ALPHA - default=false)
ContainerCheckpoint=true|false (ALPHA - default=false)
ContextualLogging=true|false (ALPHA - default=false)
CronJobsScheduledAnnotation=true|false (BETA - default=true)
CrossNamespaceVolumeDataSource=true|false (ALPHA - default=false)
CustomCPUCFSQuotaPeriod=true|false (ALPHA - default=false)
DevicePluginCDIDevices=true|false (BETA - default=true)
DisableCloudProviders=true|false (BETA - default=true)
DisableKubeletCloudCredentialProviders=true|false (BETA - default=true)
DisableNodeKubeProxyVersion=true|false (ALPHA - default=false)
DynamicResourceAllocation=true|false (ALPHA - default=false)
ElasticIndexedJob=true|false (BETA - default=true)
EventedPLEG=true|false (BETA - default=false)
GracefulNodeShutdown=true|false (BETA - default=true)
GracefulNodeShutdownBasedOnPodPriority=true|false (BETA - default=true)
HPAContainerMetrics=true|false (BETA - default=true)
HPAScaleToZero=true|false (ALPHA - default=false)
HonorPVReclaimPolicy=true|false (ALPHA - default=false)
ImageMaximumGCAge=true|false (ALPHA - default=false)
InPlacePodVerticalScaling=true|false (ALPHA - default=false)
InTreePluginAWSUnregister=true|false (ALPHA - default=false)
InTreePluginAzureDiskUnregister=true|false (ALPHA - default=false)
InTreePluginAzureFileUnregister=true|false (ALPHA - default=false)
InTreePluginGCEUnregister=true|false (ALPHA - default=false)
InTreePluginOpenStackUnregister=true|false (ALPHA - default=false)
InTreePluginPortworxUnregister=true|false (ALPHA - default=false)
InTreePluginvSphereUnregister=true|false (ALPHA - default=false)
JobBackoffLimitPerIndex=true|false (BETA - default=true)
JobPodFailurePolicy=true|false (BETA - default=true)
JobPodReplacementPolicy=true|false (BETA - default=true)
KubeProxyDrainingTerminatingNodes=true|false (ALPHA - default=false)
KubeletCgroupDriverFromCRI=true|false (ALPHA - default=false)
KubeletInUserNamespace=true|false (ALPHA - default=false)
KubeletPodResourcesDynamicResources=true|false (ALPHA - default=false)
KubeletPodResourcesGet=true|false (ALPHA - default=false)
KubeletSeparateDiskGC=true|false (ALPHA - default=false)
KubeletTracing=true|false (BETA - default=true)
LegacyServiceAccountTokenCleanUp=true|false (BETA - default=true)
LoadBalancerIPMode=true|false (ALPHA - default=false)
LocalStorageCapacityIsolationFSQuotaMonitoring=true|false (ALPHA - default=false)
LogarithmicScaleDown=true|false (BETA - default=true)
LoggingAlphaOptions=true|false (ALPHA - default=false)
LoggingBetaOptions=true|false (BETA - default=true)
MatchLabelKeysInPodAffinity=true|false (ALPHA - default=false)
MatchLabelKeysInPodTopologySpread=true|false (BETA - default=true)
MaxUnavailableStatefulSet=true|false (ALPHA - default=false)
MemoryManager=true|false (BETA - default=true)
MemoryQoS=true|false (ALPHA - default=false)
MinDomainsInPodTopologySpread=true|false (BETA - default=true)
MultiCIDRServiceAllocator=true|false (ALPHA - default=false)
NFTablesProxyMode=true|false (ALPHA - default=false)
NewVolumeManagerReconstruction=true|false (BETA - default=true)
NodeInclusionPolicyInPodTopologySpread=true|false (BETA - default=true)
NodeLogQuery=true|false (ALPHA - default=false)
NodeSwap=true|false (BETA - default=false)
OpenAPIEnums=true|false (BETA - default=true)
PDBUnhealthyPodEvictionPolicy=true|false (BETA - default=true)
PersistentVolumeLastPhaseTransitionTime=true|false (BETA - default=true)
PodAndContainerStatsFromCRI=true|false (ALPHA - default=false)
PodDeletionCost=true|false (BETA - default=true)
PodDisruptionConditions=true|false (BETA - default=true)
PodHostIPs=true|false (BETA - default=true)
PodIndexLabel=true|false (BETA - default=true)
PodLifecycleSleepAction=true|false (ALPHA - default=false)
PodReadyToStartContainersCondition=true|false (BETA - default=true)
PodSchedulingReadiness=true|false (BETA - default=true)
ProcMountType=true|false (ALPHA - default=false)
QOSReserved=true|false (ALPHA - default=false)
RecoverVolumeExpansionFailure=true|false (ALPHA - default=false)
RotateKubeletServerCertificate=true|false (BETA - default=true)
RuntimeClassInImageCriApi=true|false (ALPHA - default=false)
SELinuxMountReadWriteOncePod=true|false (BETA - default=true)
SchedulerQueueingHints=true|false (BETA - default=false)
SecurityContextDeny=true|false (ALPHA - default=false)
SeparateTaintEvictionController=true|false (BETA - default=true)
ServiceAccountTokenJTI=true|false (ALPHA - default=false)
ServiceAccountTokenNodeBinding=true|false (ALPHA - default=false)
ServiceAccountTokenNodeBindingValidation=true|false (ALPHA - default=false)
ServiceAccountTokenPodNodeInfo=true|false (ALPHA - default=false)
SidecarContainers=true|false (BETA - default=true)
SizeMemoryBackedVolumes=true|false (BETA - default=true)
StableLoadBalancerNodeSet=true|false (BETA - default=true)
StatefulSetAutoDeletePVC=true|false (BETA - default=true)
StatefulSetStartOrdinal=true|false (BETA - default=true)
StorageVersionAPI=true|false (ALPHA - default=false)
StorageVersionHash=true|false (BETA - default=true)
StructuredAuthenticationConfiguration=true|false (ALPHA - default=false)
StructuredAuthorizationConfiguration=true|false (ALPHA - default=false)
TopologyAwareHints=true|false (BETA - default=true)
TopologyManagerPolicyAlphaOptions=true|false (ALPHA - default=false)
TopologyManagerPolicyBetaOptions=true|false (BETA - default=true)
TopologyManagerPolicyOptions=true|false (BETA - default=true)
TranslateStreamCloseWebsocketRequests=true|false (ALPHA - default=false)
UnauthenticatedHTTP2DOSMitigation=true|false (BETA - default=true)
UnknownVersionInteroperabilityProxy=true|false (ALPHA - default=false)
UserNamespacesPodSecurityStandards=true|false (ALPHA - default=false)
UserNamespacesSupport=true|false (ALPHA - default=false)
ValidatingAdmissionPolicy=true|false (BETA - default=false)
VolumeAttributesClass=true|false (ALPHA - default=false)
VolumeCapacityPriority=true|false (ALPHA - default=false)
WatchList=true|false (ALPHA - default=false)
WinDSR=true|false (ALPHA - default=false)
WinOverlay=true|false (BETA - default=true)
WindowsHostNetwork=true|false (ALPHA - default=true)
ZeroLimitedNominalConcurrencyShares=true|false (BETA - default=false)

-h, --help

help for kube-scheduler

--http2-max-streams-per-connection int

The limit that the server gives to clients for the maximum number of streams in an HTTP/2 connection. Zero means to use golang's default.

--kube-api-burst int32     Default: 100

DEPRECATED: burst to use while talking with kubernetes apiserver. This parameter is ignored if a config file is specified in --config.

--kube-api-content-type string     Default: "application/vnd.kubernetes.protobuf"

DEPRECATED: content type of requests sent to apiserver. This parameter is ignored if a config file is specified in --config.

--kube-api-qps float     Default: 50

DEPRECATED: QPS to use while talking with kubernetes apiserver. This parameter is ignored if a config file is specified in --config.

--kubeconfig string

DEPRECATED: path to kubeconfig file with authorization and master location information. This parameter is ignored if a config file is specified in --config.

--leader-elect     Default: true

Start a leader election client and gain leadership before executing the main loop. Enable this when running replicated components for high availability.

--leader-elect-lease-duration duration     Default: 15s

The duration that non-leader candidates will wait after observing a leadership renewal until attempting to acquire leadership of a led but unrenewed leader slot. This is effectively the maximum duration that a leader can be stopped before it is replaced by another candidate. This is only applicable if leader election is enabled.

--leader-elect-renew-deadline duration     Default: 10s

The interval between attempts by the acting master to renew a leadership slot before it stops leading. This must be less than the lease duration. This is only applicable if leader election is enabled.

--leader-elect-resource-lock string     Default: "leases"

The type of resource object that is used for locking during leader election. Supported options are 'leases', 'endpointsleases' and 'configmapsleases'.

--leader-elect-resource-name string     Default: "kube-scheduler"

The name of resource object that is used for locking during leader election.

--leader-elect-resource-namespace string     Default: "kube-system"

The namespace of resource object that is used for locking during leader election.

--leader-elect-retry-period duration     Default: 2s

The duration the clients should wait between attempting acquisition and renewal of a leadership. This is only applicable if leader election is enabled.

--log-flush-frequency duration     Default: 5s

Maximum number of seconds between log flushes

--logging-format string     Default: "text"

Sets the log format. Permitted formats: "text".

--master string

The address of the Kubernetes API server (overrides any value in kubeconfig)

--permit-address-sharing

If true, SO_REUSEADDR will be used when binding the port. This allows binding to wildcard IPs like 0.0.0.0 and specific IPs in parallel, and it avoids waiting for the kernel to release sockets in TIME_WAIT state. [default=false]

--permit-port-sharing

If true, SO_REUSEPORT will be used when binding the port, which allows more than one instance to bind on the same address and port. [default=false]

--pod-max-in-unschedulable-pods-duration duration     Default: 5m0s

DEPRECATED: the maximum time a pod can stay in unschedulablePods. If a pod stays in unschedulablePods for longer than this value, the pod will be moved from unschedulablePods to backoffQ or activeQ. This flag is deprecated and will be removed in 1.26

--profiling     Default: true

DEPRECATED: enable profiling via web interface host:port/debug/pprof/. This parameter is ignored if a config file is specified in --config.

--requestheader-allowed-names strings

List of client certificate common names to allow to provide usernames in headers specified by --requestheader-username-headers. If empty, any client certificate validated by the authorities in --requestheader-client-ca-file is allowed.

--requestheader-client-ca-file string

Root certificate bundle to use to verify client certificates on incoming requests before trusting usernames in headers specified by --requestheader-username-headers. WARNING: generally do not depend on authorization being already done for incoming requests.

--requestheader-extra-headers-prefix strings     Default: "x-remote-extra-"

List of request header prefixes to inspect. X-Remote-Extra- is suggested.

--requestheader-group-headers strings     Default: "x-remote-group"

List of request headers to inspect for groups. X-Remote-Group is suggested.

--requestheader-username-headers strings     Default: "x-remote-user"

List of request headers to inspect for usernames. X-Remote-User is common.

--secure-port int     Default: 10259

The port on which to serve HTTPS with authentication and authorization. If 0, don't serve HTTPS at all.

--show-hidden-metrics-for-version string

The previous version for which you want to show hidden metrics. Only the previous minor version is meaningful, other values will not be allowed. The format is <major>.<minor>, e.g.: '1.16'. The purpose of this format is make sure you have the opportunity to notice if the next release hides additional metrics, rather than being surprised when they are permanently removed in the release after that.

--tls-cert-file string

File containing the default x509 Certificate for HTTPS. (CA cert, if any, concatenated after server cert). If HTTPS serving is enabled, and --tls-cert-file and --tls-private-key-file are not provided, a self-signed certificate and key are generated for the public address and saved to the directory specified by --cert-dir.

--tls-cipher-suites strings

Comma-separated list of cipher suites for the server. If omitted, the default Go cipher suites will be used.
Preferred values: TLS_AES_128_GCM_SHA256, TLS_AES_256_GCM_SHA384, TLS_CHACHA20_POLY1305_SHA256, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305, TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305, TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256, TLS_RSA_WITH_AES_128_CBC_SHA, TLS_RSA_WITH_AES_128_GCM_SHA256, TLS_RSA_WITH_AES_256_CBC_SHA, TLS_RSA_WITH_AES_256_GCM_SHA384.
Insecure values: TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_ECDSA_WITH_RC4_128_SHA, TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_RSA_WITH_RC4_128_SHA, TLS_RSA_WITH_3DES_EDE_CBC_SHA, TLS_RSA_WITH_AES_128_CBC_SHA256, TLS_RSA_WITH_RC4_128_SHA.

--tls-min-version string

Minimum TLS version supported. Possible values: VersionTLS10, VersionTLS11, VersionTLS12, VersionTLS13

--tls-private-key-file string

File containing the default x509 private key matching --tls-cert-file.

--tls-sni-cert-key string

A pair of x509 certificate and private key file paths, optionally suffixed with a list of domain patterns which are fully qualified domain names, possibly with prefixed wildcard segments. The domain patterns also allow IP addresses, but IPs should only be used if the apiserver has visibility to the IP address requested by a client. If no domain patterns are provided, the names of the certificate are extracted. Non-wildcard matches trump over wildcard matches, explicit domain patterns trump over extracted names. For multiple key/certificate pairs, use the --tls-sni-cert-key multiple times. Examples: "example.crt,example.key" or "foo.crt,foo.key:*.foo.com,foo.com".

-v, --v int

number for the log level verbosity

--version version[=true]

--version, --version=raw prints version information and quits; --version=vX.Y.Z... sets the reported version

--vmodule pattern=N,...

comma-separated list of pattern=N settings for file-filtered logging (only works for text log format)

--write-config-to string

If set, write the configuration values to this file and exit.