Kubernetes Resource Quota
Quota in Kubernetes is very helpful in administrating the
deployments. We can decided these at Deployment level and
namespace level as well. We can also configure at Objective
level like Pods, RC or RS. Users can work in different name
space with different resource quota and can decide what can
be useful for app or DB servers. We have major two types of
Quota if we named it. Compute Quota and Objective Quota.
We can further divide Compute Quota at two levels.
Compute Quota:
Deployment level
Name Space level
Deployment level:
Let’s say that app deployment can use 400mi ram while Db can
use 200Mi of RAM.
spec:
containers:
- image: nginx
name: nginx
resources:
requests:
memory: 200Mi
cpu: 200m
limits:
memory: 300Mi
cpu: 300m
request means you have a guaranteed resources you have defined
while creating deployment. Limits meaning you can go to that
value defined in limits when no other containers are using
memory but it is not guaranteed.
Note: 1 vCPU = 1000m
Reservation Use is reserved resources a node has. You can check
with below command.
# kubectl describe node worker1 | grep -iA5 'allocated'
Allocated resources:
(Total limits may be over 100 percent, i.e., overcommitted.)
Resource Requests Limits
-------- -------- ------
cpu 250m (12%) 0 (0%)
memory 0 (0%) 0 (0%)
So 12% means it is already committed to some application and
you can use after that. And 0% memory means nothing is
committed for memory.
# kubectl describe node worker3 | grep -i cpu
cpu: 2
cpu: 2
Namespace Name CPU Requests CPU Limits Memory Requests Memory Limits Age
cpu 250m (12%) 0 (0%)
Actual use is when pods are running and using resources. Now I
have deployed a deployment with resource quota.
# cat varelite6.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
creationTimestamp: null
labels:
app: varelite6
name: varelite6
spec:
replicas: 2
selector:
matchLabels:
app: varelite6
strategy: {}
template:
metadata:
creationTimestamp: null
labels:
app: varelite6
spec:
containers:
- image: nginx
name: nginx
resources:
requests:
memory: 200Mi
cpu: 200m
limits:
memory: 300Mi
cpu: 300m
status: {}
# kubectl create -f varelite6.yaml
deployment.apps/varelite6 created
# kubectl get pods -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
varelite6-6545486c46-mj62w 1/1 Running 0 12m 172.16.189.98 worker2 <none> <none>
varelite6-6545486c46-zs9xb 1/1 Running 0 12m 172.16.235.166 worker1 <none> <none>
# kubectl describe node worker1 | grep -iA5 'Allocated'
Allocated resources:
(Total limits may be over 100 percent, i.e., overcommitted.)
Resource Requests Limits
-------- -------- ------
cpu 450m (22%) 300m (15%)
memory 200Mi (7%) 300Mi (10%)
Note: If you will not set any limit in above deployment then
limit will be the request value.
Namespace level:
In this case, we can assign resources to namespace and under
that app or db can use any resources which has been assigned
to that namespace.
Let’s create a namespace first and then we will assign the
Quota on it.
# kubectl create ns varelite
namespace/varelite created
# kubectl get ns
NAME STATUS AGE
default Active 70d
kube-node-lease Active 70d
kube-public Active 70d
kube-system Active 70d
varelite Active 74s
# kubectl create quota var --hard=cpu=400m,memory=400Mi -n varelite
resourcequota/var created
# kubectl get quota -n varelite
NAME AGE REQUEST LIMIT
var 50s cpu: 0/400m, memory: 0/400Mi
# kubectl describe quota var -n varelite
Name: var
Namespace: varelite
Resource Used Hard
-------- ---- ----
cpu 0 400m
memory 0 400Mi
Note: If you have set quota at namespace level then you must
define quota at deployment as well.
# kubectl run varelite7 --image=nginx -n varelite --dry-run -o yaml > varelite7.yaml
# cat varelite7.yaml
apiVersion: v1
kind: Pod
metadata:
creationTimestamp: null
labels:
run: varelite7
name: varelite7
namespace: varelite
spec:
containers:
- image: nginx
name: varelite7
resources:
requests:
cpu: 100m
memory: 100Mi
dnsPolicy: ClusterFirst
restartPolicy: Always
status: {}
# kubectl create -f varelite7.yaml
pod/varelite7 created
# kubectl get pods -n varelite
NAME READY STATUS RESTARTS AGE
varelite7 1/1 Running 0 10s
# kubectl get pods -n varelite
NAME READY STATUS RESTARTS AGE
varelite7 1/1 Running 0 10s
root@kb-master:~/deploys# kubectl describe quota var -n varelite
Name: var
Namespace: varelite
Resource Used Hard
-------- ---- ----
cpu 100m 400m
memory 100Mi 400Mi
Now if you will any new deployment then it will use namespace
quota till it is fully utilized after that you can’t do it.
You can also go ahead and modify it using below command.
# kubectl edit quota -n varelite
resourcequota/var edited
# kubectl describe quota var -n varelite
Name: var
Namespace: varelite
Resource Used Hard
-------- ---- ----
cpu 100m 500m
memory 100Mi 500Mi
You can delete quota as well if you don’t need it.
# kubectl delete quota var -n varelite
resourcequota "var" deleted
# kubectl get pods -n varelite
NAME READY STATUS RESTARTS AGE
varelite7 1/1 Running 0 32m
You can see now there will be no quota.
# kubectl get quota -n varelite
No resources found in varelite namespace.
Objective Quota: Pod/Deployment/RS/RC/Service/PVC etc. It is very rarely used. You can limit the number of Pod, Deployment or any other objects at namespace level. The command is similar to above one just you can add count of objects.
# kubectl create quota var --hard=cpu=400m,memory=400Mi,pods=4 -n varelite
resourcequota/var created
# kubectl get quota -n varelite
NAME AGE REQUEST LIMIT
var 10s cpu: 100m/400m, memory: 100Mi/400Mi, pods: ¼
# kubectl describe quota var -n varelite
Name: var
Namespace: varelite
Resource Used Hard
-------- ---- ----
cpu 100m 400m
memory 100Mi 400Mi
pods 1 4