When to resize an instance

Modified: 26 Jan 2023 22:12 UTC

Resizing an infrastructure container running SmartOS or Container Native Linux offers the ability to scale vertically by resizing the configuration of a running container, without having to shutdown or reboot.

What is resizing?

Resizing is the ability to increase or decrease the configuration of an infrastructure container running SmartOS, bhyve (Hardware Virtualized), or Container Native Linux, including disk size, cpu cap, and memory size. Currently, KVM (Hardware Virtualized) instances cannot be resized. For more information, see resizing KVM.

Bhyve (Hardware Virtualized) instances must use the flexible disk space feature (or be resized to a package that uses the flexible disk space feature) before they will allow resizing. Changes to memory or CPU in a bhyve instance will require a restart.

Please see managing instances for detailed instructions on how to resize.

Why resize an instance?

As capacity requirements change, considerations for growth and/or scaling back an infrastructure become important business decisions. Additionally, resizing an infrastructure container running SmartOS or Container Native Linux can offer an immediate solution to unforeseen resource exhaustion.

Resizing to meet business needs

As business needs or requirements change, the configuration requirements of an infrastructure container may also need changing. For example, you can increase the size of your instance to scale for growth without provisioning a new one.

Instance runs out of disk space

If an instance runs out of disk space, resizing to a larger package will be necessary.

If you are able to SSH in to the instance, you can check the disk space using the df(1m) command (example output below):

container# df -h
Filesystem                                  Size  Used Avail Use% Mounted on
zones/4985f9fb-e674-e472-ac02-86e15242d6f7   26G  448M   25G   2% /
/lib                                        263M  237M   27M  91% /lib
/lib/svc/manifest                           2.7T  673K  2.7T   1% /lib/svc/manifest
/lib/svc/manifest/site                       26G  448M   25G   2% /lib/svc/manifest/site
/sbin                                       263M  237M   27M  91% /sbin
/usr                                        417M  379M   39M  91% /usr
/usr/ccs                                     26G  448M   25G   2% /usr/ccs
/usr/local                                   26G  448M   25G   2% /usr/local
swap                                        256M   28M  229M  11% /etc/svc/volatile
/usr/lib/libc/libc_hwcap1.so.1              417M  379M   39M  91% /lib/libc.so.1
swap                                        128M  8.0K  128M   1% /tmp
swap                                        256M   28M  229M  11% /var/run

In general, the critical filesystem to look at is / (root). If you see it reaching towards 100%, you'll want to either start cleaning up the filesystem, or resize the instance to a larger package size. 

In some cases, you may not be able to ssh into the instance to due to a 100% full filesystem. You can often verify this by running the zfs(1m) command from the compute node where the instance lives:

computenode# zfs list zones/4985f9fb-e674-e472-ac02-86e15242d6f7
=== Output from 44454c4c-3700-1039-8034-c2c04f445131 (78-2b-cb-0a-75-23):
NAME                                         USED  AVAIL  REFER  MOUNTPOINT
zones/4985f9fb-e674-e472-ac02-86e15242d6f7  21.6M  25.0G   448M  /zones/4985f9fb-e674-e472-ac02-86e15242d6f7

If the AVAIL column shows 0, the instance has run out of disk space and will need to be resized in order to regain access.

Instance runs out of memory

Whether the instance runs out of memory or swap space, the instance is likely to become inaccessible (and services may start failing).

If you attempt to ssh into an instance that is suffering from memory exhaustion, you'll often see a message relating to Resource temporarily unavailable. A reboot of the instance may temporarily resolve the problem, but will require some detective work to determine the underlying cause. In many cases, the solution is to resize the instance to a larger package size, thus allowing programs to run with more than enough memory.

You can check the memory of a instance from its compute node by running zonememstat(1m):

computenode# zonememstat -z 4985f9fb-e674-e472-ac02-86e15242d6f7
=== Output from 44454c4c-3700-1039-8034-c2c04f445131 (78-2b-cb-0a-75-23):
                                 ZONE  RSS(MB)  CAP(MB)    NOVER  POUT(MB)
 4985f9fb-e674-e472-ac02-86e15242d6f7       28      128        0         0

If you see the RSS column reaching it's CAP, and you're also seeing stats under NOVER (provides the number of times the instance has gone over its cap) and POUT (total amount of memory paged out when the zone has gone over its cap), the instance is likely suffering from memory exhaustion. Again, it's important to understand why the instance is running out of memory before concluding that a resize is the best solution.

Please also see analyzing zone memory utilization for more information on monitoring memory usage.

Considerations for resizing

Some considerations that must be taken into account prior to resizing an instance include:

Resizing a KVM instance

Resizing is not supported for KVM Hardware Virtualized instances. In place of resizing a KVM instance, you can provision a new instance on a new package, and then migrate the data from the original to new instance.

What do you want to do next?