The same handful of steps, every time a new little machine shows up. Writing them down stopped me from skipping the boring ones.
Log in with a key, then turn password logins off and confirm a second session still gets in before closing the first. Update packages. Turn on the firewall with only the ports you actually use.
Certificate renewal. Make sure the renewal method does not fight the web server for a port, and that failures are visible somewhere instead of piped to nowhere.
Disk and traffic. A small counter for monthly traffic and an occasional look at free disk space prevent most surprises on cheap plans.
A way back in. Know where the provider's web console is before you need it.