If you're running geth on a stock Ubuntu install, you might be surprised that the system allocates swap memory when regular memory is available. To prevent this from happening you need to change the "swapiness" that the kernel uses to decide which kind of memory to allocate.
Follow the instructions in this answer, then restart geth
Concretely, do this
sudo swapoff --all sudo sysctl vm.swappiness=0 sudo swapon --all
Then start geth. You'll need to do this on every reboot.