Yes, once deleted, all files, themes, plugins, and the database will be removed permanently. Make sure to take a backup if needed.
