I figured out the issue. The Serialization manager changes the environment directory to the folder where the .dspx file is located. Setting Environment.CurrentDirectory equal to something other will allow the folder to also be deleted.
I would still appreciate suggestions on how to correctly close a .dspx file.
I would still appreciate suggestions on how to correctly close a .dspx file.