Within your "Assignments" tab in a classroom, you will have an assignments list.
To see all of your assignments ever created for the given class, select the "All" bucket and ensure your filter is set to "Due: All-time"
In the list for each assignment, there will be an "Options" dropdown. Click the dropdown, and select "Delete Assignment." Deleting an assignment will delete all of that assignment's data and it will be removed from the assignment lists from both teachers and students.