If by "Open the activity and delete it there" you mean to use the Remove Data Series, I tried that and failed.
Here's a bogus entry:

- before.png (221.86 KiB) Viewed 335 times
'
Here's what I see in edit:

- edit.png (43.31 KiB) Viewed 335 times
And what I see after saving:

- after.png (262.57 KiB) Viewed 335 times
And yes, I reloaded the page after completing the edit.
I expected to find some sort of "Delete activity" checkbox in the edit screen, but can't seem to find it. My only work around has been to use my phone to access the site (instead of on my computer as usual). On it, I'm able to to move through the popup menu on the dashboard in a way that doesn't seem possible in my browser.