Did you clean your cache after making the changes and before viewing the refreshed page? That is the only reason I can think of that the (now-deleted) old image is still showing for you.
I know it's a pain to have to save what you are working on (instead of "apply") to leave to go clean your cache every time you make a change, so that is why I use this handy (FREE) little Joomla! addon:
Cache Cleaner. It saves me loads of time when I'm making changes to either an article or perhaps tweaking the css files... it shows up in the admin area as a link (to the right of "logout") and all you have to do is click the link and it doesn't take you off of the page you are working on.
You might need to clear your browser cache too... and don't forget that in FF, if you hit Ctrl+F5, it reloads the page and clears the cache for that page without logging you out!
Hope that helps!