Once upon a time, new software was of interest because it had new features or other improvements over previous versions. This attracted us to new versions, but we still wanted our old stuff to work - so the new versions would often retain old code to stay compatible with what we already had.
Today, we're not so much following the carrot of quality, but fleeing the stick of quality failure. We are often told we must get a new version because the old version was so badly made, it could be exploited to do all sorts of unwanted things. In this case, we want to break compatibility so that the old exploit techniques will no longer work!
Yet often the same vendors who drive us to "patch" or "upgrade" their products to avoid exploitation risks, still seem to think we are attracted by features, not driven by fear.
I've highlighted the long-standing problems with Sun's Java before, and they are still squirming around their promise to mend their ways. In short, they may still leave old exploitable versions of the Java JRE on your system, but it's no longer quite as easy for malware to select these as their preferred interpreter. Still, you're probably safer if you uninstall these old JREs (as Sun's Java updater typically does not do) than trust Sun to deny code access to them.
Microsoft's Side By Side
Here's an interesting article on the Windows SxS (Side By Side) facility, which aims to appease software that was written for older versions of system .DLLs and thus ease the pain of "DLL Hell". This works by retaining old versions of these .DLLs so that older software can specify access to them, via their manifest.
How is that different from Sun's accursed practice?
Well, is generally isn't, as far as I can tell, until a particular exploit situation is recognized where this behaviour poses a risk. The current crisis du jour involves exploits against GDIPlus.dll - yep the same one that was fixed before - and the patch this time includes a facility to block access to old versions of the .DLL, leveraging a feature already designed into the SxS subsystem.