When SQL Server 2005 doesn't Cleanly Remove Log Shipping

There have been times when I've setup log shipping for several DBs in SQL Server 2005 then later removed the option, only to have Log Shipping Alert job raise errors regarding these no-longer-log-shipped DBs. To clean references to them, delete (CAREFULLY) records in the msdb DB that pertains to log shipping.