So now, with v5.3 nothing will happen in SmartPlaylist when the MediaLibrary changes (for whatever reason). As long as you add new titles, everything is OK, you don't need to do anything in SmartPlaylist. But once you change some attributes (like lyrics, genre, composer, cover artwork, etc.) you need run "Empty cache after sync" to make sure that the new values are used in SmartPlaylist.
So whenever you think that SmartPlaylist does not take the latest values, it is time to "Empty cache after sync". On older devices with iOS 4, where apps don't run in the background and have to be restarted each time, this is not necessary, since like that SmartPlalyist is always started with an empty cache.
If you still don't see what you expect, then probably there is a problem with syncing/iCloud/iTunes Match.