If a money services business is exempt from state money transmitter licensing, is FinCEN registration alone sufficient, or do they still need state licenses?

If you are an MSB that is exempt from MTLs - that exemption would apply on a state by state basis. So for the states you are exempt, you are exempt, but others may require you to obtain a license.

As far as FinCEN registration is concerned, if you are deemed to be an MSB, you are required to be registered with FinCEN.

FinCEN is not a license granting institution. Registration with FinCEN is mandatory for all MSBs for purposes of federal reporting.