A year after embarking on a merger, textbook publishers McGraw-Hill and Cengage have called off the deal because they couldn’t overcome regulatory hurdles.