But he didn't bring up the Crusades so it's all good

Many of his comments were lightly delivered and unlikely to elicit much controversy, though the reaction might be different if they were given by another world leader.
Case in point: Toward the beginning of his address, Francis alluded to religious extremism, noting that "no religion is immune" from it. His full quote as prepared for delivery: 

"Our world is increasingly a place of violent conflict, hatred and brutal atrocities, committed even in the name of God and of religion. We know that no religion is immune from forms of individual delusion or ideological extremism. This means that we must be especially attentive to every type of fundamentalism, whether religious or of any other kind. A delicate balance is required to combat violence perpetrated in the name of a religion, an ideology or an economic system, while also safeguarding religious freedom, intellectual freedom and individual freedoms." 

The remarks echo those delivered – albeit with more explicit historical references – by President Barack Obama at the National Prayer Breakfast in February. 

"Lest we get on our high horse and think this is unique to some other place, remember that during the Crusades and the Inquisition, people committed terrible deeds in the name of Christ. In our home country, slavery and Jim Crow all too often was justified in the name of Christ … So this is not unique to one group or one religion. There is a tendency in us, a sinful tendency that can pervert and distort our faith." 

Obama's comments were criticized as "offensive," "wrongheaded" and detached from present-day reality. Of course, it's more politically acceptable to criticize the president then the pope – time will tell if Francis' remarks draw a similar response.