Al-Shaatibi wrote:

The difference between the hardship that is not normally considered hardship is whether or not the practice, if done on a continual basis, would lead to its being discontinued, totally or partially, or lead to some harm or defect in the person, his wealth or his circumstances. If [any of the later occur], then it would be considered a type of hardship that falls outside of the normally accepted hardship.  If none of those latter matters usually occurs, it would not normally be considered hardship.

However, many people confuse wanting to perfect one’s worship with extremism, but this is not the case, as Ibn al-Munir says, as quoted in Fath al-Baari:

The meaning isn ot preventing the seeking of completeness in one’s worship.  [Completeness of one’s worship] is obviously a praiseworthy affair.  What is meant to be prevented is going to the extreme such that it leads one to boredom or such exaggeration in one’s voluntary deeds that he leaves what is more virtuous.