Whenever a character does something they are meant to be doing (healing, fighting, killing, flinging, etc.) they will gain some XP every turn. If they kill a character, they get a bonus to their XP on that turn. Once they've reached enough XP they level up.
When a character levels up their health heals to full in the fight. Every level of a character grants them a higher % of their base health, damage, and defense. You can check to see the benefits of this if you have a high level character and a lower level character with the same type of equipment on.
Levels are different in PVP, temporarily. Every fighter is set to be the same level and you DO NOT gain any XP in Versus modes, so you cannot level up in Versus. (Standard loot for participating in PVP has been introduced as of Update 2. Local PVP will not yield rewards as always.)