Ivory is an Ancient Falmer ,who, much like Gelebor was a knight paladin who had a small group of survivors in Solsthiem. Having never agreed with the sacking of Sarthaal way back then, and survived a battle with a band of humans on this attack, he saw the ferocity of their inner strength to survive and grow. He along with around fifty paladins veterans after that attack went into hiding, fearing with worst of the Nord's vengeance and the Dwemer's unflattering cruelty. He was not the one leading the group, However once the leader was killed by an arrow from one of their own who wanted to go to the Dwemer for help, he took charge to punish the betrayer by stripping him of his Magicka reserves for life. Something he never figured out how to do again. As a respect to his fallen brother who called Solsthiem his home he was dedicated to staying there.
As time passed, their numbers dwindled either by Nature, Nords, or even other races who enticed those who lost hope. Ivory's motivation was fueled more and more, remembering those yellow eyed Nords who he fought so valiantly against him. The passion for their people as they were slaughtered by his own fueled him to discover that humans, despite everything were a being to respect even if young. This didn't stop him from growing hatred for them as his people dwindled into nothing as centuries passed. over the Eras he's survived to the point of being alone, he watched as time passed him by, watched as the world continued on without his people.
All of his time was dedicated to doing anything to survive, to preserve himself and keep his people alive even if he was the last remaining Falmer. While he is not immortal like a Vampire, he has Mastered Restoration beyond that expected of anyone, even an Elf. He refused to die, refused to be taken by times hand. He didn't spend his time just sitting in a dark cave all his life however. He studied humans, Nords, Dark Elves, High elves, anyone that traveled to Solsthiem. He learned the ways of the Voice, but didn't except using the power of the Voice to worship the Gods. Instead, taking it as the God's gift to him to help him survive longer. He decided to Travel to Skyrim, go back to his home and find out what it became for himself. He stayed in the shadows of the northern snows to survive. The Nords didn't take kindly to him, and it showed even when getting off the boat.
After avoiding the conflicts of Tamriel up to this point, and feeling the sadness of there being no trace of his people at all, he grew to love this New Skyrim as he calls it and the people in it. One night, he was ambushed by werewolves and having easily survived them before, he managed to hold them off before being cornered by three of them. It was just enough, having taken damage to his left eye in the scuffle and a painful bite to his right shoulder, he saw those yellow eyes on the two humans as the werewolves change back to themselves and one elf. In a blind rage to survive, he once more as he did so many Eras ago, slaughtered the trio. Again these Golden eyed Nords were almost his death. After a week or so of feeling weak and sick, he merged in the den of the dead werewolves as one himself. After a lot of strife, he worked to gain control over himself but failed again and again, wanting to find a cure that just wasn't there. He heard of Hircine's ring and made it his biggest pursuit. It wouldn't be till his infamy as the White Werewolf nearly one hundred years later that he would find Sinding and help him against the Hunters and be rewarded with Hircine's Ring. He came to settle in Riverwood, peaceful, and the nords there (save for Gerdur and her family) didn't mind his presence as long as he worked. Soon, he heard of the rumors of a Dragonborn while he was out of town and saw Alduin himself overhead. As he returned to town, he bumped into a tan female Nord with white hair like his and noticed her eyes. She was wearing nothing but the usual mercenary outfit for the time and after his reluctance to help this woman get through Bleakfalls Barrow, he went along with her. Never realizing that she was not only the Dragonborn, but the woman he was going to love. Not to say the hardships he would endure as he saw what has become not only of his people, but the people of Skyrim and the Empire in the future.