This is mostly going to focus on Sans' part of the story. Sans is one of the saddest parts of the entire game.
First, I want to tackle the similar story of Sans and Monika from Doki Doki Literature Club. DDLC covers the horror of self awareness, the idea that the world you live in is controlled by some higher power and that the world you live in is not real or entirely insignificant. This is the kind of mind bending terror that Sans experiences every time he fights Chara. The idea that no matter what happens he and his friends will infinitely suffer no matter how hard he tries. This is also partially why Dusttale at least in my opinion, is entierely canon. No matter how strong a person's will is insanity prevails in these situations of hopelesness, sadness and fear. Sans keeps a cool exterior, making jokes, puns and generally having fun. This is only an outer mask however, as we see him give us a life threat on the "date". He clearly despises the human because of what he has seen Frisk or more accurately, Chara do. His face is also used for intimidation and concentration. Sans simply does not care for the human at best because he knows that no matter how many times Frisk does a pacifist route, he knows soon enough, another genocide run will happen. He only keeps going because he wants to see his friends smile and live in peace but does not care about himself as we see him walking back to the underground, waiting for the next genocide or neutral route. Sans suffers eternally from the horrible effects of reliving the same suffering, watching his friends die over and over and over. Even when he sees Frisk/Chara killing his brother, he refuses to kill the human on the spot due to his promise to Toriel and even in the end fight when all of his friends have died, attempts to warn Frisk about the suffering both will go through if she steps over the line. Even in that moment, he cares about Toriel and hence all his friends enough to risk his life to try to keep his promise. So at it's core Undertale is the story of Sans eternally seeing his friends die over and over. I feel AUs like Glitchtale and Dusttale to be canon because in the endless amount of repeats something different happens. Thus the creations of AUs. Outer tampering like Gaster would also shift timelines. The thought of endless possibility is what makes undertale an amzing game. A sad story is hidden behind it which is what engages the player and causes them to really empathise with sans or any of the monsters.
End of post.