Saul is a Hungarian Jewish prisoner in Auschwitz near the end of the war. He works as a Sonderkommando, Jewish prisoners forced to staff the gas chambers, including disposing of the dead. Sonderkommando worked for months and were then killed and replaced. A young boy survives the gassing, and Saul watches as a camp physician suffocates him.
Saul is obsessed with providing the boy with a proper burial, and one of the doctors - a prisoner like Saul - helps him smuggle the body into his own quarters. He asks a fellow Sonderkommando who is also a rabbi to help with the burial, but he tells Saul that he can do nothing but say the Kaddish prayer. Saul says that's not good enough and searches for another rabbi. The other Sonderkommando know they'll be killed soon and are planning an uprising; they involve Saul in their plans, but he cares for nothing but burying the boy's body and loses a package of gunpowder entrusted to him. When asked he tells them the child was his son, but the others don't believe him.
The Nazis speed up the extermination process beyond what they can handle as the Soviet army closes in on the camp. Saul searches for a rabbi in a group of new prisoners and finds a bearded man who lets Saul identify him as a rabbi, and Saul rescues him from death. The Sonderkommando rebel and Saul and his companion escape, Saul carrying the boy's corpse. He tries to bury him in the woods, but his companion is not a rabbi and can't say the Kaddish. The other Sonderkommando arrive, fleeing pursuing Nazi soldiers, and they join them. While swimming across a river, Saul begins to drown because the corpse he's carrying weighs him down, and the real rabbi saves him. Saul loses his grip and watches helplessly as the boy's body floats down the river.
The escaped prisoners stop to rest in an abandoned building. They discuss joining the approaching Soviet army; Saul says nothing. A local boy, the same ago as the boy Saul claimed as his son, appears. Saul sees him and smiles. The boy leaves just as the Nazi soldiers find the building. He runs away as the Nazi soldiers shoot the escaped prisoners.