The Reds were pegged back in the second period as Victor Wanyama's fine long-distance strike cancelled out Mohamed Salah's third-minute opener.
But they looked to have won it at the death when, after Loris Karius had saved a Harry Kane penalty, Salah scored an incredible individual goal, weaving through the Spurs defence before finishing smartly.
However, Mauricio Pochettino's team were awarded another spot-kick in the third minute of injury time, and Kane made no mistake this time to ensure the score finished 2-2.
Reflecting on the game, Milner told the BBC: "We played well first half, second half they got on top a bit and it was hard for us to keep the ball.
"That was disappointing but I thought we defended pretty well as a team, they didn't have too many clear cut chances and they're a very good side.
"It was disappointing that we didn't control the game a bit more second half but we stuck to it and probably wish we'd done enough to get that extra goal to see us over the line but it wasn't to be."
Milner admitted the hosts found it difficult to continously execute a high press against opponents who ended the match having enjoyed 66 per cent of the possession.
But he was pleased with the way he and his teammates held their shape throughout, despite the end result.
He added: "They're a good side and move the ball quickly so it's impossible [to press] for 95 minutes, there are times when you have to get back into shape.
"But we do it week in, week out and against the sides who move the ball quickly it's very different.
"In a game where you have the majority of the ball, those reactions and the quick pressing you're not doing it too often.
"But obviously Spurs controlled the game a bit more second half and it wasn't as easy to get up to them, but first half I thought we played well so it was disappointing that we couldn't keep the ball a bit better."