Philippe Coutinho scored a brilliant first goal of the season and Jamie Vardy had a penalty saved as Liverpool survived a Leicester fightback to end their run of four games without a win.
The Foxes won 2-0 in their Carabao Cup third-round meeting on Tuesday, but the Reds exacted revenge on their return to Premier League action.
Coutinho set up Mohamed Salah for the opener and then struck a majestic free-kick to give his side a two-goal lead at King Power Stadium.
But Shinji Okazaki poked home from close range just before the break to give Leicester hope, before a frantic five-minute spell in the second half saw the balance of the game fluctuate dramatically.
Jordan Henderson netted from a Liverpool counter attack to make it 3-1 and seemingly secure the points but the hosts responded a minute later when Vardy headed into an empty net after Simon Mignolet pushed Demarai Gray's shot into his path.
Vardy then earned a penalty when he was brought down by Mignolet, but the Belgian kept out the England striker's spot-kick, which went straight down the middle and Liverpool had few further scares to hold on for victory.
Jurgen Klopp's side moved up to fifth in the table, while Leicester hover above the relegation zone with a win and a draw from their opening seven matches.
Philippe Coutinho, on his second Premier League start of the season, showed the key part he still has to play for Klopp's side.
The former Inter Milan playmaker's deep cross from the edge of the box was perfectly flighted for summer signing Salah to head in at the far post with Foxes keeper Kasper Schmeichel caught out by the Egypt winger's neat finish.
In contrast, Schmeichel had no chance of stopping the Brazilian's free-kick, which bent beautifully around the wall and into the corner of the net.
It looked like Coutinho's magnificent first-half performance would be the talking point, but as ever with a Liverpool team that struggles to defend with assurance, there was more drama to follow.
Prior to this match, Liverpool had lost 20 points from winning positions since the start of last season and it looked like Vardy was going to haunt them once again.
The England striker had scored two goals in each of his last two appearances against the Reds, but he was denied an equaliser and another double by a smart penalty save from Mignolet, who had conceded the penalty despite making contact with the ball before he clattered into the home striker.
Liverpool boss Klopp has faced plenty of criticism over their recent form - which has seen them draw two and lose two in all competitions since the emphatic win over Arsenal - and he celebrated with trademark gusto at the final whistle after picking up all three points.
"I really think in a moment like this so many things are positive," he said. "The three points are the proof for the public that we are still here, everything is good. Let's make the next step together."
This story has been reproduced from today's media. It does not necessarily represent the position of Liverpool Football Club.