Jurgen Klopp's side were heading for a frustrating draw after wasting a string of chances when Mane prodded past Wayne Hennessey for his second league goal of the campaign.
Palace, beaten 3-0 at home to Huddersfield in their opening match, should have scored when the game was goalless.
Christian Benteke was unmarked and eight yards out but blazed over the bar from Ruben Loftus-Cheek's pass.
Welcome clean sheet for Reds
This was far from vintage Liverpool who, for once, struggled in front of goal despite peppering the Palace defence with 16 shots - half of them on target.
Joel Matip headed a great chance wide, while Mane missed a glorious opportunity before he scored - both chances created by debutant Andrew Robertson.
The full-back impressed going forward while helping Liverpool produce a much-needed clean sheet after the defensive horror show in the 3-3 draw at Watford last week.
Klopp made five changes to the team that started the Champions League play-off win at Hoffenheim in midweek and surprisingly left summer signing Mohamed Salah on the bench.
Daniel Sturridge produced flashes of his best form in Salah's place before the £34m signing came on to help his team - who were much better in the second half - to a hard-fought victory.
Mane's goal punctured the frustration which was building inside Anfield.
A one-two with substitute Dominic Solanke seemed to have been dealt with by Palace, only for the ball to roll kindly for the Senegal forward to poke past Hennessey.
Improvement from Palace
Benteke scored both goals in Palace's 2-1 win at Anfield in April and the former Liverpool striker will surely still be wondering how he failed to find the net in this game.
Matip did a fine marking job on Benteke throughout. However, Benteke was well placed to give Palace a 57th-minute lead after excellent work by Loftus-Cheek but somehow hammered the ball over the bar. It was a huge let-off for Liverpool.
Christian Benteke has scored one goal in his last seven Premier League games
Nevertheless, this was a much-improved performance by Frank de Boer's team after their Huddersfield defeat, and they were well organised and solid at the back in the first half.
They also lost their opening two games of last season before bouncing back to finish seven points above the relegation zone in 14th spot.
Man of the match - Andrew Robertson (Liverpool)
No wonder defender Andrew Robertson was smiling after his Premier League debut for Liverpool. He produced two wonderful crosses which should have resulted in goals, before being denied himself late on from long range.
'We defended brilliantly' - manager reaction
Liverpool manager Jurgen Klopp:
"It was difficult. We could have played a little clearer in the first half. We defended brilliantly and we had the ball all of the time - it felt like 85% possession. You need to make the right decisions and in the first half we didn't. We could have finished more often than we did, but had our moments.
"The second half was much better and I loved the goal we scored because it was not our best one-two in the game. It was a nice goal and we deserved it a lot.
"At 1-0 up we made the ball do the work. We could have closed the game earlier, we didn't. It was an all-inclusive package."
Crystal Palace manager Frank de Boer:
"We were quite concentrated for almost all of the match. I saw a team who wants to fight for each other. It could be a draw, a Liverpool win or the other way - with our two chances it could have been different.
"A lot of times they were frustrated because we played very organised.
"Today this was the best tactic to get the result, but the coin could have fallen the other way."
'Impressive debut' - the stats
- Robertson, who made his Premier League debut for the Reds, created more chances (three) and had more touches of the ball (134) than any other Liverpool player.
- De Boer is only the second Dutch manager to suffer defeat in his first two Premier League games after Martin Jol, who lost his opening three in the competition.
- Liverpool have kept five clean sheets in their past six Premier League games, as many as in the previous 21.
- Crystal Palace have lost six of their past seven games in the Premier League, failing to score in each defeat.
- The Reds kept a clean sheet against Crystal Palace for the first time in 14 competitive matches.
- Liverpool have lost just one of their past 14 Premier League matches - this fixture last season.