Mahrez gave City the lead with a fine free kick in the 53rd minute of Wednesday's third-round game, before Alvarez scored the second five minutes later at the Etihad Stadium.
Chelsea's impressive youngster Lewis Hall made two good saves from Stefan Ortega but City, looking to win the competition for the seventh time in 10 seasons, were good value for the win.
Another piece of good news for the home club - and potentially for England - was that midfielder Kalvin Phillips had a 40-minute run.
Phillips had been out since undergoing shoulder surgery in September and the game was his last chance to prove his fitness before England manager Gareth Southgate named his squad for the World Cup on Thursday.