Build the trip around walking loops rather than a checklist. On the first day, take the upper town from the Citadelle along Dufferin Terrace past the Château Frontenac, drop through the funicular or the Breakneck Stairs, and come back up through Place Royale and Petit Champlain. That is one loop, and it takes most of a day if you stop for lunch.
Then do it again after dark, when the shops are lit, and the tour groups have gone.
Upper Town or Lower Town for Sleeping
Upper Town puts you beside the terrace, the museums and the Plains of Abraham. Lower Town puts you in the stone lanes with dinner downstairs and the river at the end of the street. I would sleep low for a short trip and high for a longer one, and stay inside the walls either way, because a hotel on the highway edge turns every evening into a taxi ride.
Three Days, Five Days, or a Week
How many days do you actually need? Three covers the old town properly, adds Montmorency Falls and leaves one evening free. Five is the comfortable version: the falls and the island on separate days, a museum afternoon, a slow morning on the Plains of Abraham. A week means Montreal as well.
When to Come
Winter is the version people underestimate. The Plains of Abraham turns into ski terrain, the Ice Hotel runs from January to March, and the old town is quiet enough to hear your own boots. It is also genuinely cold, and you will spend more on a coat than on coffee.
Summer brings the river cruises and the island with every farm stand open, along with everyone else. Autumn is the best time to visit Quebec: cool mornings, color along the St. Lawrence, a food scene moving into its heavy season.

What I would tell a friend is simple. Book two nights inside the walls, walk the loop twice, eat in the lower town, give the falls one morning. The city does the rest.