The area marked in a red frame is the old city. The blue area is the fountain square, Nizami street and surrounding streets where you'll see only old and architectural buildings of late 19th - early 20th centuries. The boulevard (green frame) and Maiden Tower (yellow frame) are marked for orientation only. I can't recommend which hotel exactly to choose in this area, because as a local I never had to stay in any of them. Booking.com has "reviews" option which may help you make the right decision.
Route of the bus number 65 around Old City walls:
Route marked in red; blue dots are the bus stops. Trip fair for all buses in downtown is 0.20 AZN. 
Here they accept more than just regular (more than just USD, EUR, GBP) currencies. Blue dot is the location.
It's in a 5 minute walking distance from fountain square. You will see a "Romashka" sign on it. It's the red dot on the map.
Find the red dots in case you need to buy medicine or pills.
In case you get into trouble or something, find the blue dot.
The staring point is Hyatt Hotel nearby Bakhikanov bridge. From there you can take a cab (5-10 AZN at most for the whole ride back to downtown) or the bus number 95 (makes a small turn between Inshaatchilar and H.Javid avenues / you'll waste about 5 minutes). If you decide to walk, the whole path will take about 30 minutes to reach the end of Parliament avenue (mountainous park). If you do take a walk, you can return back to downtown from the ending point of Parliament avenue via bus number 95 or 18 (bus stop mark - yellow dot) or take a funicular ride in the mountainous park (yellow dot again / funicular ride is still free). If you decide to walk from there to downtown, just walk downwards by the road after a turn to the left (you'll see a stone plated road soon). It leads right to the downtown (15 minute walk distance).
