28 different countries don’t have any hassle getting a visa for Kyrgyzstan. See the government website for details or go to http://www.kyrgyzstan.org
Just go to any embassy, fill in the forms and pay your money. You can generally get the visa in 24 hours.
If there is no embassy you can go to a Kazakhstan embassy but you will then need a letter of invitation available from Stan Tours amongst many others.
Renewal is easy and can be done any time to add 30 days on to your visa.
The same 28 nationalities can get visa on arrival.
You must have the visa when you arrive. There is no Visa on Arrival (Yet!)
China: There are two routes into China from Kyrgyzstan, the easiest is the Irkeshtan Pass near Osh (275km away), the other crossing point at the Torugart Pass (3752m), near Naryn involves paying to be driven across the border and you must arrange all this beforehand. The main problem is the visa, best to already have it from your home country…as for the last two years it has been very difficult to get Chinese visas in Kyrgyzstan or anywhere in Central Asia.
Tajikistan: you will need a Tajik visa and a GBAO permit to cross at the only border allowed for foreigners (near Sary Tash). This is easy to get in Bishkek at ITMC or Stan Tours can also help. You can travel with a faxed version and pick up the original in Khorog or other towns in Tajikistan.
Kazakhstan: LOI needed
Uzbekistan: LOI needed
See the individual country pages for more info