CIS (Ukraine, Georgia, Kazakhstan) and Baltic States (Estonia, Latvia, Lithuania) visa regulations:

CIS and Baltic countries (Commonwealth of Independent States - former USSR) have different visa regulations from Russia. If you are traveling to Russia overland, you need to make sure you know whether you need a visa or not to avoid problems at the border. Below you will find the most up-to-date information about visa regulations in each country. However, it is always better to contact the consulate of the country you're traveling to directly to make sure the regulations are still the same.
 

Belarus:

Citizens of all countries, except CIS, require a visa to Belarus. The current regulations are outlined on the website of Belarus in USA and they are quite similar to the requirements for Russia.
In order to get a transit visa you need to present the visa to Russia and a copy of your tickets (not required in some cases). The transit visa costs $25 US and can be obtained at the consulate in 1 business day.
In order to get a visa valid for 1 month, you need visa support.
 

Georgia:

Citizens of the following countries can get a Georgian visa upon arrival to one the county's airports (Tbilisi or Batumi) for $10 US if the duration of the visit is not longer than 1 month: USA, Austria, Belgium, Germany, Denmark, GB, Spain, Estonia, Japan, Ireland, Israel, Italy, Canada, Cyprus, Latvia, Lithuania, Luxemburg, Hungary, Poland, Portugal, Greece, France, Slovakia, Finland, Czech Republic, Sweden, Switzerland, Holland.
If you are a citizen of any other country or want to travel for longer than 1 month, you need to apply for a visa to the Georgian consulate. No invitation papers are necessary, however, you need to specify the name of the inviting person or organization on your application form. The processing time in this case is 5 to 14 business days and the visa costs $10-$20 US.
No visa is required for cruise ships that stop in Georgian ports for less
than 72 hours.
If you need a transit visa the same rules as above apply.
More information can be found on the website of Georgian Consulate in USA
If you require Georgian visa, you can apply for a Georgian visa support online.

Ukraine:

Citizens of EU, Switzerland and Liechtenstein: starting 1 September 2005 visas are not required for EU or Swiss citizens who enter Ukraine or transit through its territory for the period of time of up to 90 days with a valid travel passport.
Citizens of USA and Canada: starting July 1, 2005 visas are not required for the US and Canadian citizens who enter Ukraine within six months after their previous trip to Ukraine or transit through its territory for a period of up to 90 days with a valid American travel passport.
The US citizens need to make sure that their current medical insurance has coverage in Ukraine.
Citizens of Japan: starting August 1, 2005 visas are not required for the Japanese citizens who enter Ukraine or transit through its territory for a period time of up to 90 days with a valid Japanese travel passport.
When visas are required for the US, Canadian, EU, Swiss, Japanese citizens: For the citizens who enter Ukraine with the purpose of employment, permanent residency, study and work at the diplomatic missions in Ukraine or with any other purpose if they are going to stay in Ukraine for more than 90 days, visas are required. In order to get a visa, you need visa support (invitation), which can be obtained online.

If you are a citizen of any other country or the rules above do not apply to your situation, then you need to apply for a Ukrainian visa at the consulate. No invitation letters are required for citizens of the US, Canada, Japan, Slovakia, and Turkey for obtaining official, business, private, cultural, or sports visas.
No visas are required for citizens of Hungary, Poland, Mongolia, Lithuania and the countries of the Commonwealth of Independent States (except Turkmenistan).
If you require a transit visa the same rules as above apply.
More information can be found on the website of Ukrainian Consulate in USA

Kazakhstan:

Citizens of EU, Switzerland, USA, Australia, Japan require a visa to Kazakhstan, however, no invitation is needed to get it if the period of the visit is not longer than 1 month. You should simply apply to the consulate in your country and fill in the application form to obtain the visa.
Citizens of CIS countries do not require a visa to Kazakhstan.
Citizens of all other countries require a visa to Kazakhstan and need an invitation in order to get it.
If you want to get a transit visa, you need to present a copy of the visa to the country you're traveling to (if it is required) and your tickets.
More information can be found on the website of Kazakh Embassy in USA
Invitations needed to get a Kazakh visa can be obtained online.