1) Outgassing
Gas dissolved in rock (magma) within the Earth is released to the atmosphere when the magma rises as lava flows on the surface. The gas escapes the rock due to the decrease in pressure above the Earth surface.

2) Accretion of Planetesimals rich in Volatiles
Recall the inhomogeneous model for Earth's formation? In the final stages of formation, planetesimals made primarily of the lighter elements (called volatiles because they readily vaporize at low temperatures) deposited much if not most of our water, carbon dioxide, etc.

Evolution of the Earth's Atmosphere
As the earth surface cooled and volcanism slowed down, more and more of the atmosphere was condensed into the warm oceans. Water, Carbon Dioxide and Nitrogen were the most abundant gases (NO Oxygen gas!), Later nearly all the water and CO₂ will reside in the oceans, leaving a mostly Nitrogen rich atmosphere (like we have today).

The Greenhouse Effect and the Carbon Cycle
Atmospheric CO₂ traps thermal radiation (heat) coming from the Earth's surface, keeping it from escaping to cold, empty space. Our Earth has an important buffer cycle, the Carbon Cycle, which keeps the greenhouse effect and thus the temperature of Earth in check:

If Earth gets too cool..
Cool oceans absorb less CO₂. It precipitates out as carbonate sediments to the ocean floor. The carbon is released back into the atmosphere during volcanism. Increased atmospheric CO₂ increases and strengthens the greenhouse effect, bringing the temperature back up to normal.

If Earth gets too warm...
Warm oceans more readily absorb atmospheric CO₂. If you remove CO₂ from the atmosphere, the Earth will cool due to its reduced `greenhouse' effect, bringing the temperature back down. In this way, the oceans and volcanism kept the Earth at a very similar temperature. This allowed for liquid water over the last 4 billion years.