They really do exactly what their title says. They take an idea for something, a character, a location, an enemy, a weapon, etc. and sketch it out, color it, and just make it look good in many different positions.
They take what the game designers write down, and serve as a mediator between the designers and the 3d artists. They show the 3D artists exactly how they want that particular game asset to look like. Also, they communicate with other leads at the company to ensure that their design is what the designers wanted.
For a good reference of concept art, I suggest looking for some video game books, There is the art of Halo, The art of World of Warcraft,etc. there are many final fantasy books, and books made specifically by their artists, both very good.
a great example is from gears of war, Gears of Art
This is actual work, from basic words on a game document that this industry proffessional created. The 3D artists did a good job ( if you've played the game ) of keeping close to this art style and design.
Really, they give the first visual representations of their game. While many concept artists do help with some box art, posters, etc. their main job is doing the art that the rest of the game is based off of.