GPUs are really good at doing math. The Achilles heel is when it comes to 64-bit double precision math. GPUs, at least consumer grade, are not built for high performance FP64. This is because they are targetted towards gamers and game developers, who do not really care about high precision compute. So vendors like NVIDIA and AMD do not cram FP64 compute cores in their GPUs.
For example, on a GTX 780, the FP64 performance is 1/24 FP32. Which means in an ideal case, running the same code by only changing float types to double types, would yield the double precision run time to be about 1/24th of the single precision time.

How double precision performs really depends on the architecture of the GPU. Lets take 3 almost identical cards: GTX 780 Ti, GTX Titan Black and the Tesla K40c. All are <a href="http://www.nvidia.com/content/PDF/kepler/NVIDIA-Kepler-GK110-Architecture-Whitepaper.pdf" target="_blank">GK110</a> based GPUs, with the same number of SMX and cores (15 SMX, 2880 cores) and the same bus width (384-bit). The 780 Ti and Titan Black even have nearly same base clock speeds (~880MHz; K40c is 745MHz) and identical memory clock speeds (7GHz; K40 is 6GHz).
With respect to single precision performance, all 3 are fairly in the same ball park. Which the 780 Ti and Titan Black sit just around 5.1 TFlops, owing to their similar clock speeds, the Tesla K40c drops in at 4.3 TFlops. You could give the Tesla a pass because it has a lower clock speed.

All 3 only vary significantly in 3 categories.

1. Memory
This doesn't affect performance very much (at least when the sizes fit in all GPUs). The price difference based on market prices of other GPUs with similar memory size variations should not be that big either.

2. Price
At launch, the GTX 780 Ti was priced at $699, $999 for the Titan Black and an estimated $5500 for the K40c. Well, that escalated quickly.
(Price Sources: <a href="http://www.anandtech.com/show/7492/the-geforce-gtx-780-ti-review" target="_blank">GTX 780 Ti</a> <a href="http://www.anandtech.com/show/7765/nvidias-geforce-gtx-titan-black-no-compromises-for-gaming-compute" target="_blank">GTX Titan Black</a> <a href="http://www.anandtech.com/show/7521/nvidia-launches-tesla-k40" target="_blank">K40</a>)

3. Double Precision Performance
So if the 780 Ti and the Titan Black are practically the same in every respect, why is there a $300 difference in their price at launch?
The 780 Ti is physically locked at 1:24 FP32.
For the Titan Black, the magic happens in the driver. The Titan black's driver gives the user an option to choose the double precision performance between 1:3 and 1:24 FP32. When the double precision performance is set to 1:24 FP32, which is the same as the 780 Ti, the the single precision performance between the Titan Black and 780 Ti are identical. But when the user sets the double precision performance to 1:3 FP32, the single precision performance is compromised to boost double precision performance and make it equal to the K40c.
The K40c has a double precision performance of 1:3 without compromising the single precision performance. This is because the K40 is given a special double precision unit for every 3 single precision cores. It combines the best of both worlds.

<iframe width="600" height="371" seamless frameborder="0" scrolling="no" src="https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=381229711&amp;format=interactive"></iframe>
<iframe width="600" height="371" seamless frameborder="0" scrolling="no" src="https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=722963661&amp;format=interactive"></iframe>
<iframe width="600" height="371" seamless frameborder="0" scrolling="no" src="https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=1083990145&amp;format=interactive"></iframe>
<iframe width="600" height="371" seamless frameborder="0" scrolling="no" src="https://docs.google.com/spreadsheets/d/1gEjX5751zdYWom0_JG1FFLhM17l0OyWbUgWK90of59Q/pubchart?oid=1438814030&amp;format=interactive"></iframe>