The cells of our bodies need energy to work. They get this energy from glucose. Our bodies keep an energy reserve by binding together a bunch of glucose molecules into a storage form called glycogen. Glycogen is stored in the liver as well as in muscle, playing an integral role in exercise capacity. 

In order to be stored though, glycogen needs water. Therefore, when we deprive our bodies of carbohydrates, we break down and deplete these glycogen storages, which causes us to excrete water and prevent glycogen-related water retention.

So, does the number on the scale go down after this? Absolutely. Often precipitously so. BUT, this is generally neither a sustainable nor a substantive way to lose weight. In fact, it can be a pretty good way to burn up and catabolize hard-earned, lean muscle and other body tissues. Why? Because without access to the energy-providing glucose we get from ingested and stored carbohydrates, the body will actually start converting protein, which is essential for muscle maintenance, into the energy-rich glucose we so lack.