not sure what you mean exactly by 'shrink', but I think the the answer is yes;
This is due to the fact that you are not using your muscles any more (or as heavily) so blood flow to the area (i.e. muscles) is decreased, consequently leading to smaller muscles (or the appearance of smaller muscles)/less vascularity
Location: Space Cadet City, ToughCookieVille, United Kingdom (Great Britain)
Stats: 5'1", 118 lbs
Rep Power: 53781
If your talking about the Pump you get disappearing then yes it does go away the reason for this is your blood stops being congested in that area.
"The Pump" is the tight, blood-congested feeling in a muscle after it has been intensely trained. Muscle pump is caused by a rapid influx of blood into the muscles to remove fatigue toxins and replace supplies of fuel and oxygen. A good muscle pump indicates that you have optimally worked a muscle group.