One of the sites I maintain recently had a server upgrade to Ubuntu 14.04.4, which included a new version of varnish (version 3.05) which required an update to the Varnish configuration. After setting up a new default.vcl file I was monitoring the headers sent and received to make sure that what was supposed to be cached was in fact being cached by Varnish. While testing I discovered that I was not getting any Cache-Control headers from static files like images.
Cache-Control defines how, and for how long the individual response can be cached by the browser and other intermediate caches, which allows intelligent caching decisions to be made, saving server resources and reducing page load time. If you use The Google developers page speed insights tool and get the following warning: "Leverage browser caching. Setting an expiry date or a maximum age in the HTTP headers for static resources instructs the browser to load previously downloaded resources from local disk rather than over the network" then you probably don't have cache-control headers set or they are too set too short.
In Drupal, if the page cache is enabled on the performance page (/admin/config/development/performance), then the web page's Cache-Control: Max-age is set by the value chosen in the "Expiration of cached pages" select list. For files the default value comes from a rule in the root .htaccess file, but this only works if Apache's Expires module (mod_expires) is enabled. Enable it, and your static assets will be able to be cached for 2 weeks (the default set in .htaccess). While you're about, it go ahead and enable the headers module too to enable the correct serving of compressed CSS and javascript files.
Out of interest, this is what Google recommends in terms of minimum cache time:
"We recommend a minimum cache time of one week and preferably up to one year for static assets, or assets that change infrequently. If you need precise control over when resources are invalidated we recommend using a URL fingerprinting or versioning technique - see invalidating and updating cached responses link above."