There many factors that can affect the loading speed of your site, below are some important tips to help you fix or prevent having that problem on your website and its server.

• Declutter – Remove all the old files and unnecessary files. Get rid of spam, unused plugins, theme, and any other files you’re not using anymore.
• Update – After removing all the unnecessary files, updating the plugins, themes, and website itself is a MUST. It is to ensure your site’s security, efficiency, and stability.
• Cache – Install a plugin to help define what files should and should not be cached so your site’s content is delivered faster to returning visitors.
• Upload – Upload your videos, music, or pictures to services designed for that purpose instead of storing the media on your own server. E.g. Upload your videos to Youtube, Vimeo instead of hosting in on your own server.
• Compress – For audio, use compressed files such as MP3s. For images, compress them as well as specify their dimensions.

Once you are done doing all the items above but still have an unaccepatable loading speed issue, consider contacting your –

• Web Developer / Tech Guy : Your Developer can check if you have corrupted/damaged files that needs to be replaced or some codes that needs to rewrite due to quality issue.
• Web Hosting Provider : There might be a significant increase on your site traffic, if so, you might need to upgrade your hosting package. On the other hand, there might be just some technical glitches with the web server(power outage, virus attack, etc) that causing a loading issues. If you think this is too technical for you,  ask your developer to do this on your behalf.
• Computer Technician : Lastly, have your workstation and internet connection check by your technician, to make sure that the issues are not coming from your end.

Source : http://view.e.hostingmessages.com/?qs=cf072e6fb179ff3578e25129c85800c0482ea40cfc2ac4047071b50ea536126c10178dcb0c14d0469a259a7f7a789e00246c4791210f6cde94dd8a529b163ff236e8054de5340db3