This may seem a silly little problem but it confused me for a while and may bother you one day. In the good old days if you wanted to delete a file you just found it in the folder and either hit the Delete key or found Delete on a right click menu. That seemed to work for just about everything as I recall. In the course of tidying up some files today, though, I couldn't get rid of any of the PDFs.
There's no way they were being 'used' in the normal sense of the word and Trying again didn't make any difference.
As with just about any problem these days, you'll find the answer on a forum somewhere on the net! The reason it wouldn't delete was because I had a Preview pane showing the document and that, technically in Windows 8 at any rate, means the document was 'in use'. Odd. I would have thought I would have encountered that before and can only assume that there is a different bit of code being utilised to display the document which interferes with the delete command whereas before it didn't.
So, all you have to do is change the display in the View menu to Details pane instead of Preview pane. Then you can delete to your heart's content.