This answer also pertains to Windows 7 (All versions) as well.
Fast Answer, AVG is only getting rid of them, BUT doesn't get rid of backup file where Windows 8/8.1 sends them too. And when you restart Windows they reappear. Best bet, go into Microsoft Windows Help and Support, delete all saved restore points going back to when the problem first started, and then turn off Backup and Restore, rerun AVG restart the your PC, turn back on Backup & Restore and do a new restore point. That will solve your problem. ALSO if you are running a 3rd party backup software those backups need to be deleted as well. BUT I would suggest is completely removing that backup program, and after everything is back in order re-install it.
This answer also pertains to Windows 7 (All versions) as well.