On Tuesday Digg announced that they were finally rolling out some major updates to their duplicate detection technology, and let’s be honest - it’s about time!  The technology they had in place before was hardly reliable.

The way that it used to work is that you’d go ahead and insert your URL and all the details for the story you were submitting, and then after you had done all that, Digg would ask you if you were sure it wasn’t a duplicate.  Along with that it would show you a list of stories that it thought could be similar to yours.  Many were not the slightest bit related, and some were submitted days, weeks, or even months ago.  Nevertheless, if you assured Digg that your link was in fact NOT a duplicate (even if it was), you could proceed and submit your link.

According to Digg, and Brent Csutoras, they’ve updated the way that their software looks for duplicates.  They said that most commonly the types of duplicate stories being submitted were the same stories from the same site, but with different URLs.  So solve this problem, they devised a solution tha